[QPKG] sherpa: a mini-package-manager (CLI) beta
- OneCD
- Guru
- Posts: 12155
- Joined: Sun Aug 21, 2016 10:48 am
- Location: "... there, behind that sofa!"
Re: [QPKG] sherpa mini-package-manager (CLI)
Excellent, good work guys!
Now, I just need to understand what's required so I can fix sherpa.
@zyxmon, I switched to installing the 'python3-cryptography' and 'python3-cffi' IPKGs, so I don't understand why PIP was still trying to compile CFFI on x19? It doesn't do that on my a64?
How should I fix this? What does the 'include' archive downloaded earlier do?
Now, I just need to understand what's required so I can fix sherpa.
@zyxmon, I switched to installing the 'python3-cryptography' and 'python3-cffi' IPKGs, so I don't understand why PIP was still trying to compile CFFI on x19? It doesn't do that on my a64?
How should I fix this? What does the 'include' archive downloaded earlier do?
-
- Experience counts
- Posts: 1001
- Joined: Mon Oct 27, 2014 2:27 pm
- Location: Moscow, Russia
- Contact:
Re: [QPKG] sherpa mini-package-manager (CLI)
I do not know how pip works and handles dependencies. One thing to note - sometimes there are missing symlinks to so - openwrt was designed to run binaries and not for native compilation.
The second one - headers were not installed for x19. Are they installed for other architectures? Headers are usually needed for compilation. Some headers are installed with gcc package.
The problems solved today are missing ffi.h header and missing libffi.so (symlink, not the library). Similar problems may occur for other architectures and pip install commands.
-
- Experience counts
- Posts: 1001
- Joined: Mon Oct 27, 2014 2:27 pm
- Location: Moscow, Russia
- Contact:
Re: [QPKG] sherpa mini-package-manager (CLI)
Code: Select all
pip install [some package] --no-deps
-
- Experience counts
- Posts: 1001
- Joined: Mon Oct 27, 2014 2:27 pm
- Location: Moscow, Russia
- Contact:
Re: [QPKG] sherpa mini-package-manager (CLI)
One more thing the new cryptography package needs rust that is missing.
- OneCD
- Guru
- Posts: 12155
- Joined: Sun Aug 21, 2016 10:48 am
- Location: "... there, behind that sofa!"
Re: [QPKG] sherpa mini-package-manager (CLI)
OK.
I'm not sure, I think I previously resolved the header issue by installing 'python3-dev' (which still occurs today). sherpa no-longer downloads anything other than IPKGs for Entware - so no direct files anymore. It used to do this when I was downgrading the Python version some time ago, but not any more.
I think this is going to require some experimentation to get right.
Ah, thank you. Looks like it will cause fewer problems if we stick with the cryptography IPKG module instead of installing it via 'pip'.
-
- Experience counts
- Posts: 1001
- Joined: Mon Oct 27, 2014 2:27 pm
- Location: Moscow, Russia
- Contact:
Re: [QPKG] sherpa mini-package-manager (CLI)
IMHO the problem is - pip ignores already installed packages for x19 reinstalls them and failes because of missing *.h or *.so files.
This may help -
It can be used for all architectures.
This may help -
Code: Select all
PIP_IGNORE_INSTALLED=0 pip install [some package]
- OneCD
- Guru
- Posts: 12155
- Joined: Sun Aug 21, 2016 10:48 am
- Location: "... there, behind that sofa!"
- OneCD
- Guru
- Posts: 12155
- Joined: Sun Aug 21, 2016 10:48 am
- Location: "... there, behind that sofa!"
Re: [QPKG] sherpa mini-package-manager (CLI)
Setup my "new" old ARMv5 NAS this morning and first thing I installed was SABnzbd via sherpa: no problems, no compilation errors - just my luck!
Anyway, based on @zyxmon's great suggestions, I'll start modifying sherpa so it will (hopefully) rely a lot less on PIP module compilation. At the very least, it should make for faster execution.
Anyway, based on @zyxmon's great suggestions, I'll start modifying sherpa so it will (hopefully) rely a lot less on PIP module compilation. At the very least, it should make for faster execution.
-
- Starting out
- Posts: 27
- Joined: Thu Mar 15, 2012 6:33 pm
Re: [QPKG] sherpa mini-package-manager (CLI)
I finally had time to try it out again. I just reinstalled ew, sabnzbd... and that worked . I don't know why it works now, but the only thing I did, I installed a update for my Qnap yesterday.OneCD wrote: ↑Thu Apr 08, 2021 9:09 amApologies: I've still a long way to go with regard-to debugging these Python module installs.
I can't make this error occur on my test NAS (TS-230 with ARMv8), no-matter what I do. I suspect it's a compiling error only occurring on ARMv5.
I wonder if installing the 'wheel' pip module is needed on your arch to get 'cffi' installed?
After a lot of installs/reinstalls and testing on my own NAS this morning, here's the process I'm hoping you'll be willing to run through to confirm this. There are 2 methods: if the first fails, please run the second.
Method 1Method 2
- The quick-and-dirty way, run this on your existing config:
Code: Select all
/opt/bin/pip3 install wheel cffi
- Remove the existing Entware installation. We're going to replace it with a vanilla Entware instead:
When Entware has finished installing, please logout your terminal session then back in again (to refresh $PATH).Code: Select all
sherpa remove ew cd /share/Public wget http://bin.entware.net/other/Entware_1.03std.qpkg sh Entware_1.03std.qpkg
Now check the 'pip' module list:
- Then run the following commands. These will install a basic Python module set, upgrade 'pip', then install 'wheel' (which I'm hoping will be the only required fix here) and 'apprise' (so 'cffi' gets pulled-in as a dependency):
Code: Select all
opkg install python3-pip python3-setuptools python3-dev /opt/bin/python3 -m pip install --upgrade pip /opt/bin/pip3 install wheel apprise
Code: Select all
/opt/bin/pip3 list
So I didn't try out the above commands... but thanks for all your help!
Code: Select all
sherpa reinstall ew
sherpa reinstall sabnzbd
-
- Know my way around
- Posts: 120
- Joined: Wed Apr 08, 2015 11:42 pm
Re: [QPKG] sherpa mini-package-manager (CLI)
Getting an error starting OMedusa using the latest firmware 4.5.3.1652
Code: Select all
>>>>>>>>>> datetime:'Thu Apr 29 23:54:22 BST 2021',request:'starting',QPKG:'210325',app:'0.5.11' <<<<<<<<<<
daemon: NOT active
file (/opt/bin/git): exists
update 'OMedusa' from remote repository: OK
current git branch: master
file (/opt/bin/python3): exists
load UI ports from application: OK
start daemon: failed!
= ExecuteAndLog()
output: "ffprobe: error while loading shared libraries: libx264.so.161: cannot open shared object file: No such file or directory
Traceback (most recent call last):
File "/share/CACHEDEV1_DATA/.qpkg/OMedusa/OMedusa/start.py", line 6, in <module>
from medusa.__main__ import main
File "/share/CACHEDEV1_DATA/.qpkg/OMedusa/OMedusa/medusa/__init__.py", line 24, in <module>
initialize()
File "/share/CACHEDEV1_DATA/.qpkg/OMedusa/OMedusa/medusa/init/__init__.py", line 39, in initialize
_configure_knowit()
File "/share/CACHEDEV1_DATA/.qpkg/OMedusa/OMedusa/medusa/init/__init__.py", line 240, in _configure_knowit
api.initialize({'mediainfo': suggested_path})
File "/share/CACHEDEV1_DATA/.qpkg/OMedusa/OMedusa/ext/knowit/api.py", line 35, in initialize
available_providers[name] = provider_cls(config, context.get(name) or config.general.get(name))
File "/share/CACHEDEV1_DATA/.qpkg/OMedusa/OMedusa/ext/knowit/providers/ffmpeg.py", line 218, in __init__
self.executor = FFmpegExecutor.get_executor_instance(suggested_path)
File "/share/CACHEDEV1_DATA/.qpkg/OMedusa/OMedusa/ext/knowit/providers/ffmpeg.py", line 108, in get_executor_instance
executor = exec_cls.create(os_family, suggested_path)
File "/share/CACHEDEV1_DATA/.qpkg/OMedusa/OMedusa/ext/knowit/providers/ffmpeg.py", line 131, in create
output = ensure_text(check_output([candidate, '-version']))
File "/opt/lib/python3.9/subprocess.py", line 424, in check_output
File "/opt/lib/python3.9/subprocess.py", line 528, in run
subprocess.CalledProcessError: Command '['ffprobe', '-version']' returned non-zero exit status 127."
result: [1]
TS-451
-
- Experience counts
- Posts: 1001
- Joined: Mon Oct 27, 2014 2:27 pm
- Location: Moscow, Russia
- Contact:
Re: [QPKG] sherpa mini-package-manager (CLI)
The problem is probably with libx264 package. Here are possible solutions - https://github.com/Entware/Entware/issues/644
- OneCD
- Guru
- Posts: 12155
- Joined: Sun Aug 21, 2016 10:48 am
- Location: "... there, behind that sofa!"
Re: [QPKG] sherpa mini-package-manager (CLI)
Thanks mate!zyxmon wrote: ↑Fri Apr 30, 2021 2:37 pm The problem is probably with libx264 package. Here are possible solutions - https://github.com/Entware/Entware/issues/644
- dirkonline
- Easy as a breeze
- Posts: 305
- Joined: Mon Oct 18, 2010 4:08 pm
Re: [QPKG] sherpa mini-package-manager (CLI)
With the latest QTS Update 4.5.3.165 , OMedusa no longer starts
Is that an issue for anybody else? Ideas? I wish they would stop breaking things all the time, sigh
Reiinstall did not fix the issue
Is that an issue for anybody else? Ideas? I wish they would stop breaking things all the time, sigh
Reiinstall did not fix the issue
Code: Select all
>>>>>>>>>> datetime:'Fri Apr 30 15:51:05 CEST 2021',request:'starting',QPKG:'210325',app:'0.5.11' <<<<<<<<<<
daemon: NOT active
file (/opt/bin/git): exists
update 'OMedusa' from remote repository: OK
current git branch: master
file (/opt/bin/python3): exists
load UI ports from application: OK
start daemon: failed!
= ExecuteAndLog()
output: "ffprobe: error while loading shared libraries: libx264.so.161: cannot open shared object file: No such file or directory
Traceback (most recent call last):
File "/share/CACHEDEV1_DATA/.qpkg/OMedusa/OMedusa/start.py", line 6, in <module>
from medusa.__main__ import main
File "/share/CACHEDEV1_DATA/.qpkg/OMedusa/OMedusa/medusa/__init__.py", line 24, in <module>
initialize()
File "/share/CACHEDEV1_DATA/.qpkg/OMedusa/OMedusa/medusa/init/__init__.py", line 39, in initialize
_configure_knowit()
File "/share/CACHEDEV1_DATA/.qpkg/OMedusa/OMedusa/medusa/init/__init__.py", line 240, in _configure_knowit
api.initialize({'mediainfo': suggested_path})
File "/share/CACHEDEV1_DATA/.qpkg/OMedusa/OMedusa/ext/knowit/api.py", line 35, in initialize
available_providers[name] = provider_cls(config, context.get(name) or config.general.get(name))
File "/share/CACHEDEV1_DATA/.qpkg/OMedusa/OMedusa/ext/knowit/providers/ffmpeg.py", line 218, in __init__
self.executor = FFmpegExecutor.get_executor_instance(suggested_path)
File "/share/CACHEDEV1_DATA/.qpkg/OMedusa/OMedusa/ext/knowit/providers/ffmpeg.py", line 108, in get_executor_instance
executor = exec_cls.create(os_family, suggested_path)
File "/share/CACHEDEV1_DATA/.qpkg/OMedusa/OMedusa/ext/knowit/providers/ffmpeg.py", line 131, in create
output = ensure_text(check_output([candidate, '-version']))
File "/opt/lib/python3.9/subprocess.py", line 424, in check_output
File "/opt/lib/python3.9/subprocess.py", line 528, in run
subprocess.CalledProcessError: Command '['ffprobe', '-version']' returned non-zero exit status 127."
result: [1]
-
- Experience counts
- Posts: 1001
- Joined: Mon Oct 27, 2014 2:27 pm
- Location: Moscow, Russia
- Contact:
Re: [QPKG] sherpa mini-package-manager (CLI)
It is probably the problem with libx256 entware package upgrade (from the opkg point of view - it is a downgrade). opkg cannot handle it. See my previous post with the probable solutions.dirkonline wrote: ↑Fri Apr 30, 2021 9:53 pm With the latest QTS Update 4.5.3.165 , OMedusa no longer starts
Is that an issue for anybody else? Ideas? I wish they would stop breaking things all the time, sigh
Reiinstall did not fix the issue
I do not know sherpa, if there is an option to reinstall entware - it will also fix the problem.
- OneCD
- Guru
- Posts: 12155
- Joined: Sun Aug 21, 2016 10:48 am
- Location: "... there, behind that sofa!"
Re: [QPKG] sherpa mini-package-manager (CLI)
I've been testing a fresh install with only the Entware and OMedusa QPKGs installed via sherpa.zyxmon wrote: ↑Sat May 01, 2021 3:54 amIt is probably the problem with libx256 entware package upgrade (from the opkg point of view - it is a downgrade). opkg cannot handle it. See my previous post with the probable solutions.dirkonline wrote: ↑Fri Apr 30, 2021 9:53 pm With the latest QTS Update 4.5.3.165 , OMedusa no longer starts
Is that an issue for anybody else? Ideas? I wish they would stop breaking things all the time, sigh
Reiinstall did not fix the issue
'ffmpeg' isn't installed as it's not required by these packages. Which means 'libx264' also isn't installed. Medusa uses the 'mediainfo' package to extract video dimensions instead of 'ffmpeg'. edit: .. but it will use 'ffmpeg' if 'mediainfo' isn't installed.
I think 'ffmpeg' is leftover from a previous sherpa version.
Might be better to remove 'libx264' and dependent packages like 'ffmpeg' as they're not needed anyway:
Code: Select all
opkg remove libx264 --force-removal-of-dependent-packages