[QPKG] sherpa: a mini-package-manager (CLI) beta

[entrican]

Just installed sherpa_201116.qpkg. Last time I messed with any of this was in August of this year when sherpa.sh was still in use. What do I need to know before running any commands as it relates to the old way of doing things? I read the previously linked "sherpa for older users" ... but it doesn't say anything about Entware issues like we had to contend with back in August nor any references to SABnzbd versus SABnzbd3. Do I simply run sherpa (no .sh) from shell and it will magically resolve all updates?

I'm running 4.5.1.1480 and Sickchill is actually launching, but still has the nag-banner about using the "dev branch" instead of "master".

Should I just leave it alone?

[OneCD]

Do I simply run sherpa (no .sh) from shell and it will magically resolve all updates?

If run correctly - yes.

Code: Select all

sherpa check

I'm running 4.5.1.1480 and Sickchill is actually launching, but still has the nag-banner about using the "dev branch" instead of "master".

Should I just leave it alone?

The current sherpa SickChill package uses 'master' (and I must remember to update this branch name when the devs do-so).

You can either edit the branch name from 'develop' to 'master' in the SickChill service script, or else reinstall/upgrade SickChill which will do this for you:

Code: Select all

sherpa upgrade sc

If you'd like to stick with the 'develop' branch, there's a SickChill configuration key-value that will suppress UI warnings: viewtopic.php?p=763253#p763253

[rs1001]

Quick question: Does sherpa have a command to start-up apps?
Eg; I've shut down mylar and the gui for QTS doesn't have the "Start" option any more for the Sherpa-installed apps.

I could go find the .sh and load it up (or restart) but just wondered if there's a command-line driven option for it.

[OneCD]

Quick question: Does sherpa have a command to start-up apps?

Yes, you can start multiple specific apps with:

Code: Select all

sherpa start mylar sab

... or start them all with:

Code: Select all

sherpa start-all

Eg; I've shut down mylar and the gui for QTS doesn't have the "Start" option any more for the Sherpa-installed apps.

Ah, sounds like the QTS App Center isn't "in-sync" with the app - it happens. The QTS App Center doesn't perform any live monitoring of the application's status.

If you start or stop a sherpa app at the CLI, the service script for that app will also update the icon status in App Center. But you'll need to reload the App Center to see the updated status.

[gbt92]

Hi,

I'm migrating to a new Qnap (TS451+) and I'm using Sherpa to install sickChill.
But it seems to be blocked since 30 mn ....

How long is "a while"

Here is the logs :
------sherpa---

Code: Select all

dbug: (II) --------------------------------------------------------------------------------------------
dbug: (**)  HARDWARE:                   model: TS-451+
dbug: (**)  HARDWARE:                     RAM: 1925264 kB
dbug: (**)  FIRMWARE:        firmware version: 4.5.1
dbug: (**)  FIRMWARE:          firmware build: 20201123
dbug: (**)  FIRMWARE:                  kernel: 4.14.24-qnap x86_64
dbug: (**) USERSPACE:               OS uptime: 27 min
dbug: (**) USERSPACE:             system load: 1 min=1.46, 5 min=2.12, 15 min=2.18
dbug: (**) USERSPACE:                   $USER: admin
dbug: (**) USERSPACE:                   $EUID: 0
dbug: (**) USERSPACE:                    BASH: GNU bash, version 3.2.57(1)-release (x86_64-QNAP-linux-gnu)
dbug: (**) USERSPACE:          default volume: /share/50000F001BA1CB00_DATA
dbug: (**) USERSPACE:                    /opt: /share/50000F001BA1CB00_DATA/.qpkg/Entware
dbug: (**) USERSPACE:                   $PATH: /opt/bin:/opt/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/u...
dbug: (**) USERSPACE:          'python2' path: /usr/local/bin/python2
dbug: (**) USERSPACE:       'python2' version: Python 2.7.5
dbug: (WW) USERSPACE:          'python3' path: <not present>
dbug: (**) USERSPACE:           'python' path: /usr/local/bin/python
dbug: (**) USERSPACE:        'python' version: Python 2.7.5
dbug: (**) USERSPACE: unparsed user arguments: 'install SickChill'
dbug: (**)    SCRIPT:               logs path: /share/50000F001BA1CB00_DATA/.qpkg/sherpa/logs
dbug: (**)    SCRIPT:               work path: /share/50000F001BA1CB00_DATA/.qpkg/sherpa/cache
dbug: (**)    SCRIPT:   object reference hash: 2cac0f9a823dde1c96947b13b8c86f99
dbug: (**)      QPKG:       Entware installer: std
dbug: (**)      QPKG:                    arch: x64
dbug: (**)      QPKG:   upgradable package(s):
dbug: (II) --------------------------------------------------------------------------------------------
dbug: (>>) QPKGs.Assignment.Check()
dbug: (>>) QPKGs.DepAndIndep.Build()
dbug: (<<) QPKGs.DepAndIndep.Build()|0|13ms
dbug: (>>) QPKGs.InstallationState.Build()
dbug: (<<) QPKGs.InstallationState.Build()|0|162ms
dbug: (>>) QPKGs.Upgradable.Build()
dbug: (<<) QPKGs.Upgradable.Build()|0|9ms
dbug: (>>) QPKGs.Assignment.List()
dbug: (**)      QPKG:             to download: SickChill
dbug: (**)      QPKG:               to backup:
dbug: (**)      QPKG:            to uninstall:
dbug: (**)      QPKG:                 to stop:
dbug: (**)      QPKG:        to force-upgrade:
(Enter:next line Space:next page Q:quit R:show the rest)

Code: Select all

[/share/50000F001BA1CB00_DATA/.qpkg/sherpa] # tail -f  sherpa.debug.log
dbug: (<<) IPKGs.Install()|0|23902ms
dbug: (<<) Packages.Install.Addons()|0|24038ms
dbug: (>>) Packages.Upgrade.Dependants()
dbug: (<<) Packages.Upgrade.Dependants()|0|9ms
dbug: (>>) Packages.Reinstall.Dependants()
dbug: (<<) Packages.Reinstall.Dependants()|0|8ms
dbug: (>>) Packages.Install.Dependants()
dbug: (>>) QPKG.Install()
proc: installing 'SickChill' - this may take a while ...
dbug: (>>) RunAndLogResults()

Code: Select all

[/share/50000F001BA1CB00_DATA/.qpkg/sherpa] # cat /var/log/SickChill.log
no configuration file found: using default
>>>>>>>>>> datetime:'Sat Nov 28 16:59:01 CET 2020',request:'start',QPKG:'200916',app:'' <<<<<<<<<<
daemon: NOT active
file (/opt/bin/git): exists

[gbt92]

Additional info : Git clone Sickchill is quick for me

# time git clone http://github.com/sickchill/sickchill.git
Cloning into 'sickchill'...
warning: redirecting to https://github.com/sickchill/sickchill.git/
remote: Enumerating objects: 94, done.
remote: Counting objects: 100% (94/94), done.
remote: Compressing objects: 100% (75/75), done.
remote: Total 116867 (delta 39), reused 36 (delta 14), pack-reused 116773
Receiving objects: 100% (116867/116867), 248.99 MiB | 15.00 MiB/s, done.
Resolving deltas: 100% (72460/72460), done.
Updating files: 100% (6500/6500), done.

real 0m30.687s
user 0m45.945s
sys 0m4.409s

[OneCD]

But it seems to be blocked since 30 mn ....

How long is "a while"

Good question!

Admittedly, the SickChill clone is the largest available in sherpa, but 30 minutes is way too long. Sounds like another temporary error. I may need to impose a timeout on those 'git' commands.

Can you try installing SickChill with sherpa but in debug mode?

Code: Select all

sherpa install sc debug

[gbt92]

Of course

Code: Select all

[~] # sherpa reinstall sc debug

dbug: (vv) $_object_session_debug_to_screen_flag_ [true]
dbug: (vv) $_object_session_summary_flag_ [true]
dbug: (>>) Session.ParseArguments()
dbug: (<<) Session.ParseArguments()|0|23ms
dbug: (vv) $_object_session_debug_to_file_flag_ [true]
dbug: (II) ============================================================================================
dbug: (**)    SCRIPT:                 started: Sat Nov 28 20:56:55 CET 2020
dbug: (**)    SCRIPT:                 version: package: 201116, manager: 201128, loader 200923
dbug: (**)    SCRIPT:                     PID: 4266
dbug: (II) --------------------------------------------------------------------------------------------
dbug: (II) Markers: (**) detected, (II) information, (WW) warning, (EE) error, (LL) log file,
dbug: (II) (==) processing, (--) done, (>>) f entry, (<<) f exit, (vv) variable name & value,
dbug: (II) ($1) positional argument value
dbug: (II) --------------------------------------------------------------------------------------------
dbug: (>>) QPKGs.DepAndIndep.Build()
dbug: (<<) QPKGs.DepAndIndep.Build()|0|22ms
dbug: (>>) QPKGs.InstallationState.Build()
dbug: (<<) QPKGs.InstallationState.Build()|0|161ms
dbug: (>>) QPKGs.Upgradable.Build()
dbug: (<<) QPKGs.Upgradable.Build()|0|46ms
dbug: (<<) Session.Init()|0|785ms
dbug: (>>) Session.Validate()
dbug: (II) --------------------------------------------------------------------------------------------
dbug: (**)  HARDWARE:                   model: TS-451+
dbug: (**)  HARDWARE:                     RAM: 1925264 kB
dbug: (**)  FIRMWARE:        firmware version: 4.5.1
dbug: (**)  FIRMWARE:          firmware build: 20201123
dbug: (**)  FIRMWARE:                  kernel: 4.14.24-qnap x86_64
dbug: (**) USERSPACE:               OS uptime: 4:26
dbug: (**) USERSPACE:             system load: 1 min=1.70, 5 min=1.72, 15 min=2.27
dbug: (**) USERSPACE:                   $USER: admin
dbug: (**) USERSPACE:                   $EUID: 0
dbug: (**) USERSPACE:                    BASH: GNU bash, version 3.2.57(1)-release (x86_64-QNAP-linux-gnu)
dbug: (**) USERSPACE:          default volume: /share/50000F001BA1CB00_DATA
dbug: (**) USERSPACE:                    /opt: /share/50000F001BA1CB00_DATA/.qpkg/Entware
dbug: (**) USERSPACE:                   $PATH: /opt/bin:/opt/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/u...
dbug: (**) USERSPACE:          'python2' path: /usr/local/bin/python2
dbug: (**) USERSPACE:       'python2' version: Python 2.7.5
dbug: (**) USERSPACE:          'python3' path: /opt/bin/python3
dbug: (**) USERSPACE:       'python3' version: Python 3.8.5
dbug: (**) USERSPACE:           'python' path: /opt/bin/python
dbug: (**) USERSPACE:        'python' version: Python 3.8.5
dbug: (**) USERSPACE: unparsed user arguments: 'reinstall sc debug'
dbug: (**)    SCRIPT:               logs path: /share/50000F001BA1CB00_DATA/.qpkg/sherpa/logs
dbug: (**)    SCRIPT:               work path: /share/50000F001BA1CB00_DATA/.qpkg/sherpa/cache
dbug: (**)    SCRIPT:   object reference hash: 2cac0f9a823dde1c96947b13b8c86f99
dbug: (**)      QPKG:       Entware installer: std
dbug: (**)      QPKG:                    arch: x64
dbug: (**)      QPKG:   upgradable package(s):
dbug: (II) --------------------------------------------------------------------------------------------
dbug: (>>) QPKGs.Assignment.Check()
dbug: (>>) QPKGs.DepAndIndep.Build()
dbug: (<<) QPKGs.DepAndIndep.Build()|0|21ms
dbug: (>>) QPKGs.InstallationState.Build()
dbug: (<<) QPKGs.InstallationState.Build()|0|161ms
dbug: (>>) QPKGs.Upgradable.Build()
dbug: (<<) QPKGs.Upgradable.Build()|0|45ms
dbug: (>>) QPKGs.Assignment.List()
dbug: (**)      QPKG:             to download: SickChill
dbug: (**)      QPKG:               to backup:
dbug: (**)      QPKG:            to uninstall:
dbug: (**)      QPKG:                 to stop:
dbug: (**)      QPKG:        to force-upgrade:
dbug: (**)      QPKG:              to upgrade:
dbug: (**)      QPKG:            to reinstall: SickChill
dbug: (**)      QPKG:              to install:
dbug: (**)      QPKG:              to restore:
dbug: (**)      QPKG:                to start: Entware
dbug: (**)      QPKG:              to restart:
dbug: (<<) QPKGs.Assignment.List()|0|252ms
dbug: (<<) QPKGs.Assignment.Check()|0|621ms
dbug: (II) --------------------------------------------------------------------------------------------
dbug: (<<) Session.Validate()|0|1249ms
dbug: (>>) Packages.Download()
dbug: (>>) QPKG.Download()
dbug: (II) local package (SickChill_200916.qpkg) checksum correct, so skipping download
dbug: (<<) QPKG.Download()|0|45ms
dbug: (<<) Packages.Download()|0|83ms
dbug: (>>) Packages.Backup()
dbug: (<<) Packages.Backup()|0|16ms
dbug: (>>) Packages.Stop()
dbug: (<<) Packages.Stop()|0|16ms
dbug: (>>) Packages.Uninstall()
dbug: (>>) IPKGs.Uninstall()
dbug: (>>) IPKGs.Uninstall.Batch()
dbug: (>>) CalcAllIPKGDepsToUninstall()
dbug: (II) IPKGs requested:
dbug: (==) excluding IPKGs not installed ...
dbug: (--) complete
dbug: (II) IPKGs to uninstall:
dbug: (<<) CalcAllIPKGDepsToUninstall()|0|86ms
dbug: (<<) IPKGs.Uninstall.Batch()|0|124ms
dbug: (<<) IPKGs.Uninstall()|0|340ms
dbug: (<<) Packages.Uninstall()|0|399ms
dbug: (>>) Packages.Reinstall.Independents()
dbug: (<<) Packages.Reinstall.Independents()|0|20ms
dbug: (>>) Packages.Install.Independents()
dbug: (<<) Packages.Install.Independents()|0|17ms
dbug: (>>) Packages.Start.Independents()
dbug: (>>) QPKG.Enable()
dbug: (<<) QPKG.Enable()|0|28ms
dbug: (>>) QPKG.Start()
proc: starting 'Entware' ...
dbug: (>>) RunAndLogResults()
dbug: (==) executing '/sbin/qpkg_service start Entware' ...
Enable Entware/opkg
dbug: (LL) --------------------------------------------------------------------------------------------
dbug: (LL) adding external log to main log ...
dbug: (vv) $_object_session_debug_to_screen_flag_ [true]
dbug: (LL) --------------------------------------------------------------------------------------------
dbug: (<<) RunAndLogResults()|0|1171ms
done: started 'Entware'
dbug: (WW) unable to get status of 'Entware' service. It may be a non-sherpa package, or a package earlier than 200816c that doesn't support service results.
dbug: (<<) QPKG.Start()|0|1257ms
dbug: (<<) Packages.Start.Independents()|0|1361ms
dbug: (>>) Packages.Install.Addons()
dbug: (vv) $_object_session_ipkgs_install_flag_ [true]
dbug: (--) adjusted $PATH for Entware
dbug: (vv) $PATH [/opt/bin:/opt/sbin::/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin/X11:/usr/local/sbin:/usr/local/bin]
dbug: (II) patch: do the "opt shuffle" - already done
dbug: (II) 'Entware' package list was updated less than 60 minutes ago
dbug: (>>) IPKGs.Install()
dbug: (>>) IPKGs.Upgrade.Batch()
dbug: (<<) IPKGs.Upgrade.Batch()|0|137ms
dbug: (>>) IPKGs.Install.Batch()
dbug: (>>) CalcAllIPKGDepsToInstall()
proc: calculating IPKGs required ...
dbug: (II) IPKGs requested: ca-certificates gcc git git-http nano python3-dev python3-pip python3-setuptools
dbug: (>>) IPKGs.Archive.Open()
dbug: (>>) RunAndLogResults()
dbug: (==) executing '/usr/local/sbin/7z e -o/share/50000F001BA1CB00_DATA/.qpkg/sherpa/cache /opt/var/opkg-lists/entware' ...

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=C,Utf16=off,HugeFiles=on,64 bits,4 CPUs x64)

Scanning the drive for archives:
1 file, 292477 bytes (286 KiB)

Extracting archive: /opt/var/opkg-lists/entware
--
Path = /opt/var/opkg-lists/entware
Type = gzip
Headers Size = 19

Everything is Ok

Size:       1111982
Compressed: 292477
dbug: (<<) RunAndLogResults()|0|70ms
dbug: (<<) IPKGs.Archive.Open()|0|114ms
dbug: (==) finding IPKG dependencies ...
dbug: (==) iteration 1 ...
dbug: (==) iteration 2 ...
dbug: (==) iteration 3 ...
dbug: (==) iteration 4 ...
dbug: (==) iteration 5 ...
dbug: (==) iteration 6 ...
dbug: (==) iteration 7 ...
dbug: (==) iteration 8 ...
dbug: (==) iteration 9 ...
dbug: (==) iteration 10 ...
dbug: (==) iteration 11 ...
dbug: (==) iteration 12 ...
dbug: (==) iteration 13 ...
dbug: (--) complete
dbug: (II) found all IPKG dependencies in 13 iterations
dbug: (II) IPKGs requested + dependencies: ar binutils ca-bundle ca-certificates ca-certs file gcc git git-http libbfd libbz2 libc libctf libcurl libdb47 libexpat libffi libgcc libgdbm libiconv-full libintl-full liblzma libmagic libncurses libncursesw libopcodes libopenssl libpthread librt libsqlite3 libssp libstdcpp libtirpc libuuid libxml2 nano objdump python-pip-conf python3 python3-asyncio python3-base python3-cgi python3-cgitb python3-codecs python3-ctypes python3-dbm python3-decimal python3-dev python3-distutils python3-email python3-gdbm python3-lib2to3 python3-light python3-logging python3-lzma python3-multiprocessing python3-ncurses python3-openssl python3-pip python3-pkg-resources python3-pydoc python3-setuptools python3-sqlite3 python3-unittest python3-urllib python3-xml terminfo zlib
dbug: (==) excluding IPKGs already installed ...
dbug: (--) complete
dbug: (II) IPKGs to download:
done: no IPKGs are required
dbug: (<<) CalcAllIPKGDepsToInstall()|0|1160ms
dbug: (<<) IPKGs.Install.Batch()|0|1201ms
dbug: (<<) IPKGs.Install()|0|1580ms
dbug: (<<) Packages.Install.Addons()|0|1762ms
dbug: (>>) Packages.Upgrade.Dependants()
dbug: (<<) Packages.Upgrade.Dependants()|0|19ms
dbug: (>>) Packages.Reinstall.Dependants()
dbug: (>>) QPKG.Reinstall()
proc: re-installing 'SickChill' - this may take a while ...
dbug: (>>) RunAndLogResults()
dbug: (==) executing '/bin/sh /share/50000F001BA1CB00_DATA/.qpkg/sherpa/cache/qpkgs/SickChill_200916.qpkg' ...
Install QNAP package on TS-NAS...
19+1 records in
19+1 records out
58774 bytes (57.4KB) copied, 0.001155 seconds, 48.5MB/s
./
./built_info
./qinstall.sh
./package_routines
./qpkg.cfg
1+1 records in
1+1 records out
38294 bytes (37.4KB) copied, 0.000085 seconds, 429.6MB/s
37+1 records in
37+1 records out
38294 bytes (37.4KB) copied, 0.000471 seconds, 77.5MB/s
SickChill 200916 is already installed. Setup will now perform package upgrading.
Firmware check is fine.
daemon: NOT active
Link service start/stop script: sickchill.sh
Set QPKG information in /etc/config/qpkg.conf
daemon: NOT active
file (/opt/bin/git): exists

[OneCD]

Of course

Code: Select all

file (/opt/bin/git): exists

Weird. Works fine here.

Are you able to edit the 'sickchill.sh' service script?

Code: Select all

nano $(getcfg SickChill Install_Path -f /etc/config/qpkg.conf)/sickchill.sh

Please locate the StartQPKG() function and add a couple of 'set' commands to the middle of that function so it looks like this:

Code: Select all

    fi
set -x
    [[ $(type -t WaitForGit) = 'function' ]] && { WaitForGit || return 1 ;}
    [[ $(type -t PullGitRepo) = 'function' ]] && PullGitRepo "$QPKG_NAME" "$SOURCE_GIT_URL" "$SOURCE_GIT_BRANCH" "$SOURCE_GIT_DEPTH" "$QPKG_PATH"

    WaitForLaunchTarget || return 1
    [[ $(type -t UpdateLanguages) = 'function' ]] && UpdateLanguages
set +x
    EnsureConfigFileExists

... then restart SickChill with:

Code: Select all

/etc/init.d/sickchill.sh restart

You'll see a bunch of additional information displayed as that part of the function runs. Eventually, it should stop at around the same message indicated by your previous logs:

Code: Select all

file (/opt/bin/git): exists

When this happens, press CTRL+C to stop the script running, then copypaste all that text back here into code tags please.

[gbt92]

I prefer vi to nano

Here are the logs

Code: Select all

[~] # /etc/init.d/sickchill.sh restart
daemon: NOT active
++ type -t WaitForGit
+ [[ function = \f\u\n\c\t\i\o\n ]]
+ WaitForGit
+ WaitForFileToAppear /opt/bin/git 300
+ [[ -z /opt/bin/git ]]
+ [[ -n 300 ]]
+ MAX_SECONDS=300
+ [[ ! -e /opt/bin/git ]]
++ FormatAsFileName /opt/bin/git
++ echo '(/opt/bin/git)'
+ DisplayCommitToLog 'file (/opt/bin/git): exists'
+ Display 'file (/opt/bin/git): exists'
+ echo 'file (/opt/bin/git): exists'
file (/opt/bin/git): exists
+ CommitLog 'file (/opt/bin/git): exists'
+ IsNotStatus
+ [[ restart = status ]]
+ IsNotLog
+ [[ restart = log ]]
+ echo 'file (/opt/bin/git): exists'
+ return 0
+ . /etc/profile

Control-C cannot break the current command. It's freezed.

The issue seems to be in /etc/profile
The last line of my profile is :

Code: Select all

[[ "admin" = "$USER" ]] && /sbin/qts-console-mgmt -f

[OneCD]

I prefer vi to nano

I never could get my head around 'vi'. I think it was written by a sadist.

The issue seems to be in /etc/profile
The last line of my profile is :

Code: Select all

[[ "admin" = "$USER" ]] && /sbin/qts-console-mgmt -f

Ah, yes - that would be the reason for the hang. I forgot the newer QTS shells launch that dam'n console management menu. I don't have access to the new QTS version so I don't have to deal with it myself.

This means I'll need to release new sherpa QPKG versions with updated service scripts to fix this.

Something for me to do today. I'll post back when they're ready to go.

[gbt92]

I temporary commented the last line of the /etc/profile and the installation is successful.

But, now the sickchill icon in App Center has no Open button

[OneCD]

But, now the sickchill icon in App Center has no Open button

Did the SickChill service script output indicate it had enabled the App Center icon?

[gbt92]

I disabled debug then i got these logs

Code: Select all

[~] # sherpa install sc
sherpa 201128 ⢠a mini-package-manager for QNAP NAS

done: started 'Entware'
done: no IPKGs are required
done: installed 'SickChill'

I can reinstall it with debug if you want

[OneCD]

No, you only need to restart SickChill via the service script:

Code: Select all

/etc/init.d/sickchill.sh restart