[QPKG] sherpa mini-package-manager

This is the best place for community developers to publish their genius work. Your Apps enrich the QNAP Turbo NAS.
Post Reply
captainhowie
Starting out
Posts: 29
Joined: Fri Oct 05, 2012 3:24 pm

Re: [SCRIPT] sherpa.sh

Post by captainhowie » Thu Aug 13, 2020 1:57 pm

Firstly I just wanted to say thank you for all help from this forum in particularly @OneCD.

My SABnzbdPlus stopped working yesterday and after reading a few posts I decided to bite the bullet and try the Python 3 transition. It appears to have gone swimmingly. I followed the steps on pg 115 and migrated to Sabnzbd3 and the new version of SickChill (after a clean which was also in the forum!).

Both are up an running although I have two items I'd like to correct - both of which are potentially remnants of the issues I had with the previous upgrades of Python2 versions but nonetheless it's time to clean them up.

In SABnzbd i get the following error message inbetween the queue and history lists on the main page. The error repeats multiple times. Any ideas what this is in relation to?

ERROR a few seconds ago [13/Aug/2020:15:22:40] ENGINE Error in HTTPServer.tick
Traceback (most recent call last):
File "/opt/lib/python3.8/site-packages/cheroot/server.py", line 1795, in serve
self.tick()
File "/opt/lib/python3.8/site-packages/cheroot/server.py", line 2030, in tick
self.connections.expire()
File "/opt/lib/python3.8/site-packages/cheroot/connections.py", line 107, in expire
for sock_fd, conn in timed_out_connections:
File "/opt/lib/python3.8/site-packages/cheroot/connections.py", line 102, in <genexpr>
timed_out_connections = (
File "/opt/lib/python3.8/_collections_abc.py", line 743, in __iter__
RuntimeError: dictionary changed size during iteration

The second one relates to the autoprocess back to Sickchill. After previously moving from sickbeard to sickchill I was never able to get the sabToSickBeard.py working with Sickchill.

Pre python 3 - SickBeard: Failed to post-process - Returned log from SickBeard was not as expected.!
Post python 3 - SickBeard: Failed to post-process - Server returned status 500!

I suspect I just need to start again with the post processing - any tips on where the guide is to set this up again?

Thanks!
Andrew

User avatar
OneCD
Ask me anything
Posts: 7808
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: [SCRIPT] sherpa.sh

Post by OneCD » Thu Aug 13, 2020 2:06 pm

Kamikaze01 wrote:
Thu Aug 13, 2020 1:55 pm
A VERY LOT OF THANK U !! :DD :DD :DD
captainhowie wrote:
Thu Aug 13, 2020 1:57 pm
Firstly I just wanted to say thank you for all help from this forum in particularly @OneCD.
Cheers guys. :D
captainhowie wrote:
Thu Aug 13, 2020 1:57 pm
In SABnzbd i get the following error message inbetween the queue and history lists on the main page. The error repeats multiple times. Any ideas what this is in relation to?

ERROR a few seconds ago [13/Aug/2020:15:22:40] ENGINE Error in HTTPServer.tick
Traceback (most recent call last):
File "/opt/lib/python3.8/site-packages/cheroot/server.py", line 1795, in serve
self.tick()
File "/opt/lib/python3.8/site-packages/cheroot/server.py", line 2030, in tick
self.connections.expire()
File "/opt/lib/python3.8/site-packages/cheroot/connections.py", line 107, in expire
for sock_fd, conn in timed_out_connections:
File "/opt/lib/python3.8/site-packages/cheroot/connections.py", line 102, in <genexpr>
timed_out_connections = (
File "/opt/lib/python3.8/_collections_abc.py", line 743, in __iter__
RuntimeError: dictionary changed size during iteration
I started seeing this today too. I've reported it to the SABnzbd devs as issue #1572
captainhowie wrote:
Thu Aug 13, 2020 1:57 pm
The second one relates to the autoprocess back to Sickchill. After previously moving from sickbeard to sickchill I was never able to get the sabToSickBeard.py working with Sickchill.
...
I suspect I just need to start again with the post processing - any tips on where the guide is to set this up again?
First, suggest installing the new nzbToMedia package (available via sherpa). If you have an existing nzbToMedia in your 'Download' share, it will be converted to work with the new QPKG.

Then, ensure the [autoProcessMedia.cfg] file in nzbToMedia has been correctly configured with your SickChill settings. These go in the [SickBeard] section.

Note: there's a fairly recent error occurring that happens if your SickChill API key has been entered in [autoProcessMedia.cfg]. It manifests as a crash during post-processing in the SABnzbd log. So, leave it blank for now.

When configuring the 'tv' category in SABnzbd, ensure you post-process using the 'nzbToMedia.py' script.

I don't recall seeing a guide for this, but I'm happy to assist with your setup. I had to do the same thing a few times last weekend to confirm that post-processing works with SABnzbd -> nzbToMedia -> SickChill in Python 3, so it's all still quite fresh in my mind. :geek:

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage

cav
Starting out
Posts: 11
Joined: Mon Mar 21, 2011 6:40 pm

Re: [SCRIPT] sherpa.sh

Post by cav » Thu Aug 13, 2020 2:25 pm

OneCD wrote:
Thu Aug 13, 2020 9:37 am
... and done. HTTPS port checking should now work properly. ;)

@cav, please download the latest sherpa.sh and reinstall SABnzbd:

Code: Select all

cd /share/Public
curl -skLO https://git.io/sherpa.sh && chmod +x sherpa.sh
./sherpa.sh sab
Seems to be working now. Only thing I wanted to check was, if I stopped the app in App Center and then I started it via command-line (./sabnzbd3.sh start) is the icon in App Center supposed to detect that it's started and update the icon/status in App Center?

User avatar
OneCD
Ask me anything
Posts: 7808
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: [SCRIPT] sherpa.sh

Post by OneCD » Thu Aug 13, 2020 2:33 pm

cav wrote:
Thu Aug 13, 2020 2:25 pm
Only thing I wanted to check was, if I stopped the app in App Center and then I started it via command-line (./sabnzbd3.sh start) is the icon in App Center supposed to detect that it's started and update the icon/status in App Center?
If you mean: does the App Center know whether a QPKG has been stopped or started? No. It appears to have no checks for this. :(

Hmm, but you've got me thinking: maybe that's what the 'qpkg_service' program does. I saw it a while back and couldn't work out why it was there. But I didn't think of App Center. Something for me to test.

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage

User avatar
GTunney
Been there, done that
Posts: 649
Joined: Tue Oct 14, 2014 4:16 pm

Re: [SCRIPT] sherpa.sh

Post by GTunney » Thu Aug 13, 2020 3:05 pm

OneCD wrote:
Thu Aug 13, 2020 3:56 am
GTunney wrote:
Wed Aug 12, 2020 10:02 pm
I know this probably isn't really relevant to this thread but I also got the nzbtomedia ImportError: No module named imghdr` error after upgrading to sab3.
Oh, I should have responded to this bit too. :'

The fix I used in sherpa was to ensure the Python 3 interpreter I wanted for nzbToMedia was called before any other Python. As all the sherpa packages use Python 3 installed via 'opkg', and $PATH starts with [/opt/bin], I made sure there was a symlink like this:

Code: Select all

[[ ! -L /opt/bin/python && -e /opt/bin/python3 ]] && ln -s /opt/bin/python3 /opt/bin/python
... so when nzbToMedia starts, it will use Python 3.
Thanks, can I copy my existing configs and databases from Qsab, Qcouch and sickchill etc? Is there a more easier way to migrate these over?
Model: TS-653B 8GB
Disks: 3 x 4TB Western Digital WD40EFRX - RAID 5
Total Storage: 7.2TB
Applications: Plex MS | QCouchPotato | QSabnzbd+ | QSickrage | QHeadphones | HTPC Manager | Kodi v18.4 MySQL
Other Devices: Netgear D7000 AC1900 VDSL Router | FTTC - 80/20 | Netgear GS108 Gigabit Switch

User avatar
OneCD
Ask me anything
Posts: 7808
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: [SCRIPT] sherpa.sh

Post by OneCD » Thu Aug 13, 2020 3:18 pm

GTunney wrote:
Thu Aug 13, 2020 3:05 pm
... can I copy my existing configs and databases from Qsab, Qcouch and sickchill etc?
Yes, but it'll be a process of manually copying the settings across. :(

A long time ago, sherpa included the ability to convert all those settings automatically, but I removed it as it became too time-consuming to test with old packages.

If it helps, each sherpa QPKG uses the same location name to store all userdata in: a subdirectory of the QPKG installed path called 'config'. Each package configuration is saved in a file called 'config.ini' (the only exception being Transmission - I stuck with the old 'settings.json' for legacy reasons).

Note: CouchPotato is no-longer available via sherpa as development and support ceased some time ago. The currently supported movie grabber is Watcher3. Stephane already has a package for this app if you'd like to use it. But it's also installable via sherpa.

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage

captainhowie
Starting out
Posts: 29
Joined: Fri Oct 05, 2012 3:24 pm

Re: [SCRIPT] sherpa.sh

Post by captainhowie » Thu Aug 13, 2020 3:28 pm

OneCD wrote:
Thu Aug 13, 2020 2:06 pm
captainhowie wrote:
Thu Aug 13, 2020 1:57 pm
The second one relates to the autoprocess back to Sickchill. After previously moving from sickbeard to sickchill I was never able to get the sabToSickBeard.py working with Sickchill.
...
I suspect I just need to start again with the post processing - any tips on where the guide is to set this up again?
First, suggest installing the new nzbToMedia package (available via sherpa). If you have an existing nzbToMedia in your 'Download' share, it will be converted to work with the new QPKG.

Then, ensure the [autoProcessMedia.cfg] file in nzbToMedia has been correctly configured with your SickChill settings. These go in the [SickBeard] section.

Note: there's a fairly recent error occurring that happens if your SickChill API key has been entered in [autoProcessMedia.cfg]. It manifests as a crash during post-processing in the SABnzbd log. So, leave it blank for now.

When configuring the 'tv' category in SABnzbd, ensure you post-process using the 'nzbToMedia.py' script.

I don't recall seeing a guide for this, but I'm happy to assist with your setup. I had to do the same thing a few times last weekend to confirm that post-processing works with SABnzbd -> nzbToMedia -> SickChill in Python 3, so it's all still quite fresh in my mind. :geek:
Ok I went through all the steps and I think I have located the issue. The password I have been using ends in a #. When I have gone back into the .cfg file the # doesn't seem to stay permanently and therefore I think it causing the script to fail. I removed the # from the password altogether and the script worked.

So the Q is - am I not allowed to use # in the password if I'm using this script, or is there some sort of brackets/quotation marks etc that I can use to indicate that the # is part of the password, and not part of the config file?

Thanks Again!

User avatar
OneCD
Ask me anything
Posts: 7808
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: [SCRIPT] sherpa.sh

Post by OneCD » Thu Aug 13, 2020 3:36 pm

captainhowie wrote:
Thu Aug 13, 2020 3:28 pm
So the Q is - am I not allowed to use # in the password if I'm using this script, or is there some sort of brackets/quotation marks etc that I can use to indicate that the # is part of the password, and not part of the config file?
Oh, that's a good question. I've no idea. :DD

But '#' is typically used to indicate a comment or directive. It really depends on what the parser for the config file is capable of handling.

If it's the nzbToMedia config: suggest asking Clinton Hall by creating a new issue for the nzbToMedia repo on GitHub.

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage

User avatar
GTunney
Been there, done that
Posts: 649
Joined: Tue Oct 14, 2014 4:16 pm

Re: [SCRIPT] sherpa.sh

Post by GTunney » Thu Aug 13, 2020 11:25 pm

Is there any reason at all why Server Status for Scheduler in SickChill shows everything as False.

I'm pretty sure on QSickChill some of these used to be green and actively count down.

Image
Model: TS-653B 8GB
Disks: 3 x 4TB Western Digital WD40EFRX - RAID 5
Total Storage: 7.2TB
Applications: Plex MS | QCouchPotato | QSabnzbd+ | QSickrage | QHeadphones | HTPC Manager | Kodi v18.4 MySQL
Other Devices: Netgear D7000 AC1900 VDSL Router | FTTC - 80/20 | Netgear GS108 Gigabit Switch

User avatar
OneCD
Ask me anything
Posts: 7808
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: [SCRIPT] sherpa.sh

Post by OneCD » Fri Aug 14, 2020 3:17 am

GTunney wrote:
Thu Aug 13, 2020 11:25 pm
Is there any reason at all why Server Status for Scheduler in SickChill shows everything as False.
I've no idea. I hadn't seen that until you mentioned it. :lol:
GTunney wrote:
Thu Aug 13, 2020 11:25 pm
I'm pretty sure on QSickChill some of these used to be green and actively count down.
Note: SickChill is in the process of being upgraded to work with Python 3. It's still not 'stable', so there will be problems. If you do experience application problems, please report them to the developers so they can fix them: https://github.com/SickChill/SickChill/issues

sherpa.sh is only an installer. It can't fix application issues. ;)

edit: I've just created this as issue #6637.

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage

User avatar
GTunney
Been there, done that
Posts: 649
Joined: Tue Oct 14, 2014 4:16 pm

Re: [SCRIPT] sherpa.sh

Post by GTunney » Fri Aug 14, 2020 4:29 am

No probs at all dude. Hopefully it’s something simple.

I’m now fully transferred over to Sherpa for my main packages.

I just wish Watcher had an email alert notification like couch :-(
Model: TS-653B 8GB
Disks: 3 x 4TB Western Digital WD40EFRX - RAID 5
Total Storage: 7.2TB
Applications: Plex MS | QCouchPotato | QSabnzbd+ | QSickrage | QHeadphones | HTPC Manager | Kodi v18.4 MySQL
Other Devices: Netgear D7000 AC1900 VDSL Router | FTTC - 80/20 | Netgear GS108 Gigabit Switch

lukenukum
Starting out
Posts: 19
Joined: Mon Nov 09, 2015 10:40 pm

Re: [SCRIPT] sherpa.sh

Post by lukenukum » Fri Aug 14, 2020 9:01 pm

I can only re-iterate the thanks for this project :D much appreciated the work that's gone in.
OneCD wrote:
Thu Aug 13, 2020 2:06 pm
captainhowie wrote:
Thu Aug 13, 2020 1:57 pm
In SABnzbd i get the following error message inbetween the queue and history lists on the main page. The error repeats multiple times. Any ideas what this is in relation to?

ERROR a few seconds ago [13/Aug/2020:15:22:40] ENGINE Error in HTTPServer.tick
Traceback (most recent call last):
File "/opt/lib/python3.8/site-packages/cheroot/server.py", line 1795, in serve
self.tick()
File "/opt/lib/python3.8/site-packages/cheroot/server.py", line 2030, in tick
self.connections.expire()
File "/opt/lib/python3.8/site-packages/cheroot/connections.py", line 107, in expire
for sock_fd, conn in timed_out_connections:
File "/opt/lib/python3.8/site-packages/cheroot/connections.py", line 102, in <genexpr>
timed_out_connections = (
File "/opt/lib/python3.8/_collections_abc.py", line 743, in __iter__
RuntimeError: dictionary changed size during iteration
I started seeing this today too. I've reported it to the SABnzbd devs as issue #1572
Same. This follows some issues with the Python 3 modules not being found, that was fixed with an Entware update... but now seeing the errors as per CaptainHowie above. Just to confirm, we'll live with it for moment?

Thanks again!

User avatar
OneCD
Ask me anything
Posts: 7808
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: [SCRIPT] sherpa.sh

Post by OneCD » Fri Aug 14, 2020 11:45 pm

lukenukum wrote:
Fri Aug 14, 2020 9:01 pm
Same. This follows some issues with the Python 3 modules not being found, that was fixed with an Entware update... but now seeing the errors as per CaptainHowie above. Just to confirm, we'll live with it for moment?
Yep, it's an upstream issue with cheroot. It's being worked-on at the moment.

It only seems to occur when accessing the SABnzbd web UI , and doesn't appear to affect SABnzbd in any other way.

We'll need to wait for the fix. :geek:

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage

lukenukum
Starting out
Posts: 19
Joined: Mon Nov 09, 2015 10:40 pm

Re: [SCRIPT] sherpa.sh

Post by lukenukum » Sat Aug 15, 2020 12:54 am

OneCD wrote:
Fri Aug 14, 2020 11:45 pm
lukenukum wrote:
Fri Aug 14, 2020 9:01 pm
Same. This follows some issues with the Python 3 modules not being found, that was fixed with an Entware update... but now seeing the errors as per CaptainHowie above. Just to confirm, we'll live with it for moment?
Yep, it's an upstream issue with cheroot. It's being worked-on at the moment.

It only seems to occur when accessing the SABnzbd web UI , and doesn't appear to affect SABnzbd in any other way.

We'll need to wait for the fix. :geek:
Thanks for the update OneCD. Certainly not the end of the world, it still works - just wanted to check I hadn't missed a fix further back in the thread :D

User avatar
OneCD
Ask me anything
Posts: 7808
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: [SCRIPT] sherpa.sh

Post by OneCD » Sat Aug 15, 2020 4:04 am

lukenukum wrote:
Sat Aug 15, 2020 12:54 am
Thanks for the update OneCD. Certainly not the end of the world, it still works - just wanted to check I hadn't missed a fix further back in the thread :D
BTW: an interim workaround is to downgrade the 'cheroot' Python module. v8.4.4 is the problem version, so we want something earlier than that:

Code: Select all

pip3 install 'cheroot<8.4.4'
/etc/init.d/sabnzbd3.sh restart
edit: sherpa v200815 and later will do this automatically: https://github.com/OneCDOnly/sherpa/com ... 199909daa1

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage

Post Reply

Return to “Community Apps”