Installing Newznab on QNAP

[cybercs]

I know there are lots of people that used to use NZBMatrix and Newzbin as their newsgroup indexers.

I have been trying to install the Newznab+ on my QNAP P219+ and I'm running into an issue running some PHP scripts.

This is the error I get when I attempt to run

Code: Select all

php update_binaries.php

Code: Select all

PHP Warning:  require_once(PEAR.php): failed to open stream: No such file or directory in /share/MD0_DATA/Web/www/lib/Net_NNTP/NNTP/Protocol/Client.php on line 73 PHP Fatal error:  require_once(): Failed opening required 'PEAR.php' (include_path='.:') in /share/MD0_DATA/Web/www/lib/Net_NNTP/NNTP/Protocol/Client.php on line 73 

I do have PEAR installed.

Any help would be appreciated

[forkless]

go to the GUI of your QNAP and edit your php.ini

and change;

Code: Select all

include_path = ".:/etc/config/php"

so that it reads;

Code: Select all

include_path = ".:/opt/share/pear:/etc/config/php"

After that restart your webserver so that the changes in the php.ini are read again.




ps. There was also a good comment somebody else made in another newznab thread that you can not use the path found php from the CLI, you will have to specify the full path to the 5.3.14 binary and force the -c option to look for the php.ini file.

http://forum.qnap.com/viewtopic.php?f=1 ... 14#p306589

[cybercs]

Wow great thanks!

This was a big help. I got the scans to work, thanks to that thread.

Now after it updates both binaries and releases, I should have some stuff listed on the main page? Correct?

For instance I just activated the DVD group. It scanned and completed. If I go to movies there should be items listed there now? correct?
However, nothing is listed there.

What am I missing? Also what does the Regex's do and should I be doing something with those?

Thanks again for your help!

EDIT: I have these two potential problems listed on my Newznab

Code: Select all

1	MySql my.cnf setting group_concat_max_len is too low, should be >= 8192.	
2	MySql my.cnf setting max_allowed_packet is too low, should be >= 12582912.

Any idea where I can make these changes?

[forkless]

You will need to get the donation version of newznab (plus) that will include the working regex's to filter your headers into readable releases. Unless you want to write them for each group seperately.

I for one didn't really bother to do so for a 10 pound donation. There are hundreds of them if you want to index all categories.


Regarding the my.cnf for MySQL you can find it in /etc/config

add;

Code: Select all

group_concat_max_len = 8192   
max_allowed_packet = 12582912

Just restart your MySQL instances after adding those two lines inside the [mysqld] section

[cybercs]

Thanks I will try this later tonight.

I did get the donation version. Where are the regex's located? Do I have to go to a site to download them? I wish the Newznab site had better instructions and even a forum to discuss. I tried the IRC and it was insane in there. Over 600 connected and the conversations were all over the place.

[forkless]

In 'Edit Site' under 'Lookup Settings' there is a URL that fetches the regex's at the start of the update_releases.php (ie. they should come in automatically)

You can use this adapted shell script to start the process for automated polling (save as newznab.sh and chmod 755 so it will become executable). Don't forget to edit the location for your NewzNab install;

You can fire off the script using ./newznab.sh start

Code: Select all

#!/bin/sh
#
# /etc/init.d/newznab: start and stop the newznab update script
#
# run update-rc.d newznab_ubuntu.sh defaults


### BEGIN INIT INFO
# Provides:          Newznab
# Required-Start:    $remote_fs $syslog
# Required-Stop:     $remote_fs $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start newznab at boot time
# Description:       Enable newznab service provided by daemon.
### END INIT INFO

RED=$(tput setaf 1)
GREEN=$(tput setaf 2)
NORMAL=$(tput sgr0)

col=40 

# Newznab variables 

NN_PATH="/share/Qweb/thedirectorytoyournewznab/misc/update_scripts"
NN_BINUP="update_binaries.php"
NN_RELUP="update_releases.php"
NN_PREDB="update_predb.php true"
NN_OPT="optimise_db.php"
NN_TV="update_tvschedule.php"
NN_THEATERS="update_theaters.php"
NN_SLEEP_TIME="10" # in seconds . 10sec is good for 100s of groups. 600sec might be a good start for fewer.
NN_PID_PATH="/var/run/" 
PIDFILE="newznab_sh.pid"
LASTOPTIMIZE=`date +%s`

#test -f /lib/lsb/init-functions || exit 1
#. /lib/lsb/init-functions

case "$1" in
  start)
	if [ -f ${NN_PID_PATH}${PIDFILE} ]
	then
		echo "$0 is already running."
	else
		echo -n "Starting Newznab binaries update..."
		cd ${NN_PATH}
		while :
			do 
				CURRTIME=`date +%s`
				/mnt/ext/opt/apache/bin/php -c /php.ini ${NN_BINUP}  2>&1 > /dev/null && /mnt/ext/opt/apache/bin/php -c /php.ini ${NN_RELUP}  2>&1 > /dev/null && /mnt/ext/opt/apache/bin/php -c /php.ini ${NN_PREDB}  2>&1 > /dev/null
				DIFF=$(($CURRTIME-$LASTOPTIMIZE))
				if [ "$DIFF" -gt 43200 ] || [ "$DIFF" -lt 1 ]
				then
					LASTOPTIMIZE=`date +%s`
					/mnt/ext/opt/apache/bin/php -c /php.ini ${NN_OPT}  2>&1 > /dev/null && /mnt/ext/opt/apache/bin/php -c /php.ini ${NN_TV}  2>&1 > /dev/null && /mnt/ext/opt/apache/bin/php -c /php.ini ${NN_THEATERS}  2>&1 > /dev/null
				fi
				sleep ${NN_SLEEP_TIME}
			done &
		PID=$!
		echo $PID > ${NN_PID_PATH}${PIDFILE}
		sleep 2
		if [ -f ${NN_PID_PATH}${PIDFILE} ]
		then
			printf '%s%*s%s\n' "$GREEN" $col '[OK]' "$NORMAL"
		else
			printf '%s%*s%s\n' "$RED" $col '[FAIL]' "$NORMAL"
		fi
        fi &
	;;
  stop)
        echo -n "Stopping Newznab binaries update..."
        kill -9 `cat ${NN_PID_PATH}${PIDFILE}` && kill -9 `cat ${NN_PID_PATH}${PIDFILE}` && rm ${NN_PID_PATH}${PIDFILE}
	sleep 2
	if [ -f ${NN_PID_PATH}${PIDFILE} ]
	then
		printf '%s%*s%s\n' "$RED" $col '[FAIL]' "$NORMAL"
	else
		printf '%s%*s%s\n' "$GREEN" $col '[OK]' "$NORMAL"
	fi
        ;;

  *)
        echo "Usage: $0 [start|stop]"
        exit 1
esac

[cybercs]

If my server restarts will this script start automatically or do I need to restart it manually?

Also where am I saving this script too?

Thanks again for your help!

[cybercs]

Sorry I see that I'm supposed to save it to the /etc/init.d/ folder.

Anyways, does the file execute automatically on a server restart?

[lmolenaar]

EDIT: Never mind, only now read your 2nd link

[cybercs]

Okay looks like the server is scanning the groups. However, the items are not listing under the proper categories. How do I list them on the site? I have tried to play around with the Regex's and I can test the alt.binaires.tv which displays 3 shows (I thought there would be more). But these 3 shows are not displaying on the main page under any category of TV. I looked under all the categories and none of them have anything listed under them. How do I have these categories populated? I was under the assumption that this would be automatic after running the update scripts.

What am I doing wrong?

[scoobynz]

Hi there, sorry to side-track the thread. Would love some insight into how this runs on your (our) low-powered qnap. From what I understand our qnap's may not have sufficient resources to deal with newznab? Any comments appreciated.

[cybercs]

I can't get it to work properly yet. It seems to run the web portion fine though

[lmolenaar]

In 'Edit Site' under 'Lookup Settings' there is a URL that fetches the regex's at the start of the update_releases.php (ie. they should come in automatically)

You can use this adapted shell script to start the process for automated polling (save as newznab.sh and chmod 755 so it will become executable). Don't forget to edit the location for your NewzNab install;

You can fire off the script using ./newznab.sh start

Code: Select all

#!/bin/sh
#
# /etc/init.d/newznab: start and stop the newznab update script
#
# run update-rc.d newznab_ubuntu.sh defaults


### BEGIN INIT INFO
# Provides:          Newznab
# Required-Start:    $remote_fs $syslog
# Required-Stop:     $remote_fs $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start newznab at boot time
# Description:       Enable newznab service provided by daemon.
### END INIT INFO

RED=$(tput setaf 1)
GREEN=$(tput setaf 2)
NORMAL=$(tput sgr0)

col=40 

# Newznab variables 

NN_PATH="/share/Qweb/thedirectorytoyournewznab/misc/update_scripts"
NN_BINUP="update_binaries.php"
NN_RELUP="update_releases.php"
NN_PREDB="update_predb.php true"
NN_OPT="optimise_db.php"
NN_TV="update_tvschedule.php"
NN_THEATERS="update_theaters.php"
NN_SLEEP_TIME="10" # in seconds . 10sec is good for 100s of groups. 600sec might be a good start for fewer.
NN_PID_PATH="/var/run/" 
PIDFILE="newznab_sh.pid"
LASTOPTIMIZE=`date +%s`

#test -f /lib/lsb/init-functions || exit 1
#. /lib/lsb/init-functions

case "$1" in
  start)
	if [ -f ${NN_PID_PATH}${PIDFILE} ]
	then
		echo "$0 is already running."
	else
		echo -n "Starting Newznab binaries update..."
		cd ${NN_PATH}
		while :
			do 
				CURRTIME=`date +%s`
				/mnt/ext/opt/apache/bin/php -c /php.ini ${NN_BINUP}  2>&1 > /dev/null && /mnt/ext/opt/apache/bin/php -c /php.ini ${NN_RELUP}  2>&1 > /dev/null && /mnt/ext/opt/apache/bin/php -c /php.ini ${NN_PREDB}  2>&1 > /dev/null
				DIFF=$(($CURRTIME-$LASTOPTIMIZE))
				if [ "$DIFF" -gt 43200 ] || [ "$DIFF" -lt 1 ]
				then
					LASTOPTIMIZE=`date +%s`
					/mnt/ext/opt/apache/bin/php -c /php.ini ${NN_OPT}  2>&1 > /dev/null && /mnt/ext/opt/apache/bin/php -c /php.ini ${NN_TV}  2>&1 > /dev/null && /mnt/ext/opt/apache/bin/php -c /php.ini ${NN_THEATERS}  2>&1 > /dev/null
				fi
				sleep ${NN_SLEEP_TIME}
			done &
		PID=$!
		echo $PID > ${NN_PID_PATH}${PIDFILE}
		sleep 2
		if [ -f ${NN_PID_PATH}${PIDFILE} ]
		then
			printf '%s%*s%s\n' "$GREEN" $col '[OK]' "$NORMAL"
		else
			printf '%s%*s%s\n' "$RED" $col '[FAIL]' "$NORMAL"
		fi
        fi &
	;;
  stop)
        echo -n "Stopping Newznab binaries update..."
        kill -9 `cat ${NN_PID_PATH}${PIDFILE}` && kill -9 `cat ${NN_PID_PATH}${PIDFILE}` && rm ${NN_PID_PATH}${PIDFILE}
	sleep 2
	if [ -f ${NN_PID_PATH}${PIDFILE} ]
	then
		printf '%s%*s%s\n' "$RED" $col '[FAIL]' "$NORMAL"
	else
		printf '%s%*s%s\n' "$GREEN" $col '[OK]' "$NORMAL"
	fi
        ;;

  *)
        echo "Usage: $0 [start|stop]"
        exit 1
esac

Hi forkless,

Could you please help me understand the script you posted?

• could you give a summary what this script does?
• Will it execute the PHP scripts sequentially? Once or continues?
• Will it run until a reboot?

[forkless]

It's just a slightly modified script from the one available in nix_scripts (I commented out the test function that exits the script if init-functions is not found)

The script will run the following things;

- Updating the binaries
- Create the releases
- Run the preDB script
- Fetch the TV Guide
- Fetch the Theatre releases
- Run database optimisation

These will be run sequentially in a loop until reboot.

To see what each individual script does I would suggest reading up on http://newznab.readthedocs.org/en/latest/

If you symlink this script to /etc/init.d it should start up after a reboot too. Haven't gotten around to test it yet though.

By the way, not to poop on anybodies party. Just so that people realize, I've run this script with 25+ groups on a TS-809 Pro and it had problems processing the amount of data it was getting (3gb overnight) over a days retention. I haven't even started backfilling yet. I doubt it will run any smoother on any of the ARM models. For that matter I don't think it will run decently on the Intel model I have as well, I guess time will tell.

[cybercs]

hmm this is strange...

When I attempt to run the script I get this error

Code: Select all

# ./newznab.sh start
-sh: ./newznab.sh: No such file or directory

I have saved the file in the etc/init.d folder and it has 0755 permissions. I also modified the file to point it to the proper newznab folder. Any suggestions?