[HOWTO] replace mt-daapd with forked-daapd (iTunes server)

[ADoko]

Thanks to the efforts of internaut19, forked-daapd is now available as a QPKG.
I would highly recommend that users unfamiliar with the linux shell install the QPKG instead. You can find the X86 and ARM packages here:
viewtopic.php?f=128&t=47748&p=221661#p221658

Again, kudos to internaut19 for getting this all setup and running.

-----------------------------

* This package is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. You, as the
* user of this package, assume all liability for and damage to
* or loss of data that may result from using this package.
*
* THIS PACKAGE MODIFIES THE SOFTWARE ON YOUR QNAP DEVICE. It
* replaces serveral pre-installed services with new services that
* are part of this package. You, as the package installer, accept
* all liability for any damage that may occur from the installation
* of ths package.
*

SUMMARY:
This package installs forked-daapd (iTunes daemon) and the avahi-daemon (FOSS implementation of Apple's Bonjour protocol) on your QNAP NAS. All binaries, libraries and headers are installed in /opt/unstable. The libraries and binaries are linked in such a manner as to only need to use libraries in /opt/unstable/lib. Several new scripts are installed in /opt/etc/init.d to start and stop the new daemons. There are also shim scripts that are installed in /etc/init.d that provide seamless integration with the web UI for these new daemons. There has also been an attempt to allow these changes to persist between reboots, and this seems to be working.

PREREQUISITES:
1. A linux kernel version 2.6.16 or higher. (Currently the TS109 and TS209 do not meet this requirement.)
2. The user must have enabled either telnet or ssh access to the QNAP.
3. The user must have installed and enabled the Optware/ipkg QPKG.
4. The user must have installed dbus ipkg. (As simple as typing: ipkg install dbus at the shell prompt.)

The current build of forked-daapd DOES NOT require IPv6 to be enabled on the QNAP. If you turned on IPv6 for an older version of forked-daapd (compiled prior to 2011-03-18) you can now disable IPv6 after upgrading.

STOP!!! READ THE SETUP INSTRUCTIONS PRIOR TO TRYING THEM!
If you are uncomfortable with installing binaries compiled by some random human being on the internet then please go here to learn how to compile your own:
http://wiki.qnap.com/wiki/User:AnthonyDoko

This page has all the information you will need to compile these binaries for yourself on an Ubuntu 10.10 linux machine.

I encourage you to read through all the scripts that are part of the install and part of the operation/configuration of these new daemons. I have not done anything malicious, but I may have made errors. These scripts have worked for me on my TS-419P BUT THAT IS NOT A GUARANTEE THAT THEY WILL WORK ON YOUR QNAP DEVICE. I should note that my TS-419P is configured for RAID5 across four disks. I am most concerned about the Optware.sh script. It is replaced with the version that was on my TS-419P, but I'm not clear if different QNAP models have different Optware.sh scripts. If they do then this install will not work.

About bugs:
This software probably has bugs that are beyond my control. It relies on no less than 20 libraries. Each with its own quirks and problems. The forked-daapd is also in active development so there is a high probability that it will have bugs and be evolving rapidly. I will try to keep it up to date when I see major improvements or milestone releases. Users of this package can feel free to PM me in the forums or add a post to this thread if they feel there is some outstanding issue that can be fixed by a new build.

NOTE: Playback of .wma/.ogg on ARM-based QNAPs is currently not supported. (The ARM processor does not have the requisite FPU coprocessor and can't decode the audio packets fast enough for DAAP/RAOP devices.) I'm pretty sure transcoding in the general sense isn't going to work on the ARM if it requires any amount of floating point math. However, .mp3 files work just fine.

I can only vouch for the ARM version of this package as I have tested it on my TS-419P.
Regarding the X86-based QNAP NAS: If you have an X86-based QNAP I've built a binary package for it. It runs on my Ubuntu VM install just fine. However, I haven't run it or its install script on an X86-based QNAP. If you follow this thread you will see that it has been successfully installed and run by someone other than me. Hopefully there are a few more adventurous individuals that will try it on their X86 QNAPs.

How do you know which architecture you're on?
In a shell issue this command:

Code: Select all

uname -a

If you're on ARM you'll see something like this:

Code: Select all

Linux NAS-NAME 2.6.33.2 #1 Tue Nov 9 14:05:13 CST 2010 armv5tel unknown

If you're on X86 you'll see something like this:

Code: Select all

Linux NAS-NAME 2.6.33.2 #1 SMP Tue Nov 9 23:40:50 CST 2010 i686 unknown

armv5tel is ARM and i686 is X86 Choose your forked-daapd-install.tar.gz file accordingly!


SETUP VIA INSTALL SCRIPT: (Do all of this while logged in as admin.)
1. Download the following compressed archive and place it in your
/share/Public directory:

Latest builds:
(Includes: forked-daapd v0.18, libav HEAD + ATV patches, Avahi 0.6.30, eglibc 2.14, and RAOP autoselect)
ARM http://ubuntuone.com/p/1BPP/
X86 http://ubuntuone.com/p/1BPQ/

Older builds:
(Includes: forked-daapd v0.16, ffmpeg 0.7.x + ATV patches, Avahi 0.6.30, eglibc 2.12 and RAOP autoselect)
ARM: http://ubuntuone.com/p/qY3/
X86: http://ubuntuone.com/p/qY7/

(Includes: forked-daapd v0.15, ffmpeg 0.6.1 + ATV patches, Avahi 0.6.30, eglibc 2.12 and RAOP autoselect)
ARM: http://ubuntuone.com/p/n23/
X86: http://ubuntuone.com/p/n25/

The examples below use the YYYY-MM-DD-ARM-forked-daapd-install.tar.gz. If you're on an X86 QNAP then just substitute "X86-" for "ARM-" in all the examples below.

2. Uncompress the archive and un-tar it:

Code: Select all

gunzip -c 2011-05-02-ARM-forked-daapd-install.tar.gz | tar xf -


3. Stop the iTunes, timemachine and bonjour services

Code: Select all

/opt/etc/init.d/S20dbus stop
/etc/init.d/init_iTune.sh stop
/etc/init.d/timemachine.sh stop
/etc/init.d/bonjour.sh stop


4. Run the package install script:

Code: Select all

cd ./2011-05-02-ARM-forked-daapd-install
./package install nodebug


5a. Reboot

Code: Select all

reboot -d1


OR 5b. Start the new services manually:

Code: Select all

/opt/etc/init.d/S15replace
/opt/etc/init.d/S20dbus start
/opt/etc/init.d/S30bonjour start
/opt/etc/init.d/S35timemachine start
/opt/etc/init.d/S50iTunes start


HOW TO UNINSTALL: WARNING: You'll remove /opt/unstable and this holds your songs3.db file. Move it somewhere safe before uninstalling if you don't want to loose it.

Code: Select all

/opt/etc/init.d/S20dbus stop
/opt/etc/init.d/S30bonjour stop
/opt/etc/init.d/S35timemachine stop
/opt/etc/init.d/S50iTunes stop
cd /share/Public/forked-daapd-install
./package uninstall nodebug


TO DO A DRY RUN

Code: Select all

./package install


Code: Select all

./package uninstall


I require the nodebug keyword to do the real install in order to prevent accidents. I'm not the best shell script writer and I was concerned that I'd blow away my install while testing the install script.

NOTES/THEORY:
You can always run the ./package script without the "nodebug" at the end to see what the script will do. If you want to manually install the the package then the first place to start reading is the ./package script. It's a very simple script. The install() method just issues the set of shell commands that I initially used to set up the avahi-daemon and forked-daapd.

There are two directories in the forked-daapd-install directory that have the scripts that will be installed on your system. The directories are named after their destinations. You can and should inspect the scripts before installing.

The S15replace and Optware.sh scripts are what I use to survive the reboot process. The modified Optware.sh will run all the scripts in /opt/etc/init.d on reboot. One of these is the S15replace script. It will replace the init_iTune.sh, bonjour.sh and timemachine.sh scripts in /etc/init.d during the reboot process. The replacement scripts are shims that call through to /opt/etc/init.d counterparts given the right conditions. This allows the Web UI to continue to function. I'm not terribly fond of this method as it is a bit of a hack, but it does provide survivability between reboots.

Frequently Asked Questions

Please read the forked-daapd README first. You can find it here: https://github.com/jasonmc/forked-daapd/ (Scroll down the page.) The read the FAQ. You question might be answered here.

Q. I just installed forked-daapd according to the steps above and I can't see it with iTunes. What's wrong?
A. Maybe nothing. forked-daapd needs to build a database of all your music. This can take a substantial amount of time on embedded processors like the ARM. It's really a function of processor and disk speed. You should expect that forked-daapd can process about 10 gigabytes of music per hour. It won't be very responsive during this time. If you're in a hurry to see it work then there options for pointing it at a smaller directory of music. I'll describe that below.

Q. How can I be sure that forked-daapd is really running?
A. You can issue the following commands in a shell:

Code: Select all

pidof dbus-daemon
pidof avahi-daemon
pidof forked-daapd


These are the three daemons that are required for forked-daapd to run. You should get back three process ID numbers. (One for each pidof command.) If you don't then something is wrong.

Q. How do I make forked-daapd look at a directory other than /Qmultimedia?
A. The S50iTunes script in /opt/etc/init.d contains the path for where forked-daapd will look for music. You can change this line in the file:

Code: Select all

            directories = { "$SHARE_PATH" }

You can change this to point to wherever your library of music resides on your QNAP. eg:

Code: Select all

            directories = { "/share/foobar/my_music" }

A restart of forked-daapd is required after changing this.

Code: Select all

/opt/etc/init.d/forked-daapd restart

Q. Where is the forked-daapd log?
A. /opt/unstable/var/log/forked-daapd.log

Q. How do I increase/decrease the amount of logging that forked-daapd outputs?
A. The S50iTunes script in /opt/etc/init.d contains the loglevel parameter. The available levels are: fatal, log, warning, info, debug, spam. Spam will give you the most output but it will slow down forked-daapd considerably. It will be especially slow at scanning your media. A restart of forked-daapd is required after changing this.

Code: Select all

/opt/etc/init.d/forked-daapd restart

Q. I see a lot of ffmpeg output in the forked-daapd.log file. Is this okay?
A. Yes, usually it is. ffmpeg is pretty spammy. You can disable the ffmpeg domain or any other domain output by editing the following line in /opt/etc/init.d/S50iTunes:

Code: Select all

LOG_DOMAINS=config,daap,db,httpd,main,mdns,misc,rsp,scan,xcode,event,remote,dacp,ffmpeg,artwork,player,raop,laudio

A restart of forked-daapd is required after changing this.

Code: Select all

/opt/etc/init.d/forked-daapd restart

Q. My .wma and .ogg file playback stutters or is choppy. Why?
A. If you're on an ARM processor you don't have the necessary floating-point unit required by ffmpeg to decode .wma and .ogg files. All this has to be done in software and the ARM cpu isn't fast enough to play these files in realtime. Sorry, but there's no fix other than to asked the ffmpeg guys to provied fixed-point decoders for .wma and .ogg files. However, .mp3 and .flac do have fixed-point decoders in ffmpeg and should playback just fine on ARM processors. x86 QNAPs should not have this problem.

Q. How do I pair my iPhone with forked-daapd?
A. Did you read the README? It's in there. However, it's also tricky. Here's a reply where I try to give some insight into how to make pairing work more easily:
http://forum.qnap.com/viewtopic.php?f=177&t=37472&start=180#p187324

Q. When will you support Home Sharing and Air Video?
A. Whenever Julien decides it's a priority. (I'm not the author of forked-daapd.) This is a personal project that Julien does for his own enjoyment. Feature requested should be directed to Julien. His email is in the source code. If you want to add a feature please contact him and discuss it. He's definitely willing to take patches.

Q. Forked-daapd re-scans my music every time I restart it. Why?
A. It's a known issue that Julien will address in a later version of forked-daapd.

[ADoko]

Background Information

FORKED-DAAPD:
This is the iTunes streaming daemon and the reason for all this trouble. The /opt/etc/init.dS50iTunes script creates the forked-daapd.conf file from the data in the /etc/config/uLinux file. I borrowed heavily from the script that QNAP used to configure mt-daapd. forked-daapd claims to support Airport Express. I have left this commented out in the config file since this was how the package set it by default after compilation. To turn it on just uncomment those lines in the S50iTunes script and restart the service with /opt/etc/init.d/S50iTunes restart.

I've noticed that forked-daapd does a lot of logging. The log file seems to grow into the megabytes rather rapidly. I'm still playing with the amount of logging that I like. I'll repost a new script when I have something satisfactory. In the interim you can change the logging to "fatal" in the S50iTunes script to basically eliminate messages.

I have a library of over 60 thousand songs and it does seem to take a while to download the meta data to a client when you initially connect. The cpu also spikes to near 100% for about a minute while forked-daapd streams the meta data. There are discussions about this on the internet, but the author for forked-daapd has yet to respond.

A note about playlists (.m3u) files: forked-daapd scans these in wherever it finds them. My experience is that it does a very good job of processing them. I recommend reading up on them here:
http://en.wikipedia.org/wiki/M3U.
I haven't used .m3u files very much, but it seems promising to be able to place them in any directory. I should also confess that the playlists form in the Web UI is now useless since forked-daapd doesn't have a concept of a "playlist" file like mt-daapd did. If you want to use playlists you're best bet is to put the .m3u files with your albums or make a playlist directory under Qmultimedia.

When you run forked-daapd for the first time be prepared for a lot of CPU usage (~80%) and disk activity while it indexes your media. My experience was that it took about one hour per 10,000 files.

AVAHI-DAEMON:
This is a replacement for the mDNSResponderPosix daemon that ships with the QNAP. In my opinion it's a very robust and easy to use implementation of Apple's Bonjour/Zeroconf protocol. All of its configuration files are created with the S30bonjour script in /opt/etc/init.d. You can modify its behavior by modifiying this script. You can also advertise your own Bonjour services by placing service files in /opt/unstable/etc/avahi/services. These files are human readable XML and it's a pretty straightforward process to make them. I may get around to making one to advertise my CUPS printer using the ipp protocol. The S30bonjour script is written so that any manually added .service files won't be disrupted. The only .service files that S30bonjour will manage are the ones for services available through the web UI services page.

TIMEMACHINE:
I've moved most of the support for advertising time machine into the S30bonjour script, but there is still some support in the S35timemachine script.

DBUS:
I modified the S20dbus script because the one that came with ipkg didn't want to start the dbus-daemon. dbus-daemon is critical for the proper interaction of forked-daapd and avahi. forked-daapd is a true avahi-client and it uses dbus to notify avahi-daemon when it's running. It was a design decision to use the ipkg version of dbus. Part of the build process for forked-daapd required me to build the dbus package. I opted to not use the /opt/unstable version of dbus because users may already be using the ipkg dbus. (This was my case as I had installed saned via ipkg when I first got my QNAP NAS.)

SHIMS:
These are just pass-through scripts in /etc/init.d with some protection to prevent them from starting services if dbus-daemon isn't running or the Optware.sh script hasn't run.

/opt/unstable:
This is the alternate universe of libraries and binaries based on eGlibc 2.12. All the binaries and libraries in this tree are linked against /opt/unstable/lib/libc.so.6 and use the /opt/unstable/lib/ld-linux.so.3|2 to dynamically load the libraries they depend upon. This was all done because forked-daapd requires GNU libc 2.9 or greater and the GNU libc that ships with the QNAP is currently at version 2.5. I built the world for one application to live in it.

Surviving a Firmware Upgrade:
This is tested and working.

Here's a shot of iTunes running against forked-daapd on my TS-419P:


Optional Patches

I've decided to build small patches for the basic install in the post above. These are just builds of the forked-daapd daemon that pull in recent changes from the development branch. They are very small and easy to install after you've installed the one the packages above. The hope is that they may fix basic issues that people encounter without imposing the burden of uninstalling and then re-installing the full package.

How To Install:
These instructions are for the ARM patch. Installing the INTEL patch is similar with the exception of the download URL being different and name of the compressed tar file starting with INTEL- instead of ARM-.

For example, the following commands would download the 2010-12-13 ARM patch and install it over the existing forked-daapd:

Code: Select all

cd /share/Public
wget http://redacted
/opt/etc/init.d/S50iTunes stop
cd /opt
gunzip -c /share/Public/ARM-20101213-patch.tar.gz  | tar xvf -
/opt/etc/init.d/S50iTunes start


This will replace the old forked-daapd binary. No attempt is made to back it up. If you want to go back to the original then uninstall the full package and re-install it.

[EDIT]Nothing here right now. Use the "bleeding edge" builds.[/EDIT]

[CdG80]

Hi Anthony,

I really love your post, and also your Wiki about how to compile. Great job, that is what I have been looking for since I first read about forked-daapd.

I have a QNAP TS-239 Pro II which I would love to use it for music streaming to my Airtunes speakers (actually a BOSE 1-2-3 system). The streaming itself is not the problem, as long as my MAC (with the Itunes client) is on. Unfortunately, as everyone knows who is interested in the mt-daapd (firefly) vs. forked-daapd the built-in iTunes server in our NAS's (which is the firefly/mt-daapd flavour) is not capable of something like that.

So, I found a tarball for the forked-daapd whcih then has to be compiled on the NAS (or Linux machine). I tried to, applied all prereqs but could not get it running, as it always fails compiling the libunistring stuff. Then, as of today, I found your post and was real happy that there is someone else out there who has the same idea, the the forked-daapd running on his QNAP NAS. I was so happy... until I noticed you compiled the ARM version... and my NAS is the INTEL one.

I have not been able to compile forked-daapd for the INTEL flavour... as I said, always failing in the libunistring section. I am trying to compile on the NAS directory as I do not have an ubuntu or something.

Would it be possible for you to compile the Intel version as well? My NAS is at the latest available firmware level, which is Version 3.3.6 build 1110T. The kernel is

Code: Select all

Linux BLACKHOLE 2.6.33.2 #1 SMP Tue Nov 9 23:40:50 CST 2010 i686 unknown

Your feedback would be highly appreciated!

Keep up the brilliant work and have a great day!

Cheers, Chris

[ADoko]

Would it be possible for you to compile the Intel version as well? My NAS is at the latest available firmware level, which is Version 3.3.6 build 1110T. The kernel is

Code: Select all

Linux BLACKHOLE 2.6.33.2 #1 SMP Tue Nov 9 23:40:50 CST 2010 i686 unknown

Chris,

Getting this compiled for Intel x86 It's something I plan on doing. I don't think it will be that difficult, but I'm going to need to change my compilation scripts. I'm compiling on an Intel Core-i7 system and I believe there are gcc switches to target and optimize for the Atom family. The other problem will be that it's completely untestable by me since I don't have an Atom-based QNAP. I can build an unstable.tar file that should be a drop-in for the one inside my forked-daapd-install.tar.gz file. If the Atom-based QNAPs are setup like the ARM-based ones in terms of file system then the install scripts should work too. The burden will be on the Atom-based QNAP community to test this. I'll try and tailor the build to the Atom D410 (which I believe is what is in your NAS). One cool thing is that you will get support for SSE/2/3, MMX and floating point which might make the transcoding in ffmpeg go a lot faster.

Could you do me a favor and ssh/telnet into your QNAP and run this command:

Code: Select all

/lib/libc.so.6


Then post the resulting output here. This will help me know what I'm dealing with for a C library on that NAS.

I'll try getting this working this weekend if I have a spare moment between family, Christmas shopping and the normal pandemonium that seems to occupy my life.

Regards,

Anthony

[CdG80]

Hey Anthony,

thanks so much for your quick reply, and for your plans to compile a version for the Intel/ATOM flavoured QNAP NAS's.

The command output you asked for is as follows:

Code: Select all

[/] # /lib/libc.so.6
GNU C Library stable release version 2.6.1, by Roland McGrath et al.
Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 4.2.1 (Ubuntu 4.2.1-5ubuntu4).
Compiled on a Linux >>2.6.15.7<< system on 2007-10-24.
Available extensions:
   crypt add-on version 2.1 by Michael Glad and others
   GNU Libidn by Simon Josefsson
   Native POSIX Threads Library by Ulrich Drepper et al
   BIND-8.2.3-T5B
For bug reporting instructions, please see:
<http://www.gnu.org/software/libc/bugs.html>.
[/] #


This might not be the original one as I installed a couple of packages for the compiling... Maybe someone else with an "original" system can verify and post here if different.

I am willing and looking forward to test your build for the Atom D410 (which I confirm it is inside my NAS). If you need anything else, let me know. I will monitor this thread and sit here waiting for your build... amongst the christmas shopping, regular work and family business

Cheers, Chris

[ADoko]

Chris,

Thanks for the information. You're on v2.6 of GNU libc which still isn't sufficient for forked-daapd. This isn't a big problem since I know how to compile the binaries and libraries so that they'll use the version of libc in /opt/unstable/lib.

I'll check back in when I have something more substantial. Hopefully it's not as arduous a process to build for the ATOM as it was for the ARM. (I spent at least a week and a half figuring out how to build all the packages.)

Regards,

Anthony

[ADoko]

Here is the link to the tar of the INTEL build for forked-daapd:

[EDIT]Removed. Use the link in the first post of this thread.[/EDIT]

A few words of warning for everyone reading this:
1. Back up any data you don't want to lose prior to trying any of this!!!
2. You should back up your /opt directory before attempting any of this. If things go badly you can simply restore it and reboot. The firmware will re-write the changes I made to /etc/init.d and you will be back to where you started (ie. without my changes).

Code: Select all

cd /opt
cd `dirname \`pwd -LP\``
tar cf - Optware | ( cd /share/Public; tar xfp -)


If you don't have /share/Public then pick another place with enough space to hold the copy of the Optware directory.

3. As I've mentioned before: I don't assume any liability for damage to your data, software or QNAP device. You're on your own since I don't have an INTEL QNAP to test this on.

///

Chris,

I will probably rebuild this INTEL package because of a bug in my crossbuild scripts. It won't affect the execution of the code, but I was pointing things to the wrong ld-linux.so version. I fixed it with a symlink for the time being. When you report back on your experience I'll go ahead and rebuild.

-Anthony

[CdG80]

Hey Anthony,

great job, seems you didn't go for christmas shopping the last days?

So, I was able to install your INTEL build of the forked-daapd, so far so good. See the log below, to me it looks good:

Code: Select all

[/share/Download/forked-daapd-install] # ./package install nodebug
Backing up S20dbus script in /opt/etc/init.d...
mkdir -p /opt/etc/init.d/.ORIG
cp /opt/etc/init.d/S20dbus /opt/etc/init.d/.ORIG
done.

Install new scripts in /opt/etc/init.d...
chmod 755 ./opt_etc_initd_scripts/S15replace
cp ./opt_etc_initd_scripts/S15replace /opt/etc/init.d
chmod 755 ./opt_etc_initd_scripts/S20dbus
cp ./opt_etc_initd_scripts/S20dbus /opt/etc/init.d
chmod 755 ./opt_etc_initd_scripts/S30bonjour
cp ./opt_etc_initd_scripts/S30bonjour /opt/etc/init.d
chmod 755 ./opt_etc_initd_scripts/S35timemachine
cp ./opt_etc_initd_scripts/S35timemachine /opt/etc/init.d
chmod 755 ./opt_etc_initd_scripts/S50iTunes
cp ./opt_etc_initd_scripts/S50iTunes /opt/etc/init.d
done.

Running the S15replace script to install shims...
/opt/etc/init.d/S15replace
Shutting down bonjour services:cat: /var/lock/bonjour.normal: No such file or directory
cat: /var/lock/bonjour.timemachine: No such file or directory
 mDNSResponderPosix
cat: /var/lock/bonjour.timemachine: No such file or directory
Shutting down iTunes services:done.

/opt/.ORIG not found, creating it...
mkdir -p /opt/.ORIG
done.

Backing up Optware.sh in /opt...
mv /opt/Optware.sh /opt/.ORIG/Optware.sh
done.

Install new Optware.sh in /opt...
/share/Download/forked-daapd-install/opt_scripts /share/Download/forked-daapd-install
chmod 755 ./Optware.sh
cp ./Optware.sh /opt
/share/Download/forked-daapd-install
done.

Installing the unstable tree...
/opt /share/Download/forked-daapd-install
tar -xf /share/MD0_DATA/Download/forked-daapd-install/unstable.tar
/share/Download/forked-daapd-install
done.

No avahi user found, creating it...
done.

No avahi-autoipd user found, creating it...
done.

No netdev group found, creating it...
addgroup netdev
done.

No dbus socket was found, creating it...
mkdir -p /opt/unstable/var/run/dbus
ln -sf /opt/var/run/dbus/system_bus_socket /opt/unstable/var/run/dbus/system_bus_socket
ln -sf /opt/var/run/dbus/pid /opt/unstable/var/run/dbus/pid
done.

Creating /opt/unstable/var/log for forked-daapd.log...
mkdir -p /opt/unstable/var/log
done.

Finished installing.

[/share/Download/forked-daapd-install] #

So the installation completed successfully I would say. Then, to avoid the reboot, I tried to use 5b, starting the daemons manually, which resulted in an error complaining about the missing root user (I think the QNAP devices typically do not have a root user as it is called admin).

Code: Select all

[/share/Download/forked-daapd-install] # /opt/etc/init.d/S20dbus start
Starting system message bus: Unknown username "root" in message bus configuration file S20dbus.

I then played around and finally added a dummy root user which may not do anything valuable... just to be defined. That works. Then I started the daemons as described in step 5b, everything seems fine.

I opened my iTunes on the MAC and saw nothing showing up from my NAS (beside Time Machine and AFP in Finder, which work fine). I checked the log file /share/MD0_DATA/.qpkg/Optware/unstable/var/log/forked-daapd.log which basically showed me the following:

Code: Select all

[2010-12-03 09:50:34]     main: Forked Media Server Version 0.12 taking off
[2010-12-03 09:50:34]     main: mDNS init
[2010-12-03 09:50:34]     mdns: Avahi state change: Client running
[2010-12-03 09:50:34]       db: Could not step: unknown error
[2010-12-03 09:50:34]       db: Database version check errored out, incompatible database
[2010-12-03 09:50:34]     main: Database init failed
[2010-12-03 09:50:34]     main: mDNS deinit
[2010-12-03 09:50:34]     main: Exiting.

So, this is where I am now. I googled and tried to find any useful hints but could not. My understanding is that this is not the iTunes DB but the SQLite probably? My iTunes library was last touched by iTunes 10.1 (54) on my MAC. The sqlite version on my nas is 3.7.3-1.

Do you have any idea how we can get this working? I'm out of ideas... Please let me know if you need any outputs and THANKS AGAIN for your great efforts!

Cheers, Chris

[ADoko]

Chris,
[EDIT]
I think that your /opt/etc/dbus-1/system.conf file is wrong. I'm betting the ARM and INTEL versions of ipkg dbus differ. There's a line in my system.conf file:

Code: Select all

  <!-- Run as special user -->
  <user>messagebus</user>


I bet yours says:

Code: Select all

  <!-- Run as special user -->
  <user>root</user>


I suspect you could change it to what mine says and that would rid you of the message. If that does fix it then it might be wise to remove that root user you created. I'm not sure what privileges you gave it, but the fewer user accounts the better is my motto.
[/EDIT]

Let's first look at why your S20dbus script was wanting there to be a user called root. You can do a simple check and see if your /opt/etc/init.d/S20dbus is the same as the one in forked-daapd-install/opt_etc_initd_scripts/S20dbus.

Code: Select all

ipkg install diffutils
cd /share/Public/forked-daapd-install/opt_etc_initd_scripts
diff ./S20dbus /opt/etc/init.d/S20dbus


Could you post the output of this command:

Code: Select all

/bin/cat /opt/unstable/etc/forked-daapd.conf


Let's do a few more sanity checks. Make sure the return values of these commands are empty strings:

Code: Select all

pidof mDNSResponderPosix
pidof mt-daapd


Post the output of these commands:

Code: Select all

/opt/unstable/bin/ldd /opt/unstable/sbin/forked-daapd
/opt/unstable/bin/ldd /opt/unstable/lib/libsqlite3.so.0


Make sure you have an /opt/unstable/var/cache/forked-daapd directory. My install script didn't actually set this up. You could also delete the songs3.db.

Code: Select all

/opt/etc/init.d/S50iTunes stop
rm /opt/unstable/var/cache/forked-daapd/songs3.db
/opt/etc/init.d/S50iTunes start


Last of all, you can modify the /opt/etc/init.d/S50iTunes script. There's a line in the script:

Code: Select all

loglevel = log

change it to

Code: Select all

loglevel = debug

Then issue:

Code: Select all

/opt/etc/init.d/S50iTunes stop
rm  /opt/unstable/var/log/forked-daapd.log
/opt/etc/init.d/S50iTunes start


Now you can look at the output of the log and see if anything interesting shows up. If not then repeat this process but set loglevel = spam instead.

Let me know how things go.

-Anthony

[ADoko]

Chris,

I did a little fiddling this morning with the INTEL package and got it to run on my VirtualBox build machine:


This is proof that the sqlite library does indeed work with forked-daapd. I did have a bit of trouble with some file permissions, but that was because of Ubuntu and shouldn't affect you on the QNAP. At this point I'm strongly suspicious of your songs3.db file in /opt/unstable/var/cache/forked-daapd. Maybe you copied the mt-daapd songs3.db file to that directory? In any case just delete the songs3.db file and try again.

[EDIT] Another piece of advice would be to start with a very small music directory. You can control this by editing the S50iTunes script and changing:

Code: Select all

directories = { "$SHARE_PATH" }

to

Code: Select all

directories = "/your/temporary/small/music/directory"

[/EDIT]
-Anthony

[CdG80]

Hey Anthony,

thank you so much for your help on this! Finally I found out what was causing the problems... I commented in the ApEx section in the S50iTunes script which then writes the forked-daapd.conf file. Unfortunately I forgot to comment in the closing curly bracket So, the problem was in front of the machine

Now, after having the songs3.db file deleted and the NAS rebooted, the iTunes server is showing up in my MAC's iTunes! Jippeh! Anyways, I looked at the forked-daapd.log file and noticed a lot of errors in there, but most of these coming from ffmpeg... Also, it is reporting that it cannot work with my Remote app... which seems to be due to the documented problems when using Remote App 2.x on iPhone. What about the other errors right after startup?

Code: Select all

[2010-12-03 14:06:55]     main: Forked Media Server Version 0.12 taking off
[2010-12-03 14:06:55]     main: mDNS init
[2010-12-03 14:06:55]     mdns: Avahi state change: Client running
[2010-12-03 14:06:55]       db: Could not prepare statement: no such table: admin
[2010-12-03 14:06:55]       db: Could not check database version, trying DB init
[2010-12-03 14:06:58]     raop: Couldn't make timing socket: Address family not supported by protocol
[2010-12-03 14:06:58]     raop: Couldn't make control socket: Address family not supported by protocol
[2010-12-03 14:07:00]   remote: Remote 'iChristian' not known from mDNS, ignoring
[...]
[2010-12-03 14:32:41]   ffmpeg: Error at MB: 1260
[2010-12-03 14:32:42]   ffmpeg: H263 SAC not supported
[2010-12-03 14:32:42]   ffmpeg: header damaged
[2010-12-03 14:32:42]   ffmpeg: Bad marker
[2010-12-03 14:32:42]   ffmpeg: header damaged
[2010-12-03 14:32:42]   ffmpeg: Bad H263 id
[2010-12-03 14:32:42]   ffmpeg: header damaged
[2010-12-03 14:32:42]   ffmpeg: Bad marker
[2010-12-03 14:32:42]   ffmpeg: header damaged
[2010-12-03 14:32:42]   ffmpeg: Bad marker
[2010-12-03 14:32:42]   ffmpeg: header damaged
[2010-12-03 14:32:42]   ffmpeg: picture size invalid (0x0)
[2010-12-03 14:32:42]   ffmpeg: get_buffer() failed (-1 0 0 (nil))
[2010-12-03 14:32:42]   ffmpeg: Bad marker
[2010-12-03 14:32:42]   ffmpeg: header damaged
[2010-12-03 14:32:42]   ffmpeg: Bad marker
[2010-12-03 14:32:42]   ffmpeg: header damaged
[2010-12-03 14:32:42]   ffmpeg: Bad marker
[2010-12-03 14:32:42]   ffmpeg: header damaged
[2010-12-03 14:32:42]   ffmpeg: warning: first frame is no keyframe
[2010-12-03 14:32:42]   ffmpeg: illegal ac vlc code at 2x1
[2010-12-03 14:32:42]   ffmpeg: Error at MB: 11
[2010-12-03 14:32:42]   ffmpeg: run overflow at 6x1 i:0
[2010-12-03 14:32:42]   ffmpeg: Error at MB: 15
[2010-12-03 14:32:42]   ffmpeg: illegal ac vlc code at 3x1
[2010-12-03 14:32:42]   ffmpeg: Error at MB: 12
[2010-12-03 14:32:42]   ffmpeg: illegal ac vlc code at 4x2
[2010-12-03 14:32:42]   ffmpeg: Error at MB: 22
[2010-12-03 14:32:42]   ffmpeg: illegal ac vlc code at 5x0
[2010-12-03 14:32:42]   ffmpeg: Error at MB: 5
[2010-12-03 14:33:56]   ffmpeg: Header missing


I guess it complains about my mp3, m4a, m4b and m4r files. Also, it does not distinguish between these formats, which I would have thought it does at least for the mp3/m4a (music) vs. m4b (audiobook) vs. m4r (ringtones). Probably this will be done in a future release .-)

Btw, the ldd commands did not work with the unstable directory path, had to use /sbin/ldd instead...

Code: Select all

[/] # /opt/unstable/bin/ldd /opt/unstable/sbin/forked-daapd
-sh: /opt/unstable/bin/ldd: /bin/bash: bad interpreter: No such file or directory

[/] # which ldd
/sbin/ldd

[/] # /sbin/ldd /opt/unstable/sbin/forked-daapd
        linux-gate.so.1 =>  (0xb77ab000)
        libz.so.1 => /opt/unstable/lib/libz.so.1 (0xb7795000)
        libavahi-common.so.3 => /opt/unstable/lib/libavahi-common.so.3 (0xb7789000)
        libavahi-client.so.3 => /opt/unstable/lib/libavahi-client.so.3 (0xb7779000)
        libsqlite3.so.0 => /opt/unstable/lib/libsqlite3.so.0 (0xb76f2000)
        libavcodec.so.52 => /opt/unstable/lib/libavcodec.so.52 (0xb6d84000)
        libavformat.so.52 => /opt/unstable/lib/libavformat.so.52 (0xb6cc6000)
        libswscale.so.0 => /opt/unstable/lib/libswscale.so.0 (0xb6c93000)
        libavutil.so.50 => /opt/unstable/lib/libavutil.so.50 (0xb6c7f000)
        libconfuse.so.0 => /opt/unstable/lib/libconfuse.so.0 (0xb6c74000)
        libFLAC.so.8 => /opt/unstable/lib/libFLAC.so.8 (0xb6c26000)
        libm.so.6 => /opt/unstable/lib/libm.so.6 (0xb6c01000)
        libevent_core-1.4.so.2 => /opt/unstable/lib/libevent_core-1.4.so.2 (0xb6bf8000)
        libavl.so.1 => /opt/unstable/lib/libavl.so.1 (0xb6bf5000)
        libmxml.so.1 => /opt/unstable/lib/libmxml.so.1 (0xb6beb000)
        libpthread.so.0 => /opt/unstable/lib/libpthread.so.0 (0xb6bd2000)
        libantlr3c.so => /opt/unstable/lib/libantlr3c.so (0xb6bbe000)
        libplist.so.1 => /opt/unstable/lib/libplist.so.1 (0xb6bb6000)
        libxml2.so.2 => /opt/unstable/lib/libxml2.so.2 (0xb6a9f000)
        libglib-2.0.so.0 => /opt/unstable/lib/libglib-2.0.so.0 (0xb69b9000)
        libgcrypt.so.11 => /opt/unstable/lib/libgcrypt.so.11 (0xb6947000)
        libgpg-error.so.0 => /opt/unstable/lib/libgpg-error.so.0 (0xb6942000)
        libasound.so.2 => /opt/unstable/lib/libasound.so.2 (0xb6885000)
        libunistring.so.0 => /opt/unstable/lib/libunistring.so.0 (0xb677a000)
        libc.so.6 => /opt/unstable/lib/libc.so.6 (0xb6640000)
        librt.so.1 => /opt/unstable/lib/librt.so.1 (0xb662a000)
        libdbus-1.so.3 => /opt/unstable/lib/libdbus-1.so.3 (0xb65f1000)
        libdl.so.2 => /opt/unstable/lib/libdl.so.2 (0xb65ed000)
        libogg.so.0 => /opt/unstable/lib/libogg.so.0 (0xb65e7000)
        /opt/unstable/lib/ld-linux.so.3 (0xb77ac000)
        libnsl.so.1 => /opt/unstable/lib/libnsl.so.1 (0xb65d1000)
        libresolv.so.2 => /opt/unstable/lib/libresolv.so.2 (0xb65bc000)
[/] #

[/] # /sbin/ldd /opt/unstable/lib/libsqlite3.so.0
        linux-gate.so.1 =>  (0xb7776000)
        libdl.so.2 => /opt/unstable/lib/libdl.so.2 (0xb76e9000)
        libpthread.so.0 => /opt/unstable/lib/libpthread.so.0 (0xb76d1000)
        libc.so.6 => /opt/unstable/lib/libc.so.6 (0xb7596000)
        /lib/ld-linux.so.2 (0xb7777000)
[/] #



The process is still scanning my files, I hear the harddisks being quite busy.

So, Anthony, you're my hero! forked-daapd is now working on my NAS! I just consider building it up from scratch, then reinstall forked-daapd, as I would like to get rid of the tons of ** I installed while trying to compile the stuff.

What was it with the repackaging due to the wrong ld-linux.so version? Is that related to the problem I had with ldd?

I just stopped the daemon and restarted it using the S50iTunes stop/start script- but errors are still showing up from ffmpeg, raop and remote. Now, need to figure out how streaming to my Airport Express speakers can work if I cannot use Remote 2.x.

For the system.conf file, I have the very same entry like you:

Code: Select all

  <!-- Run as special user -->
  <user>messagebus</user>

  <!-- Fork into daemon mode -->
  <fork/>


Btw, once there will be an update to forked-daapd itself, how could that be applied? Does it again required a new build like you did it now? Or is it just replacing some files and that's it?

Cheers, Chris

[ADoko]

Chris,

Very nice to see you got it working. Now I can go Christmas shopping!

I also have the same raop and ffmpeg errors that you have. I suspect it's because ffmpeg doesn't like something about the format of the .mp4/.m4a etc files. One of the troubles with this entire install is that I had to rely on bleeding edge versions of libraries when building them for forked-daapd. The author of forked-daapd made a note that ffmpeg-0.6.1 worked better than the -0.5.x version of ffmpeg. This is what I grabbed, but it's certainly still a work in progress. It could be that there are bugs in any of the libraries I built. It could be that I built the library wrong. (I really worry about GNU libc because it is a real chore to build, and required me to patch a few files.)

Here's the note in the README in forked-daapd about Streaming MPEG4:

Streaming MPEG4
---------------

Depending on the client application, you may need to optimize your MPEG4 files
for streaming. Stream-optimized MPEG4 files have their metadata at the beginning
of the file, whereas non-optimized files have them at the end.

Not all clients need this; if you're having trouble playing your MPEG4 files,
this is the most probable cause. iTunes, in particular, doesn't handle files
that aren't optimized, though FrontRow does.

Files produced by iTunes are always optimized by default. Files produced by
FAAC and a lot of other encoders are not, though some encoders have an option
for that.

The mp4creator tool from the mpeg4ip suite can be used to optimize MPEG4 files,
with the -optimize option:
$ mp4creator -optimize foo.m4a

Don't forget to make a backup copy of your file, just in case.

Note that not all tag/metadata editors know about stream optimization and will
happily write the metadata back at the end of the file after you've modified
them. Watch out for that.

The remote issues are something I can't really help with as I don't have an iPhone/iPod for testing with forked-daapd. It seems like a known issue, but you could check the readme file here for more information:
https://github.com/jasonmc/forked-daapd/blob/master/README

I think you can safely ignore the raop errors for now. They pertain to IPv6 failures and it's unlikely you're using IPv6 in your home network. If you set the log level to spam you'd see they were IPv6. I consider this a minor bug that they are printed at all and especially without the IPv6 context.

If you want to be rid of the ffmpeg errors you can use my S50iTunes script which controls the logging domains and turns off logging of ffmpeg:
http://rapidshare.com/files/434685042/S50iTunes

Just play with the LOG_DOMAINS variable until you have what you want.

What was it with the repackaging due to the wrong ld-linux.so version? Is that related to the problem I had with ldd?

No, I'm not sure why that didn't work. ldd is just a bash script. The one in /opt/unstable/bin runs on my ARM QNAP. It may be you have a broken link in /bin. On my ARM QNAP /bin/bash is a link to /bin/sh which is a link to busybox. busybox runs a modified version of bash called ash. Anyway...

The issue with ld-linux.so is that my crossbuild scripts were specifying ld-linux.so.3 which is what I used on ARM. For INTEL there is only ld-linux.so.2. So all the libraries and binaries were looking for ld-linux.so.3 and not finding it and refusing to load/run. Since ld-linux.so.2 is just a link to ld-2.9.so I just made another link from ld-linux.so.3 to ld-2.9.so and things started working. I'll fix my crossBuild scripts so this is done correctly for INTEL.

Btw, once there will be an update to forked-daapd itself, how could that be applied? Does it again required a new build like you did it now? Or is it just replacing some files and that's it?

This should only require me to rebuild the forked-daapd package and nothing more. Ideally I'll provide a write up for how people can do that on a virtual machine like I did for the ARM build. I'm still in the process of cleaning things up, but the only differences are the crossBuild scripts and the .config file for the crosstools-ng package. Everything else is the same (except for INTEL you don't need to build the GNU libc ports package).

It would be even cleaner if I could provide a gcc and binutils that run on the INTEL and ARM packages and use /opt/unstable as their root. That way people could build on the QNAP against the libraries in /opt/unstable.

The best solution would be to get QNAP to migrate their GNU libc from v2.5 to v2.9. Then all this craziness would go away. However, that's unlikely because there is a huge risk in changing from 2.5 to 2.9 in terms of stability. I work on embedded systems and you don't want to rock the boat when it comes to GNU libc versions. An old working version with fewer features is preferred over a newer less stable version that may have some new features.

So, Anthony, you're my hero! forked-daapd is now working on my NAS! I just consider building it up from scratch, then reinstall forked-daapd, as I would like to get rid of the tons of ** I installed while trying to compile the stuff.

Thanks. If your really want to go back to a clean system I think you can just follow this:
http://wiki.qnap.com/wiki/Uninstall_Optware_IPKG
Just be warned that this will remove the songs3.db file too since it's under /opt. If you want to save it then copy it to a safe place like /share/Public before you remove /opt. Then you can just put it back prior to restarting forked-daapd and you won't have to wait for it to re-index all your media.

One more thing...
It looks like you did a reboot and the forked-daapd started up correctly. If so this is cool!

Happy Holidays!
-Anthony

[CdG80]

Hey Anthony,

sorry for not getting back earlier... too busy these days

I decided to complete build up my NAS from scratch, including redesign of the volume layout. Previously I used my 2*2TB disks as JBOD and now I decided to get back to RAID1. So, moved everything off to other disks, resetted the entire box and am now moving back all desired files after I changed the disk layout and initialized/synced the RAID. Everything doing very fine at the moment...

I have not yet been able to apply IPKG, and some other packages, like I also have not been able to apply the your forked-daapd build yet. Will do so after my media files are back on.

And, yes, I rebooted the box quite a few times prior to resetting it and your forked-daapd build survived each reboot! Great job though!!

Regarding the remote problem (menaing using the Apple Remote App for ipad or iPhone) I found out that this a bug which is related to version 2.x of the Apple Remote App. Version 1 of that App worked fine with the forked-daapd 0.12 but version 2 does not because Apple changed "something". I was informed that Julien has been working on that and already included a fix in the new source code. This is either available on the web as sources (and could be compiled as needed) or will be released with the 0.13 version of forked-daapd. See following discussion: https://github.com/jasonmc/forked-daapd/issues/issue/23/#issue/23/comment/588851

Do you see any chance to build a new version using the sources as described in the other discussion? I guess there would not be much difference to building it for the 0.13 version, right? And you talked about correcting your CrossBuild scripts, so if you like to test it with the new source code... I'll be happy to test a new build

Thanks a million again for all your efforts! Hope you've been successful in Christmas Shopping?

Cheers, Chris

[ADoko]

Hey Chris,

It's done. It actually took me a little bit to figure out how to build the sources from the git repository. I had a few things working against me, namely my own stupidity, and also that I didn't read the build instructions specifying that the antlr3.jar has to match the antlr3 C-runtime. I also had to fight my overzealous stripping of symbols from object files which was rendering them unlinkable. **Mental note**: strip is not to be used haphazardly.

Eventually I got it to compile.

I'm prepared some new packages that remove a lot of files that are only needed if you plan on compiling against the libraries. There's no sense in shipping all that stuff if most people just want to run the forked-daapd daemon. The stripped down package is around 30MB while the unstripped is hovering between 130 and 160MB.

Anyway, you get to be the guinea pig here because I don't have an iPhone or iPod touch/iPad. After all this work I may just go out and get an Airport Express to see if all this was worth it.

I've put the links in the first post. Enjoy.

Regards,
Anthony

P.S. Almost done with the Christmas shopping!

[webdevnz]

DBUS:
I modified the S20dbus script because the one that came with ipkg didn't want to start the dbus-daemon. dbus-daemon is critical for the proper interaction of forked-daapd and avahi. forked-daapd is a true avahi-client and it uses dbus to notify avahi-daemon when it's running. It was a design decision to use the ipkg version of dbus. Part of the build process for forked-daapd required me to build the dbus package. I opted to not use the /opt/unstable version of dbus because users may already be using the ipkg dbus. (This was my case as I had installed saned via ipkg when I first got my QNAP NAS.)

Hi there.
What modifications did you make and how where they implemented in order for this to start, I'm having the same issue.
Cheers