[HOWTO] SpiderOak Online Backup

Backup, Restore, Netbak Replicator, Cloud Storage Services
ChillyWilly
Starting out
Posts: 21
Joined: Fri Jun 12, 2009 4:02 am

Re: [HOWTO] SpiderOak Online Backup

Post by ChillyWilly » Thu Mar 19, 2015 3:14 pm

I have moved away from using SpiderOak. Dealing with the supported backup systems is a lot easier.

datatrack
New here
Posts: 4
Joined: Fri Mar 20, 2015 6:31 pm

Re: [HOWTO] SpiderOak Online Backup

Post by datatrack » Fri Mar 20, 2015 7:10 pm

Hello Chilly Willy, thanks for sharing this nice post. This is very useful because it supports various OS.

friday
New here
Posts: 9
Joined: Wed Sep 16, 2015 11:01 pm

Re: [HOWTO] SpiderOak Online Backup

Post by friday » Wed Dec 02, 2015 5:39 am

Revised and simplified 2015 instructions for SpiderOakONE (yes, that is the official casing) v6
(No Debian or online form required)

Requirements:
X86 CPU (not ARM)
Qnapware: http://apps.qnap.community/11-community/10-qnapware (Update: Qnapware has now been replaced by Entware-ng: https://apps.qnap.community/11-community/10-entware-ng, which I haven't tested myself)

Some parts stolen from the original post by ChillyWilly. Thank you! I've only tested it on my QTS 4.2 TS-251. If you want to adapt it to work with an alternative to Qnapware you may have to change "Apps/opt" to something else. You also need to replace "/share/Download/" if you don't have one.

Follow these commented instructions:

Code: Select all

# Install prerequisites
opkg install python ar

# Create temporary directory for downloading + extracting and cd to it
mkdir /share/Download/spideroak-tmp && cd $_

# Download deb package (32 bit, not 64)
wget --no-check-certificate --output-document=spideroak.deb "https://spideroak.com/getbuild?platform=ubuntu&arch=i386"

# Extract from deb (ar and tar.gz)
ar x spideroak.deb data.tar.gz && tar -zxvf data.tar.gz

# create a conf dir
mkdir opt/SpiderOakONE/conf

# move to /Apps
mv usr/bin/SpiderOakONE /Apps/opt/bin/spideroak
mv opt/SpiderOakONE /Apps/opt/

# delete temp directory
rm -r /share/Download/spideroak-tmp


After that you need to edit the bash script. If you're not comfortable using vi you can "opkg install nano" and use it instead

Code: Select all

vi /Apps/opt/bin/spideroak


Change all references to "/opt" to "/Apps/opt" and add this code after the first line:

Code: Select all

if [ ! -d "/root/.config/SpiderOakONE" ]; then
    mkdir -p /root/.config/
    ln -s /Apps/opt/SpiderOakONE/conf /root/.config/SpiderOakONE
fi


Setup your account (this will prompt you for details and download prior deduplication data if available, which can take a while)

Code: Select all

spideroak --setup=-


Now help yourself using https://spideroak.com/faq/how-can-i-use ... mmand-line or

Code: Select all

spideroak --help


This is pretty much what you need to get started (-v for "verbose" isn't required, but helps):

Code: Select all

spideroak -v --include-dir=/path/to/directory
spideroak -v --batchmode


Add "spideroak --batchmode" to crontab to create a persistent job: http://wiki.qnap.com/wiki/Add_items_to_crontab

Also, you are likely to run into a problem because Spideroak uses /tmp, which is a small ramdisk. If you have enough ram, you can increase the size of the mount (recommended). For 512mb that would be:

Code: Select all

mount -o remount,size=512m /tmp

If you haven't got enough ram you might be able to do this, but it's risky: http://forum.qnap.com/viewtopic.php?t=44541

Either one needs to run after every reboot: http://wiki.qnap.com/wiki/Running_Your_ ... at_Startup
Last edited by friday on Mon Nov 14, 2016 6:36 am, edited 1 time in total.

unstatusthequo
New here
Posts: 7
Joined: Sat Mar 14, 2009 10:04 am

Re: [HOWTO] SpiderOak Online Backup

Post by unstatusthequo » Sat May 21, 2016 9:42 am

friday, nice instructions. Though I run into a strange issue where when I try to copy the folders to /Apps/* it basically fills the RAMdisk and then everything goes to **. Verified before and after with df -h

How did you avoid this issue?

friday
New here
Posts: 9
Joined: Wed Sep 16, 2015 11:01 pm

Re: [HOWTO] SpiderOak Online Backup

Post by friday » Sun Jun 19, 2016 11:56 pm

unstatusthequo wrote:friday, nice instructions. Though I run into a strange issue where when I try to copy the folders to /Apps/* it basically fills the RAMdisk and then everything goes to **. Verified before and after with df -h

How did you avoid this issue?


Thank you,

/Apps/ is ramdisk. Don't put anything here, as it will be wiped on every reboot. Use /Apps/opt/ as in the instructions.
/Apps/opt should be a symlink to /share/CACHEDEV1_DATA/.qpkg/Qnapware/opt (varies depending on your NAS model and possibly firmware version) set up by Qnapware. Using this is easier than setting up your own symlink, as you don't have to create another symlink and set up path (so you can run "spideroak" rather than "/path/to/spideroak" later on).

You can test this with "ls -lh /Apps". The row with "opt" should have an arrow pointing to the the symlinked directory.

Sorry for the late reply. I didn't get any notification.

chiavelli
New here
Posts: 2
Joined: Mon Jun 26, 2017 3:30 am

Re: [HOWTO] SpiderOak Online Backup

Post by chiavelli » Mon Jun 26, 2017 5:17 am

UPDATED AND VERIFIED FOR JUNE 2017

My Setup:
TS-453a
Version 4.3.3.0210

All of this was cobbled together from various other posts in this forum, primarily:
ChillyWilly's initial SpiderOak instructions
- viewtopic.php?f=15&t=31037#p133614
friday's updated 2015 SpiderOak instructions
- viewtopic.php?f=15&t=31037&start=15#p520113
CasaP's instructions for setting up autorun instructions on newer QNAP models
- viewtopic.php?t=112970


1. Add QNAP Club Store to App Center
The QNAP Club store is needed to install Entware-ng, which is no longer available through the official QNAP App Center
Follow the instructions here: https://qnapclub.eu/index.php?act=howto
I recall there being some differences between my interface and the one pictured in that link, but it was close enough that it was easy to figure out.


2. Install Entware-ng
In App Center, look for Entware-ng under the list of QNAP Club apps and install it.
Entware-ng is a package management system which will allow you to install Python in the next step

3. Launch SSH Session
For the remaining steps you will need to be logged into your QNAP on an SSH session.
I used PuTTY.

4. Download and install SpiderOakONE

Code: Select all

# Install prerequisites
opkg install python ar

# Create temporary directory for downloading + extracting and cd to it
mkdir /share/Download/spideroak-tmp && cd $_

# Download deb package (32 bit, not 64)
wget --no-check-certificate --output-document=spideroak.deb "https://spideroak.com/getbuild?platform=ubuntu&arch=i386"

# Extract from deb (ar and tar.gz)
ar x spideroak.deb data.tar.gz && tar -zxvf data.tar.gz

# create a conf dir
mkdir opt/SpiderOakONE/conf

# move to /opt
mv usr/bin/SpiderOakONE /opt/bin/spideroak
mv opt/SpiderOakONE /opt/

# delete temp directory
rm -r /share/Download/spideroak-tmp



5. Edit SpiderOak bash file
A Spideroak symlink needs to be added to the system's /root/.config folder. Your QNAP will wipe this symlink out every time the system boots, so we are going to add the symlink creation to the SpiderOak bash file, ensuring it is always there when we need it.

Open the bash script for editing. If you are uncomfortable using vi, read tutorials on how to use vi.

Code: Select all

vi /opt/bin/spideroak


Add this code after the first line to create the symlink

Code: Select all

if [ ! -d "/root/.config/SpiderOakONE" ]; then
    mkdir -p /root/.config/
    ln -s /opt/SpiderOakONE/conf /root/.config/SpiderOakONE
fi


6. Setup your account
This will prompt you for account credentials. Once it gets going it will take awhile so be patient.

Code: Select all

spideroak --setup=-

Once complete, enter this command to look at how your account has been set up on your QNAP:

Code: Select all

spideroak --selection


7. Increase size of mount for /tmp directory
You are likely to run into a problem because Spideroak uses /tmp, which is a small ramdisk. If you have enough ram, you can increase the size of the mount (recommended). For 512mb that would be:

Code: Select all

mount -o remount,size=512m /tmp


You can verify the size of your mount like so:

Code: Select all

df -T /tmp


This mount will need to be done every time your system boots. I added the mount to my system's autorun.sh via the following steps

NOTE: These steps may not work for your QNAP model, especially if you have an older one. You may be able to find some direction here: https://wiki.qnap.com/wiki/Running_Your ... at_Startup

Open autorun file in vi

Code: Select all

mount $(/sbin/hal_app --get_boot_pd port_id=0)6 /tmp/config
touch /tmp/config/autorun.sh
chmod +x /tmp/config/autorun.sh
vi /tmp/config/autorun.sh

Add this line to the file and save/exit vi:

Code: Select all

mount -o remount,size=512m /tmp

Unmount the file:

Code: Select all

umount /tmp/config



8. Add some directories to backup!
This is pretty much what you need to get started (-v for "verbose" isn't required, but helps):

Code: Select all

spideroak -v --include-dir=/path/to/directory

Check what you have selected for backup:

Code: Select all

spideroak --selection



9. Back Stuff up!
Trigger a backup like so:

Code: Select all

spideroak -v --batchmode


If you do not want to always manually trigger your backups you should add "spideroak --batchmode" to crontab to create a persistent job: http://wiki.qnap.com/wiki/Add_items_to_crontab



And, uhh... that's it! Hopefully these updated steps can help save the rest of you some headaches.

gilsn
First post
Posts: 1
Joined: Tue Sep 19, 2017 4:54 am

Re: [HOWTO] SpiderOak Online Backup

Post by gilsn » Tue Sep 19, 2017 5:05 am

Hello!

I used the last post from chiavelli to setup spideroak on my TS-251+. Thanks a lot!!!

It works perfectly fine when I run it manually. But I can't make it work with cron.
I get the following error from spideroak :
"Command line arguments not allowed during New User Setup".

I read it may be a problem with the config file in the root directory. But the symlink looks ok, and again, when I run manually it works.

I don't get the difference with cron.

Any ideas?

Thank you in advance!

friday
New here
Posts: 9
Joined: Wed Sep 16, 2015 11:01 pm

Re: [HOWTO] SpiderOak Online Backup

Post by friday » Tue Sep 19, 2017 10:37 am

It seems like you don't have the spideroak directory in home http://lukieb.blogspot.se/2013/03/comma ... lowed.html so I'm guessing either that directory doesn't exist at that time, or the command isn't run as the right user. Normally this would happen because you ran "sudo crontab -e" instead of just "crontab -e", but since busybox/qnap isn't set up like that I could only guess.

gverweijen
New here
Posts: 3
Joined: Mon May 25, 2015 4:39 am

Re: [HOWTO] SpiderOak Online Backup

Post by gverweijen » Sun Jan 21, 2018 2:23 am

Great tutorial! Thanks for taking the time to set this up.
I've been following step by step, but I get stuck at step 6(setup your account). I suspect something went wrong before, though I've followed every step to the letter.
The error I get is this:

Code: Select all

[/share/Download/spideroak-tmp] # spideroak --setup=-
/opt/bin/spideroak: line 11: /opt/SpiderOakONE/lib/SpiderOakONE: No such file or directory
/opt/bin/spideroak: line 11: /opt/SpiderOakONE/lib/SpiderOakONE: Success

It is erroring on the last line of /opt/bin/spideroak which is:

Code: Select all

exec "/opt/SpiderOakONE/lib/SpiderOakONE" "$@"


It seems that everything is there:

Code: Select all

[~] # cd /opt/SpiderOakONE//lib/
[/opt/SpiderOakONE/lib] # ls
array.so             _hashlib.so         OpenSSL.crypto.so
bcrypt._bcrypt.so          _heapq.so         OpenSSL.rand.so
binascii.so             _hotshot.so         OpenSSL.SSL.so
_bisect.so             i18n/            operator.so
BTrees._fsBTree.so          inotify_dir_watcher*   parser.so
BTrees._IFBTree.so          _io.so            persistent.cPersistence.so
BTrees._IIBTree.so          itertools.so         persistent.cPickleCache.so
BTrees._IOBTree.so          _json.so            persistent._timestamp.so
BTrees._LFBTree.so          libcrypto.so.1.0.0     Pillow-2.8.1-py2.7-linux-i686.egg
BTrees._LLBTree.so          libcurl.so.4         plugins/
BTrees._LOBTree.so          libdbusmenu-qt.so.2    pycurl.so
BTrees._OIBTree.so          libexpat.so.0         pyexpat.so
BTrees._OLBTree.so          libffi.so.6         PyQt4.QtCore.so
BTrees._OOBTree.so          libgcc_s.so.1         PyQt4.QtDeclarative.so
bz2.so                libgmp.so.3         PyQt4.QtGui.so
_cffi_backend.so          libidn.so.11         PyQt4.QtNetwork.so
_codecs_cn.so             libpng12.so.0         _random.so
_codecs_hk.so             libQtCore.so.4         resource.so
_codecs_iso2022.so          libQtDBus.so.4         select.so
_codecs_jp.so             libQtDeclarative.so.4  setuptools-4.0.1-py2.7.egg/
_codecs_kr.so             libQtGui.so.4         simplejson._speedups.so
_codecs_tw.so             libQtNetwork.so.4      sip.so
_collections.so             libQtScript.so.4       _socket.so
cPickle.so             libQtSql.so.4         SpiderOakONE*
Crypto.Cipher.AES.so          libQtSvg.so.4         SpideroakVersionMatcher-1.1-py2.7-linux-i686.egg/
Crypto.Cipher.DES3.so          libQtXmlPatterns.so.4  _sqlite3.so
Crypto.Cipher.XOR.so          libQtXml.so.4         _ssl.so
Crypto.Hash.SHA256.so          library.zip         strop.so
Crypto.PublicKey._fastmath.so  libsodium.so         _struct.so
Crypto.Util._counter.so        libsqlite3.so.0         termios.so
Crypto.Util.strxor.so          libssh2.so.1         time.so
cStringIO.so             libssl.so.1.0.0         twisted.internet._sigchld.so
_csv.so                libstdc++.so.6         twisted.protocols._c_urlarg.so
_ctypes.so             libz.so.1         twisted.python._initgroups.so
datetime.so             _locale.so         unicodedata.so
dl.so                _lsprof.so         zlib.so
fcntl.so             math.so            zope.interface._zope_interface_coptimizations.so
_functools.so             mmap.so
grp.so                _multibytecodec.so


anyone any ideas how to proceed?
I've got a QNAP TS-469 pro
Current firmware version: 4.3.4.0435

friday
New here
Posts: 9
Joined: Wed Sep 16, 2015 11:01 pm

Re: [HOWTO] SpiderOak Online Backup

Post by friday » Sun Jan 21, 2018 3:14 am

Yeah, me too. It seems like a simple problem with paths not being set or being incorrect, but I verified it really don't think I screwed anything up. The spideroak error messages are less than helpful. Without getting more specific errors like "libsomething.so.0: No such file or directory" it's hopeless. After a qnap firmware upgrade that didn't recognize my old raid I had to set this up again, and between all the possible things that could have been changed (Entware-ng, Python version, Qnap firmware, possible issues with paths and spideroak changes) I wasn't able to figure it out.

One possibility would be to use Container Station with a Docker image for this, but I haven't got around to it. It's probably a lot easier to set up, but likely slower and less performant.

gverweijen
New here
Posts: 3
Joined: Mon May 25, 2015 4:39 am

Re: [HOWTO] SpiderOak Online Backup

Post by gverweijen » Sun Jan 21, 2018 3:38 am

Indeed.. also spideroak release an update to v7.0 a few weeks ago.
Have you considered asking spideroak for some help?
Would you update us here if you find a solution?

friday
New here
Posts: 9
Joined: Wed Sep 16, 2015 11:01 pm

Re: [HOWTO] SpiderOak Online Backup

Post by friday » Sun Jan 21, 2018 4:05 am

Indeed.. also spideroak release an update to v7.0 a few weeks ago.

I tried using several older versions of Spideroak, and this was in November. So I don't think the issue is with any recent Spideroak version.

Have you considered asking spideroak for some help?

No, asking for help goes against my beliefs ;)
Also really doubt cloud storage providers with unlimited plans such as Spideroak want to support NAS. It drives their costs up.

Would you update us here if you find a solution?

Yes, but don't count on it. I'm very busy and haven't touched this since November. It's a shame. I was going to simplify the guide. Spideroak has a slackware release now, so installation is much easier (no entware, ar and temporary paths needed).

For future reference if it helps someone (this will ignore the shellscript which you can copy manually from one of guides and change the path to whereever you want $INSTALLATION_PATH to be):

Code: Select all

cd $INSTALLATION_PATH; wget -qO- "http://spideroak.com/getbuild?platform=slackware&arch=x64" | tar xz opt/SpiderOakONE/ --strip-components=1

pharahiti
New here
Posts: 2
Joined: Thu Sep 20, 2012 11:51 pm

Re: [HOWTO] SpiderOak Online Backup

Post by pharahiti » Sun Mar 25, 2018 2:54 am

I experienced the 'no such file' issue also, and managed to fix it by downloading the latest version of SpiderOakOne from this URL instead:

https://spideroak.com/opendownload/

I downloaded it to a linux PC, extracted the .deb file using dpkg -x, and then tarred and transferred the resulting files to my NAS. I then more or less followed the instructions posted by others to edit the launch script and add the lines to create the symlink and corrected the library and binary paths for my installation.

Hope this helps others..

friday
New here
Posts: 9
Joined: Wed Sep 16, 2015 11:01 pm

Re: [HOWTO] SpiderOak Online Backup

Post by friday » Sun Mar 25, 2018 9:52 pm

I don't think either of those are related to the problem. The different links just redirects to the same file, and ar should work perfectly to extract the deb (even if not, I also tried the slackware release). Getting the paths wrong certainly should lead to an error such as "No such file", but since /opt was mounted at /opt for me there wasn't anything to change.

It's good to know it's working for someone though.

gverweijen
New here
Posts: 3
Joined: Mon May 25, 2015 4:39 am

Re: [HOWTO] SpiderOak Online Backup

Post by gverweijen » Thu Aug 16, 2018 1:07 am

Hi Guys,

Any news on this issue?
Today I retried all the steps using latest release SpiderOakONE 7.2.1
I still get stuck on the same exact error.

also tried renaming the /opt/bin/spideroak to /opt/bin/SpiderOakONE
also tried putting it in /usr/bin/SpiderOakONE

Still the same error.

Post Reply

Return to “Backup & Restore”