[HOWTO] Daily incremental backup with rsnapshot
- AndyChuo
- Experience counts
- Posts: 2388
- Joined: Thu Sep 13, 2007 11:56 am
- Location: Taipei, Taiwan
Re: Daily incremental backup: rsync + snapshots via shell script
Stickied. Thanks for sharing and we'd like to see how it goes after running for a certain period of time and will consider adding to the current firmware.
Andy
Andy
=============================================================>>>
TS-659-Pro [RAID6] rtorrent+SABnzbdplus+SickBeard+Couchpotato [Best PVR] Plex+PMS [Ultimate Streamer]
Apple iPad [Best Tablet] HTC One M8 [Mobile Phone] Samsung UA46ES6100 [My Screen] KRK Rokit 6 [Audio Speakers]
Chrome Cast [Screen Casting] Philips Hue [Personal Lighing]
Buffalo WZR-1750DHP [My Wifi Hub] D-Link DGS-1005D [Gbit Network]
=============================================================>>>
TS-659-Pro [RAID6] rtorrent+SABnzbdplus+SickBeard+Couchpotato [Best PVR] Plex+PMS [Ultimate Streamer]
Apple iPad [Best Tablet] HTC One M8 [Mobile Phone] Samsung UA46ES6100 [My Screen] KRK Rokit 6 [Audio Speakers]
Chrome Cast [Screen Casting] Philips Hue [Personal Lighing]
Buffalo WZR-1750DHP [My Wifi Hub] D-Link DGS-1005D [Gbit Network]
=============================================================>>>
-
- Starting out
- Posts: 40
- Joined: Fri May 08, 2009 5:27 am
Re: Daily incremental backup: rsync + snapshots via shell script
this looks like a great solution for my Small Office (4 users on a server) however it is missing 2 things to make this an ideal solution for me.
1) a web GUI
2) a daily backup log of changed files (viewable in HTML)
it would be great if the Qnap team could package this for easy install.
thanks,
Adrian
1) a web GUI
2) a daily backup log of changed files (viewable in HTML)
it would be great if the Qnap team could package this for easy install.
thanks,
Adrian
-
- Getting the hang of things
- Posts: 76
- Joined: Thu Mar 26, 2009 8:56 pm
Re: [HOWTO] Daily incremental backup with rsnapshot
Ok I have this in production at work now using two TS-639s, one of which is being used as a file server, and the other as a backup server running rsnapshot over SSH with key-based authentication. So far it is working well, but I'll post back after our users have pounded on the system for a month or so.
Also attached: Slightly revised and corrected rsnapshot HOWTO guide. Please use this version as the last one had an error in the command to relaunch the crontab (required to make changes persistent). It should have read:
EDIT: Please see post #22 below for the final version of the installation guide, I have deleted the old version.
Also attached: Slightly revised and corrected rsnapshot HOWTO guide. Please use this version as the last one had an error in the command to relaunch the crontab (required to make changes persistent). It should have read:
Code: Select all
crontab /etc/config/crontab
Last edited by crushdepth on Sun Jul 19, 2009 1:49 pm, edited 1 time in total.
-
- Starting out
- Posts: 18
- Joined: Tue Nov 25, 2008 3:39 pm
Re: [HOWTO] Daily incremental backup with rsnapshot
Hi Crushdepth,
excellent paper - I just implemented this on a TS-509 locally and it worked great. Next step will be to incorporate some remote PCs with key-based authentication.
Again, many thanks for this great contribution to this wonderful forum.
excellent paper - I just implemented this on a TS-509 locally and it worked great. Next step will be to incorporate some remote PCs with key-based authentication.
Again, many thanks for this great contribution to this wonderful forum.
-
- Starting out
- Posts: 18
- Joined: Wed Jan 28, 2009 11:40 pm
Re: [HOWTO] Daily incremental backup with rsnapshot
Great! Just what I was looking for.
Of course, with a GUI it would be even better.
/Kent
Of course, with a GUI it would be even better.
/Kent
------------------------------------------------------------------
QNAP TS-509Pro, Firmware version: 4.1.0
5 x Samsung Spinpoint F1 1TB
QNAP TS-509Pro, Firmware version: 4.1.0
5 x Samsung Spinpoint F1 1TB
- Spag
- Know my way around
- Posts: 227
- Joined: Tue Nov 18, 2008 7:42 pm
- Location: Denmark
Re: [HOWTO] Daily incremental backup with rsnapshot
Can anyone tell me how the folder structure or backup image looks like on the backup server?
TS-569 Pro, Firmware 4.1.0 Build 1101 (Beta)
TS-469 Pro, Firmware 4.1.0 Build 1101 (Beta)
TS-239 Pro, Firmware 4.0.4 Build 20131001
TS-209 Pro II, Firmware 3.3.2 Build 0918
TS-469 Pro, Firmware 4.1.0 Build 1101 (Beta)
TS-239 Pro, Firmware 4.0.4 Build 20131001
TS-209 Pro II, Firmware 3.3.2 Build 0918
-
- Getting the hang of things
- Posts: 76
- Joined: Thu Mar 26, 2009 8:56 pm
Re: [HOWTO] Daily incremental backup with rsnapshot
You designate a backup directory on the backup server. Inside this, you will get a collection of snapshots building up over time, eg:
hourly.0
hourly.1
hourly.2
hourly.3
daily.0
daily.1
daily.2
daily.3
weekly.0....etc
monthly.0 ...etc
Each one of these will contain a file structure identical to the target file system at the time the snapshot was taken.
hourly.0
hourly.1
hourly.2
hourly.3
daily.0
daily.1
daily.2
daily.3
weekly.0....etc
monthly.0 ...etc
Each one of these will contain a file structure identical to the target file system at the time the snapshot was taken.
-
- Getting the hang of things
- Posts: 76
- Joined: Thu Mar 26, 2009 8:56 pm
Re: [HOWTO] Daily incremental backup with rsnapshot
Ok I've been running rsnapshot in our office for a month now between our two TS-639s. We use one as a file server and the other as a backup device. I've also set it to backup our remote webserver/website as well. I'm happy to say it is working very well and no problems at all. I have added a small section to the install guide on monitoring the rsnapshot log, this will be the final version.
By the way, if anyone has any feedback on how well (or not) this is working for them, I would appreciate it.
By the way, if anyone has any feedback on how well (or not) this is working for them, I would appreciate it.
You do not have the required permissions to view the files attached to this post.
-
- New here
- Posts: 4
- Joined: Wed Jan 28, 2009 3:21 pm
Re: [HOWTO] Daily incremental backup with rsnapshot
Thanks for the how to was excellent!
-
- Starting out
- Posts: 16
- Joined: Wed Aug 26, 2009 7:51 pm
Re: [HOWTO] Daily incremental backup with rsnapshot
I have been meaning to add some comments here for a while.
Firstly - Thanks Crushdepth - a great job with the "How To". For what this can do, you made it all quite straight forward.
A couple of things I found;
When you download the Optware-IPKG file, you need to unzip it before you install it - I know it's obvious, and it was to me when it didn't work the first time. It may even be IE changing the extension, as I have seen it do funning things like that before.
I've installed this on two TS-239 units - on one (was running 2.1.7 firmware if that matters, while the other was 3.1.1), I had to reboot the NAS before I could run "ipkg update". Maybe it was just a one off or something else I did.
As for editors, I like to use WinSCP, as it helps me locate the files - seasoned command line users please yourself.
When configuring rsnapshot.conf, I found "interval" very confusing, as it may or may not have anything to do with how often you take snapshots.
Reading the man page at rsnapshot.org, "interval" is now a depreciated alias for the keyword "retain" which makes much more sense with its use.
Firstly - Thanks Crushdepth - a great job with the "How To". For what this can do, you made it all quite straight forward.
A couple of things I found;
When you download the Optware-IPKG file, you need to unzip it before you install it - I know it's obvious, and it was to me when it didn't work the first time. It may even be IE changing the extension, as I have seen it do funning things like that before.
I've installed this on two TS-239 units - on one (was running 2.1.7 firmware if that matters, while the other was 3.1.1), I had to reboot the NAS before I could run "ipkg update". Maybe it was just a one off or something else I did.
As for editors, I like to use WinSCP, as it helps me locate the files - seasoned command line users please yourself.
When configuring rsnapshot.conf, I found "interval" very confusing, as it may or may not have anything to do with how often you take snapshots.
Reading the man page at rsnapshot.org, "interval" is now a depreciated alias for the keyword "retain" which makes much more sense with its use.
-
- New here
- Posts: 2
- Joined: Thu Sep 17, 2009 8:41 pm
Re: [HOWTO] Daily incremental backup with rsnapshot
Hi,
I've been running rsnapshot on a 219P for a while now and had automated brilliantly. Thanks for the guide and help!!
However, I'm having problems running rsnapshot-diff. It seems to fall over as soon as it hits a large file (greater than 2GB I think). Is anyone else seeing this problem???
Output looks like this:
BTW, dir2 contains hundreds of files but file1 is 3114364980 bytes big.
Cheers,
JP
I've been running rsnapshot on a 219P for a while now and had automated brilliantly. Thanks for the guide and help!!
However, I'm having problems running rsnapshot-diff. It seems to fall over as soon as it hits a large file (greater than 2GB I think). Is anyone else seeing this problem???
Output looks like this:
Code: Select all
snapshot-diff daily.1/localhost/share/dir2 daily.0/localhost/share/dir2
Comparing daily.1/localhost/share/dir2 to daily.0/localhost/share/dir2
Use of uninitialized value in numeric eq (==) at /opt/bin/rsnapshot-diff line 220.
Can't compare across devices.
(looking at daily.1/localhost/share/dir2/file1)
Cheers,
JP
-
- Getting the hang of things
- Posts: 54
- Joined: Sun Jul 26, 2009 12:52 am
Re: [HOWTO] Daily incremental backup with rsnapshot
Hi crushdepth,
Re: rsnapshot.conf
I can only find only 2 places needs to be amended?
1. #snapshot_root /opt/var/rsnapshot/
- should I change the /opt/var/rsnapshot/ to /share/myownfolder ?
2. if I want to backup /Qusb on the source, however I got errors with the below:
backup admin@myowndomain:/share/Qusb / localhost/
don't know where the problem is.
appreciate your help.
regards
Jim
Re: rsnapshot.conf
I can only find only 2 places needs to be amended?
1. #snapshot_root /opt/var/rsnapshot/
- should I change the /opt/var/rsnapshot/ to /share/myownfolder ?
2. if I want to backup /Qusb on the source, however I got errors with the below:
backup admin@myowndomain:/share/Qusb / localhost/
don't know where the problem is.
appreciate your help.
regards
Jim
TS 239 + TS 119 + 469U
-
- Getting the hang of things
- Posts: 76
- Joined: Thu Mar 26, 2009 8:56 pm
Re: [HOWTO] Daily incremental backup with rsnapshot
I'm not sure if it would matter, but it is probably a good idea to set up a different folder as your snapshot root as you say. My root path looks like this (RAID 6, yours might look slightly different if you are using a different kind of RAID):
Your backup point looks ok, but try using the IP address of the source machine rather than a domain name. eg:
backup admin@192.168.10.42:/share/MD0_DATA/ orca/ [this is another NAS on my local network, I backup the whole share folder]
backup root@89.24.126.98:/home/penguins/public_html/ server.penguins.com/ [this is a remote webserver]
Have you got any other backup points working? What does the error message say?
Code: Select all
snapshot_root /share/MD0_DATA/snapshots/
backup admin@192.168.10.42:/share/MD0_DATA/ orca/ [this is another NAS on my local network, I backup the whole share folder]
backup root@89.24.126.98:/home/penguins/public_html/ server.penguins.com/ [this is a remote webserver]
Have you got any other backup points working? What does the error message say?
-
- Getting the hang of things
- Posts: 54
- Joined: Sun Jul 26, 2009 12:52 am
Re: [HOWTO] Daily incremental backup with rsnapshot
thanks!
it's still not working. i got below when input 'rsnapshot configtest'
don't know where the problem is?
haven't tested others yet.
it's still not working. i got below when input 'rsnapshot configtest'
Code: Select all
rsnapshot encountered an error! The program was invoked with these options:
/opt/bin/rsnapshot configtest
----------------------------------------------------------------------------
ERROR: /opt/etc/rsnapshot.conf on line 162:
ERROR: backup admin@125.107.82.10:/share/HDA_DATA/Qusb/ localhost/
ERROR: ---------------------------------------------------------------------
ERROR: Errors were found in /opt/etc/rsnapshot.conf,
ERROR: rsnapshot can not continue. If you think an entry looks right, make
ERROR: sure you don't have spaces where only tabs should be.
haven't tested others yet.
TS 239 + TS 119 + 469U
-
- Getting the hang of things
- Posts: 76
- Joined: Thu Mar 26, 2009 8:56 pm
Re: [HOWTO] Daily incremental backup with rsnapshot
Ok if there's an error in the config file it usually means you have a space character in there somewhere. Everything has to be separated with tab characters, spaces are illegal. Is there any spaces in that line?