[HOWTO] Daily incremental backup with rsnapshot
-
- Getting the hang of things
- Posts: 76
- Joined: Thu Mar 26, 2009 8:56 pm
[HOWTO] Daily incremental backup with rsnapshot
I'm looking at buying a TS-439 as a file server for work, and a TS-639 as a backup server (for the 439). Our office needs a decent level of backup protection, so I would like to retain:
1. Daily backups, for the last seven days.
2. Weekly backups, for the last 4 weeks.
3. Monthly backups, for the last 3 months.
I was thinking of implementing this using rsync and snapshots. Since the QNAP admin interface doesn't support snapshots, could I run the job manually via a shell script?
Thanks.
1. Daily backups, for the last seven days.
2. Weekly backups, for the last 4 weeks.
3. Monthly backups, for the last 3 months.
I was thinking of implementing this using rsync and snapshots. Since the QNAP admin interface doesn't support snapshots, could I run the job manually via a shell script?
Thanks.
Last edited by crushdepth on Tue Jun 09, 2009 4:28 pm, edited 1 time in total.
-
- Starting out
- Posts: 15
- Joined: Mon Mar 30, 2009 4:00 am
Re: Daily incremental backup: rsync + snapshots via shell script
There is an IPKG-Package called rSnapshot http://www.rsnapshot.org. You can run it as cronjob periodically. It automatically removes/merges superfluous backups when backup frequency changes.
On the rsnapshot homepage they say
I personally have used it some years ago and it worked very well.
On the rsnapshot homepage they say
so I would say it is more advanced than your initial proposal.rsnapshot was originally based on an article called Easy Automated Snapshot-Style Backups with Linux and Rsync, by Mike Rubel.
I personally have used it some years ago and it worked very well.
-
- Getting the hang of things
- Posts: 76
- Joined: Thu Mar 26, 2009 8:56 pm
Re: Daily incremental backup: rsync + snapshots via shell script
Thanks, rSnapshot would be perfect, if its possible. I have the Optware IPKG installed now, looking at the list of feeds, it looks they are model (or processor) specific.
Anyone know if there are TS-439 specific feeds available?
Anyone know if there are TS-439 specific feeds available?
-
- Getting the hang of things
- Posts: 76
- Joined: Thu Mar 26, 2009 8:56 pm
Re: Daily incremental backup: rsync + snapshots via shell script
Ok I've just about got this working now - having a few problems getting my cron jobs to persist after reboot. rsnapshot is *excellent*. Will write up a detailed guide on how to implement this shortly.
Edit: It works!!! The first run of rsnapshot (rsync) is horribly slow and CPU intensive - took over an hour. But the subsequent runs only take a few seconds. Cool!!!!
Edit: It works!!! The first run of rsnapshot (rsync) is horribly slow and CPU intensive - took over an hour. But the subsequent runs only take a few seconds. Cool!!!!
-
- Starting out
- Posts: 44
- Joined: Tue Mar 31, 2009 8:26 am
Re: Daily incremental backup: rsync + snapshots via shell script
When is the detailed write up due?
Cheers,
Adrian
Cheers,
Adrian
-
- Getting the hang of things
- Posts: 76
- Joined: Thu Mar 26, 2009 8:56 pm
Re: Daily incremental backup: rsync + snapshots via shell script
Ok the tutorial on installation of rsnapshot on the TS-439 is available for download. You will also need to download the official rsnapshot HOWTO which provides a tutorial on configuration. rsnapshot really is great if you want to create an automatically rotating time-series of backups, or if you want an easy way to automate rsync.
Summary
rsnapshot is a file system backup utility based on rsync. Using rsnapshot is possible to take snapshots of your filesystems at different points in time. Using hard links, rsnapshot creates the illusion of multiple full backups, while only taking up the space of one full backup plus changes. When coupled with ssh, it is possible to take snapshots of remote file systems as well. This document describes how to set up rsnapshot on the QNAP TS-439 (but I imagine it would work on all TS-x39 series) of NAS boxes with a view to schedule a nested and rotating series of incremental backups, eg:
• Hourly incremental backups, for the past 24 hours.
• Daily incremental backups, for the past 7 days.
• Weekly incremental backups, for the past 4 weeks.
• Monthly incremental backups, for the past 3 months.
Summary
rsnapshot is a file system backup utility based on rsync. Using rsnapshot is possible to take snapshots of your filesystems at different points in time. Using hard links, rsnapshot creates the illusion of multiple full backups, while only taking up the space of one full backup plus changes. When coupled with ssh, it is possible to take snapshots of remote file systems as well. This document describes how to set up rsnapshot on the QNAP TS-439 (but I imagine it would work on all TS-x39 series) of NAS boxes with a view to schedule a nested and rotating series of incremental backups, eg:
• Hourly incremental backups, for the past 24 hours.
• Daily incremental backups, for the past 7 days.
• Weekly incremental backups, for the past 4 weeks.
• Monthly incremental backups, for the past 3 months.
-
- Starting out
- Posts: 12
- Joined: Thu Apr 16, 2009 8:04 pm
Re: Daily incremental backup: rsync + snapshots via shell script
..the domain seems to be down... cannot download it =(
// Filiph
Ts-119
Ts-109 II
Ts-119
Ts-109 II
- Q
- Experience counts
- Posts: 1436
- Joined: Wed Sep 05, 2007 4:18 pm
- Location: Switzerland
- Contact:
Re: Daily incremental backup: rsync + snapshots via shell script
that's really great, thanks for the tutorial! would love to see such functionality integrated by default with a nice gui because versioning and snapshots is highly demanded by enterprise customers as well as home users these days... HINT @QNAP
@filiphw: download worked for me, maybe just try again
@filiphw: download worked for me, maybe just try again
-
- Starting out
- Posts: 12
- Joined: Thu Apr 16, 2009 8:04 pm
Re: Daily incremental backup: rsync + snapshots via shell script
Tried it again without success... can anyone attach the pdf to this forum thread?
// Filiph
Ts-119
Ts-109 II
Ts-119
Ts-109 II
- Q
- Experience counts
- Posts: 1436
- Joined: Wed Sep 05, 2007 4:18 pm
- Location: Switzerland
- Contact:
Re: Daily incremental backup: rsync + snapshots via shell script
sure
You do not have the required permissions to view the files attached to this post.
-
- Starting out
- Posts: 12
- Joined: Thu Apr 16, 2009 8:04 pm
-
- Getting the hang of things
- Posts: 76
- Joined: Thu Mar 26, 2009 8:56 pm
Re: Daily incremental backup: rsync + snapshots via shell script
Yeah sorry it looks like my website is having some DNS problems. Thanks for posting the zip file!
I agree. Simple replication just isn't enough for business users. Having some versioning / history in your backup is essential. rsnapshot is a mature and well-known tool which is easy to use. A killer feature that would differentiate QNAP from other NAS! Please integrate it into the web interface!would love to see such functionality integrated by default with a nice gui because versioning and snapshots is highly demanded by enterprise customers as well as home users these days... HINT @QNAP
-
- New here
- Posts: 4
- Joined: Fri May 01, 2009 7:16 pm
Re: Daily incremental backup: rsync + snapshots via shell script
this is great thanks , i have just posted to find out if i can replicate multiple boxes to 1 main box, did you have any need for that as a function ?
-
- Getting the hang of things
- Posts: 76
- Joined: Thu Mar 26, 2009 8:56 pm
Re: Daily incremental backup: rsync + snapshots via shell script
I think you could. You can define what directories you want backed up in the rsnapshot config file, and according to the howto that includes remote file systems. So you could probably s_uck* in several NAS to a bigger one (but I haven't tried it).
*Note to mods: s_uck is not a swear word, it does have legitimate uses. Please don't filter it
*Note to mods: s_uck is not a swear word, it does have legitimate uses. Please don't filter it
-
- Getting the hang of things
- Posts: 76
- Joined: Thu Mar 26, 2009 8:56 pm
rsnapshot: howto guide for TS-x39
I've updated the rsnapshot installation howto (attached) with a section on how to set up key-based authentication. This enables rsnapshot jobs to run over SSH without requiring password login to remote servers.
It's now a fully-functioning solution for backup with versioning on your QNAP. Enjoy!
[Edit: Please see the revised howto in post below, I have removed the old version from this post as it contained an error]
It's now a fully-functioning solution for backup with versioning on your QNAP. Enjoy!
[Edit: Please see the revised howto in post below, I have removed the old version from this post as it contained an error]
Last edited by crushdepth on Sun Jun 14, 2009 1:09 pm, edited 2 times in total.