[HOWTO] Daily incremental backup with rsnapshot

Discussion on remote replication.
Post Reply
jimguo
Getting the hang of things
Posts: 54
Joined: Sun Jul 26, 2009 12:52 am

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by jimguo » Wed Nov 18, 2009 1:09 am

you're absolute right! Found there's a space in Backup ******.
Changed by TAB & the testing backup is working. thanks a lot!
Still have two questions:
1. how to run rsnapshot manually? I mean by command. as I want to test it intead of waiting for hourly.
2. how to not shown hourly.0 hourly.1 ...... in the destination?
- it's the best to backup files included only instead of hourly.0/share/HDA_DATA/***/***
thanks again.
TS 239 + TS 119 + 469U

crushdepth
Getting the hang of things
Posts: 76
Joined: Thu Mar 26, 2009 8:56 pm

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by crushdepth » Wed Nov 18, 2009 9:50 am

1. You can run it manually by typing 'rsnapshot hourly' or 'rsnapshot daily' etc.

2. I don't know if you can change that. I think it will insist on putting the whole file path in.

jimguo
Getting the hang of things
Posts: 54
Joined: Sun Jul 26, 2009 12:52 am

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by jimguo » Wed Nov 18, 2009 4:14 pm

1. Thanks! it works
2. I'm testing. Seems you're right.
BTW, if I setup backup every 1 hour, will there're 24 dirs: hourly.0, hourly.1, hourly.2,..... hourly.23? My concern is that if I don't have any amendment in the source, will these 24 folders have the same contents? - I tested a few, it looks so.
- or in hourly.1, hour.2... they're hard links as you informed?
TS 239 + TS 119 + 469U

crushdepth
Getting the hang of things
Posts: 76
Joined: Thu Mar 26, 2009 8:56 pm

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by crushdepth » Wed Nov 18, 2009 4:35 pm

Yes you will get 24 'hourly' folders. However only the files that change are duplicated, those that don't change are just hard links, so they don't take up any space. If you have a lot of data and it changes often then you might have problems fitting so many copies on your NAS. But if your contents doesn't change much then it won't matter.

But either way maybe you can reduce the backup frequency and let your drives spin down to save power. I do my work file server once every 4 hours / once a day for a week / once a week for 4 weeks / once a month for 3 months. We are fairly 'normal use' and I find the backup takes up about 30% more file space than the source files, but we have 20 restore points to choose from.

jimguo
Getting the hang of things
Posts: 54
Joined: Sun Jul 26, 2009 12:52 am

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by jimguo » Wed Nov 18, 2009 4:56 pm

Well, then how to restore them if emergency happens?
Just copy 'hourly.0' files to the destination as all hard linked files in other folders will be copied automatically?
TS 239 + TS 119 + 469U

crushdepth
Getting the hang of things
Posts: 76
Joined: Thu Mar 26, 2009 8:56 pm

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by crushdepth » Wed Nov 18, 2009 6:50 pm

Yes that's right. It's exactly like working with a normal directory. If you need to find an old version of a file, you can also look in older snapshots to find the document in an earlier state.

jimguo
Getting the hang of things
Posts: 54
Joined: Sun Jul 26, 2009 12:52 am

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by jimguo » Wed Nov 18, 2009 7:47 pm

that's great. thanks for all your help!
TS 239 + TS 119 + 469U

gregoryx
Starting out
Posts: 32
Joined: Sun Nov 01, 2009 4:39 am

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by gregoryx » Sun Nov 22, 2009 1:23 pm

Thanks for this. It looks like exactly what I'd like to do (and have been doing previously with my ReadyNAS units).

Do you know if the rsnapshot needs to be loaded on both systems? I'm now setting up the Qnap as primary with one of the old ReadyNAS as secondary and have rsync working between them. The issue (of course) is that Qnap's config, logging, and notifications seem pretty sorely lacking; hence the interest in this.

crushdepth
Getting the hang of things
Posts: 76
Joined: Thu Mar 26, 2009 8:56 pm

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by crushdepth » Mon Nov 23, 2009 10:28 am

You only need to install it on the machine that will be storing the backups.

jimguo
Getting the hang of things
Posts: 54
Joined: Sun Jul 26, 2009 12:52 am

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by jimguo » Wed Nov 25, 2009 12:39 am

Hi crushdepth,
My hourly backup works perfect. however, the daily backup doesn't work :?
anything wrong below?

Code: Select all

0 9-19/2 * * * /opt/bin/rsnapshot hourly
0 2 * * 1-6/1 /opt/bin/rsnapshot daily

- when I input command 'rsnapshot daily', the new 'daily.*' comes however there's no true backup run...
Jim
TS 239 + TS 119 + 469U

linuxnutzer
Starting out
Posts: 12
Joined: Wed Jun 24, 2009 5:23 pm

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by linuxnutzer » Wed Dec 02, 2009 5:49 am

crikeyjp wrote:Hi,

I've been running rsnapshot on a 219P for a while now and had automated brilliantly. Thanks for the guide and help!!

However, I'm having problems running rsnapshot-diff. It seems to fall over as soon as it hits a large file (greater than 2GB I think). Is anyone else seeing this problem???

Output looks like this:

Code: Select all

snapshot-diff daily.1/localhost/share/dir2 daily.0/localhost/share/dir2
Comparing daily.1/localhost/share/dir2 to daily.0/localhost/share/dir2
Use of uninitialized value in numeric eq (==) at /opt/bin/rsnapshot-diff line 220.
Can't compare across devices.
(looking at daily.1/localhost/share/dir2/file1)


BTW, dir2 contains hundreds of files but file1 is 3114364980 bytes big.

Cheers,
JP

It is a problem of the perl package. There was a missing compiler option to enable large file support. I contacted the maintainer of the perl ipkg and he fixed this issue today. You should upgrade the perl package.

Linuxnutzer

GeoffC
Starting out
Posts: 16
Joined: Wed Aug 26, 2009 7:51 pm

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by GeoffC » Thu Dec 03, 2009 5:01 am

jimguo wrote:Hi crushdepth,
My hourly backup works perfect. however, the daily backup doesn't work :?
anything wrong below?

Code: Select all

0 9-19/2 * * * /opt/bin/rsnapshot hourly
0 2 * * 1-6/1 /opt/bin/rsnapshot daily

- when I input command 'rsnapshot daily', the new 'daily.*' comes however there's no true backup run...
Jim


The fact that the daily directory is being created, probably means that it is working.
You are doing hourly backups at 09:00, 11:00, 13:00, 15:00, 17:00 & 19:00, so I would guess you have "retain hourly 6" or "interval hourly 6" in your rsnaphot.conf file.
When you perform the daily snapshot, hourly.5 will be moved to daily.0 (after shuffling all the other daily.* files along).

So in effect, there is no true backup run, it just moves the oldest hourly backup.

sk8ter
First post
Posts: 1
Joined: Sat Dec 05, 2009 11:30 am

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by sk8ter » Sat Dec 12, 2009 4:24 am

Hi,

I just began using rsnapshot to backup my data. The problem I'm faced with, is that rsnapshot always fails with the following message.

Code: Select all

rsync: connection unexpectedly closed (90 bytes received so far) [receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(635) [receiver=3.0.3]
rsync: connection unexpectedly closed (59 bytes received so far) [generator]
rsync error: unexplained error (code 255) at io.c(635) [generator=3.0.3]
----------------------------------------------------------------------------
rsnapshot encountered an error! The program was invoked with these options:
/usr/bin/rsnapshot hourly
----------------------------------------------------------------------------
ERROR: /usr/bin/rsync returned 255 while processing admin@mouse.pyramid.lan:/share/MD0_DATA/upload/
/usr/bin/logger -i -p user.err -t rsnapshot /usr/bin/rsnapshot hourly: \
    ERROR: /usr/bin/rsync returned 255 while processing \
    admin@mouse.pyramid.lan:/share/MD0_DATA/upload/


After a while I noticed that this problem occurs when rsnapshot tries to back up an empty directory.

Is there an option in rsync that allows to backup empty dirs?

Grumpy Bob
New here
Posts: 3
Joined: Tue Dec 22, 2009 11:27 am

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by Grumpy Bob » Wed Dec 30, 2009 3:10 am

I've used this howto to get a backup system running on my new TS-239 . Very easy, and clearly written. Many thanks, crushdepth.

Robert

efaden
Getting the hang of things
Posts: 65
Joined: Sun Nov 22, 2009 6:42 am

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by efaden » Sat Jan 02, 2010 10:43 am

Is there any way to send SMS/Email Notifications on a failure?

Post Reply

Return to “Remote Replication/ Disaster Recovery”