[HOWTO] Daily incremental backup with rsnapshot

Discussion on remote replication.
Post Reply
jimguo
Getting the hang of things
Posts: 54
Joined: Sun Jul 26, 2009 12:52 am

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by jimguo »

you're absolute right! Found there's a space in Backup ******.
Changed by TAB & the testing backup is working. thanks a lot!
Still have two questions:
1. how to run rsnapshot manually? I mean by command. as I want to test it intead of waiting for hourly.
2. how to not shown hourly.0 hourly.1 ...... in the destination?
- it's the best to backup files included only instead of hourly.0/share/HDA_DATA/***/***
thanks again.
TS 239 + TS 119 + 469U
crushdepth
Getting the hang of things
Posts: 76
Joined: Thu Mar 26, 2009 8:56 pm

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by crushdepth »

1. You can run it manually by typing 'rsnapshot hourly' or 'rsnapshot daily' etc.

2. I don't know if you can change that. I think it will insist on putting the whole file path in.
jimguo
Getting the hang of things
Posts: 54
Joined: Sun Jul 26, 2009 12:52 am

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by jimguo »

1. Thanks! it works
2. I'm testing. Seems you're right.
BTW, if I setup backup every 1 hour, will there're 24 dirs: hourly.0, hourly.1, hourly.2,..... hourly.23? My concern is that if I don't have any amendment in the source, will these 24 folders have the same contents? - I tested a few, it looks so.
- or in hourly.1, hour.2... they're hard links as you informed?
TS 239 + TS 119 + 469U
crushdepth
Getting the hang of things
Posts: 76
Joined: Thu Mar 26, 2009 8:56 pm

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by crushdepth »

Yes you will get 24 'hourly' folders. However only the files that change are duplicated, those that don't change are just hard links, so they don't take up any space. If you have a lot of data and it changes often then you might have problems fitting so many copies on your NAS. But if your contents doesn't change much then it won't matter.

But either way maybe you can reduce the backup frequency and let your drives spin down to save power. I do my work file server once every 4 hours / once a day for a week / once a week for 4 weeks / once a month for 3 months. We are fairly 'normal use' and I find the backup takes up about 30% more file space than the source files, but we have 20 restore points to choose from.
jimguo
Getting the hang of things
Posts: 54
Joined: Sun Jul 26, 2009 12:52 am

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by jimguo »

Well, then how to restore them if emergency happens?
Just copy 'hourly.0' files to the destination as all hard linked files in other folders will be copied automatically?
TS 239 + TS 119 + 469U
crushdepth
Getting the hang of things
Posts: 76
Joined: Thu Mar 26, 2009 8:56 pm

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by crushdepth »

Yes that's right. It's exactly like working with a normal directory. If you need to find an old version of a file, you can also look in older snapshots to find the document in an earlier state.
jimguo
Getting the hang of things
Posts: 54
Joined: Sun Jul 26, 2009 12:52 am

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by jimguo »

that's great. thanks for all your help!
TS 239 + TS 119 + 469U
gregoryx
Starting out
Posts: 32
Joined: Sun Nov 01, 2009 4:39 am

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by gregoryx »

Thanks for this. It looks like exactly what I'd like to do (and have been doing previously with my ReadyNAS units).

Do you know if the rsnapshot needs to be loaded on both systems? I'm now setting up the Qnap as primary with one of the old ReadyNAS as secondary and have rsync working between them. The issue (of course) is that Qnap's config, logging, and notifications seem pretty sorely lacking; hence the interest in this.
crushdepth
Getting the hang of things
Posts: 76
Joined: Thu Mar 26, 2009 8:56 pm

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by crushdepth »

You only need to install it on the machine that will be storing the backups.
jimguo
Getting the hang of things
Posts: 54
Joined: Sun Jul 26, 2009 12:52 am

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by jimguo »

Hi crushdepth,
My hourly backup works perfect. however, the daily backup doesn't work :?
anything wrong below?

Code: Select all

0 9-19/2 * * * /opt/bin/rsnapshot hourly
0 2 * * 1-6/1 /opt/bin/rsnapshot daily
- when I input command 'rsnapshot daily', the new 'daily.*' comes however there's no true backup run...
Jim
TS 239 + TS 119 + 469U
linuxnutzer
Starting out
Posts: 17
Joined: Wed Jun 24, 2009 5:23 pm

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by linuxnutzer »

crikeyjp wrote:Hi,

I've been running rsnapshot on a 219P for a while now and had automated brilliantly. Thanks for the guide and help!!

However, I'm having problems running rsnapshot-diff. It seems to fall over as soon as it hits a large file (greater than 2GB I think). Is anyone else seeing this problem???

Output looks like this:

Code: Select all

snapshot-diff daily.1/localhost/share/dir2 daily.0/localhost/share/dir2
Comparing daily.1/localhost/share/dir2 to daily.0/localhost/share/dir2
Use of uninitialized value in numeric eq (==) at /opt/bin/rsnapshot-diff line 220.
Can't compare across devices.
(looking at daily.1/localhost/share/dir2/file1)
BTW, dir2 contains hundreds of files but file1 is 3114364980 bytes big.

Cheers,
JP
It is a problem of the perl package. There was a missing compiler option to enable large file support. I contacted the maintainer of the perl ipkg and he fixed this issue today. You should upgrade the perl package.

Linuxnutzer
GeoffC
Starting out
Posts: 16
Joined: Wed Aug 26, 2009 7:51 pm

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by GeoffC »

jimguo wrote:Hi crushdepth,
My hourly backup works perfect. however, the daily backup doesn't work :?
anything wrong below?

Code: Select all

0 9-19/2 * * * /opt/bin/rsnapshot hourly
0 2 * * 1-6/1 /opt/bin/rsnapshot daily
- when I input command 'rsnapshot daily', the new 'daily.*' comes however there's no true backup run...
Jim
The fact that the daily directory is being created, probably means that it is working.
You are doing hourly backups at 09:00, 11:00, 13:00, 15:00, 17:00 & 19:00, so I would guess you have "retain hourly 6" or "interval hourly 6" in your rsnaphot.conf file.
When you perform the daily snapshot, hourly.5 will be moved to daily.0 (after shuffling all the other daily.* files along).

So in effect, there is no true backup run, it just moves the oldest hourly backup.
sk8ter
First post
Posts: 1
Joined: Sat Dec 05, 2009 11:30 am

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by sk8ter »

Hi,

I just began using rsnapshot to backup my data. The problem I'm faced with, is that rsnapshot always fails with the following message.

Code: Select all

rsync: connection unexpectedly closed (90 bytes received so far) [receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(635) [receiver=3.0.3]
rsync: connection unexpectedly closed (59 bytes received so far) [generator]
rsync error: unexplained error (code 255) at io.c(635) [generator=3.0.3]
----------------------------------------------------------------------------
rsnapshot encountered an error! The program was invoked with these options:
/usr/bin/rsnapshot hourly 
----------------------------------------------------------------------------
ERROR: /usr/bin/rsync returned 255 while processing admin@mouse.pyramid.lan:/share/MD0_DATA/upload/
/usr/bin/logger -i -p user.err -t rsnapshot /usr/bin/rsnapshot hourly: \
    ERROR: /usr/bin/rsync returned 255 while processing \
    admin@mouse.pyramid.lan:/share/MD0_DATA/upload/ 
After a while I noticed that this problem occurs when rsnapshot tries to back up an empty directory.

Is there an option in rsync that allows to backup empty dirs?
Grumpy Bob
New here
Posts: 3
Joined: Tue Dec 22, 2009 11:27 am

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by Grumpy Bob »

I've used this howto to get a backup system running on my new TS-239 . Very easy, and clearly written. Many thanks, crushdepth.

Robert
efaden
Getting the hang of things
Posts: 65
Joined: Sun Nov 22, 2009 6:42 am

Re: [HOWTO] Daily incremental backup with rsnapshot

Post by efaden »

Is there any way to send SMS/Email Notifications on a failure?
Post Reply

Return to “Remote Replication/ Disaster Recovery”