RSYNC not working so far

Backup, Restore, Netbak Replicator, Cloud Storage Services
redghost
Starting out
Posts: 12
Joined: Wed Jun 23, 2010 5:57 am

Re: RSYNC not working so far

Post by redghost »

Dear Everyone

I just noticed this thread about rsync over ssh. I use rsync over ssh to rsync.net. I wanted to do it via the web interface but the web interface doesn't allow it and rsync.net require ssh so I ended up doing it directly by logging in to the NAS via ssh/telnet. It has worked well but just lately I have had some difficulty (see this thread: http://forum.qnap.com/viewtopic.php?f=15&t=70375). The title says it is broken but I have got it working somewhat since then. I am just editing the thread.

To get it to work is really easy. SSH and Rsync are both present on the NAS so I set up a script to run it overnight via cron but can easily log in to my QNAP NAS via ssh or telnet and just issue the rsync command. My command is something like the following.I had to run it manually the first time to save the keys from the other side but that will depend on your set up.

The general command I use is

# /usr/bin/rsync -av --delete --progress -e /usr/bin/ssh /share/bkup/pst_file_to_be_backed_up.zip my_id@host.com:

Edit: To pull I think you just reverse the two locations but I am not a rsync expert so someone else can advise depending on the details of your system.

I hope this helps

Red
Last edited by redghost on Wed Jan 23, 2013 12:55 am, edited 1 time in total.
bsg75
Getting the hang of things
Posts: 84
Joined: Fri Dec 07, 2012 6:55 am

Re: RSYNC not working so far

Post by bsg75 »

@redghost : thanks for this contribution. I will certainly check this out. Are you working in a pure Unix environment or a mixed type one (locally Windows, remote Unix)?
redghost
Starting out
Posts: 12
Joined: Wed Jun 23, 2010 5:57 am

Re: RSYNC not working so far

Post by redghost »

I'll be honest I have no idea whatsoever whehter the remote host is windows or unix. In fact it has never come up from rsync.net so I don't know that it matters. I am not tech savvy (I admit) but I suppose you can just try.

Red
bsg75
Getting the hang of things
Posts: 84
Joined: Fri Dec 07, 2012 6:55 am

Re: RSYNC not working so far

Post by bsg75 »

At your site, do you use Windows PC's?
redghost
Starting out
Posts: 12
Joined: Wed Jun 23, 2010 5:57 am

Re: RSYNC not working so far

Post by redghost »

On our network all the PCs we have (6 in total) are various flavours of windows (XP to 7), the QNAP NAS is Linux (I think), there is a raspberry pi running linux (openelec), a Humax PVR which I think is linux and a blackberry. I don't think there is anything else but definitely no Macs or i-anything (I don't know if that is good or bad but nobody has asked for one).

For the rsync you will log in to the NAS and execute from there either using a script or via SSH or Telnet. I don't know much about these things but I can't see that the other computers on the network will impact this. It sounds like your knowledge might exceed mine so can you tell me why it should matter?

Red
bsg75
Getting the hang of things
Posts: 84
Joined: Fri Dec 07, 2012 6:55 am

Re: RSYNC not working so far

Post by bsg75 »

My concern is when it comes to restore a full set of data and make sure that all data attributes, ACL (access control list), permissions, file access information, are restored as they were on the Windows platform. Most off site backup platforms are Linux based using a different file system to the Windows one. And most of backup companies don't talk much of this. Thanks to your post, I was able to visit rsync.net offer. These guys claim
Your filesystem is a normal filesystem, and will preserve permissions, access times, and other attributes of your files. You can preserve your NT filesystem ACLs on our filesystem.
.
This is really what we need. I downloaded the agent that manages backup: it seems to be an excellent piece of software. I might give a shot. The price tag is quite high though.
redghost
Starting out
Posts: 12
Joined: Wed Jun 23, 2010 5:57 am

Re: RSYNC not working so far

Post by redghost »

When I started with rsync.net I just asked if they could give me a lower rate because I was a home user. I was at a loss really and just wanted something I could use to back up my NAS (a buffalo terastation at the time) and I couldn't find anything else that would back up a NAS remotely. I was in touch with so many people to ask. Anyway having found them I just asked rsync.net for a reduced rate and as a home user they gave me a reduced rate. I don't know if they still do this kind of thing but you don't have anything to lose by asking.

If you need help getting it set up rsync.net are extremely helpful.

Red
bsg75
Getting the hang of things
Posts: 84
Joined: Fri Dec 07, 2012 6:55 am

Re: RSYNC not working so far

Post by bsg75 »

@redghost: Taking into account the troubles you recently had with rsync, did you consider to use rync.net Windows agent that could be installed on a PC?
Nicolas
redghost
Starting out
Posts: 12
Joined: Wed Jun 23, 2010 5:57 am

Re: RSYNC not working so far

Post by redghost »

It's a good question and you asking has made me think.

For me the purpose of the offsite backup is to provide me with diasater recovery. If a computer breaks and I need to restore the data onto a new one it is going to be quicker to restore from a local backup rather than pull it from a remote site. The set up I have is the PCs back up to the NAS and the NAS is backed up both locally (for *when* it fails so I can restore without having to pull the remote) and offsite in case the house burns down or some similar disaster. In the past I have ended up paying a lot of money to have a hard drive professionally recovered which, even though it worked, led to a lot of work sorting things out so I am paranoid about not losing anything in the future.

When I started doing offsite backups I had a very slow internet connection and it took hours and hours to make the backups so leaving the NAS to just do it was great. I remember the initial upload took days and days (weeks and weeks in fact) of non stop uploading at something less than 0.5 Mbps. I have recently looked at the backup agent on the rsync.net website and it turns out it is a cut-down version of superflexible file synchronizer (SFFS) which is exactly what I use a full version of on the PCs in the house to back them up to the NAS. It is very good and the support is excellent. I could use SFFS to make the backups to rsync.net but I'd still want to make a local backup and because I don't want to have all the computers tied up backing up offsite I would just rather do it on the NAS when everyone is asleep. However modern upload speeds are starting to make it a more realistic option to back up the PCs offsite directly.

As for the rsync over ssh I have it working on the NAS again now using the --max-size and --min-size -bwlimit commands. I've described this in the other thread so I'll spare you the details. Suffice to say it is working now but I have had to change things.

Red
bsg75
Getting the hang of things
Posts: 84
Joined: Fri Dec 07, 2012 6:55 am

Re: RSYNC not working so far

Post by bsg75 »

The way we handle users’ data at our client site is a bit different leading to a different approch: all users’ data in on the NAS with 4 disks under RAID 6 (2 disks may fail without interrupting production). In other words there is no important data on each PC. As a matter of fact, an image of each PC is still made from time to time, in order to be able to rebuild the PC very quickly in case of disk disaster, and a daily backup of users’ profile is made as well.
Then the NAS is backed up remotely every day. For the time being the embedded QNAP rsync facility is used. But it does not support rsync over SSH (to be exact it is supported only to another QNAP gear) and it does not support file encryption on the fly.
In that sense, rsync.net provides a nice solution, but using Super Flexible File Synchronizer rebranded Syncovery is a neat solution as well since it allows to contract with other off site back up service providers. I was asking myself would it be more efficient (and comfortable) to use the backup agent on a local PC (or server in our case) backing up a network share (ie the NAS), or directly installing the corresponding Linux agent on the QNAP gear, and let it do the job. However, I’m not very favorable (unless there is a strong argument for it) to the second solution, because no one knows exactly what happens in case of firmware upgrade, you’ve been a witness of such a problem by the way.
geva

Re: RSYNC not working so far

Post by geva »

Just for the sake of stating this, rsync over ssh not being supported to know on QNAP devices is only a problem in the web GUI interface. You can always issue RSync commands from the command line and using cron jobs to initiate an rsync job over ssh.

Sent from my Nexus 7 using Tapatalk HD
bsg75
Getting the hang of things
Posts: 84
Joined: Fri Dec 07, 2012 6:55 am

Re: RSYNC not working so far

Post by bsg75 »

You're absolutely right
Post Reply

Return to “Backup & Restore”