Rsync Pull with QNAP

Backup, Restore, Netbak Replicator, Cloud Storage Services
qnixx
New here
Posts: 4
Joined: Wed Feb 14, 2018 11:39 pm

Rsync Pull with QNAP

Post by qnixx » Thu Feb 15, 2018 12:15 am

I have several remote servers which are located on a DMZ network and host webservices on the internet. I want the ability to automatically and efficiently replicate the data from these servers to my QNAP which I have located within a seperate (secure) internal network. It is very important to me that the replication be scheduled on the QNAP and that no credentials be stored on these remote servers. From what I've read on this forum I already understand that neither the Backup Station or Hybrid Backup Station can perform a rsync pull from a remote server to the local QNAP. I can't imagine why such a feature would be excluded from the web UI but since the QNAP runs on linux I can set up a cron job to do this. I am posting because I want to know if anyone else has manually setup rsync pull on their QNAP. What have been your experiences and more importantly, is there a potential for the cron settings to be lost after a firmware upgrade?

I am thinking of following the instructions provided at this blog: https://www.contentecontent.com/blog/20 ... o-qnap-nas

User avatar
storageman
Ask me anything
Posts: 5511
Joined: Thu Sep 22, 2011 10:57 pm

Re: Rsync Pull with QNAP

Post by storageman » Thu Feb 15, 2018 1:14 am

In HBS can't you just create a Sync Local to Remote but pick your remote RSYNC server. This is pull replication.
I tested a pull from a Syno box using RSYNC

qnixx
New here
Posts: 4
Joined: Wed Feb 14, 2018 11:39 pm

Re: Rsync Pull with QNAP

Post by qnixx » Thu Feb 15, 2018 2:53 am

storageman, I have tried that and as the name implies I can only sync from local storage to remote storage. The remote server's storage is only listed in the destination drop down. To my knowledge the only supported method HBS has for replicating remote storage to local storage via Rsync is to enable the Rsync daemon on the QNAP and configure the remote server to act as the client and connect to the QNAP and do a push.

P3R
Guru
Posts: 12387
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Rsync Pull with QNAP

Post by P3R » Thu Feb 15, 2018 6:24 am

Does it have to be Rsync? Hybrid Backup Sync can do pull with RTRR, CIFS/SMB and FTP. Maybe FTP is possible instead of Rsync?
HBS_pull.jpg
You do not have the required permissions to view the files attached to this post.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

User avatar
storageman
Ask me anything
Posts: 5511
Joined: Thu Sep 22, 2011 10:57 pm

Re: Rsync Pull with QNAP

Post by storageman » Thu Feb 15, 2018 6:48 pm

Sorry, I too can't get pull to work either, the layout of the GUI confuses you into thinking the source can be a remote location.

P3R, you can't use RTRR for pull off anything other than Qnap (he is talking about remote servers). Can you show CIFS/SMB pull working from anything other than a Qnap?

I really don't think it is supported but I'm open to being corrected.

P3R
Guru
Posts: 12387
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Rsync Pull with QNAP

Post by P3R » Thu Feb 15, 2018 7:42 pm

storageman wrote:P3R, you can't use RTRR for pull off anything other than Qnap (he is talking about remote servers).
I know that but for completeness I listed all the protocols available for that particular use.
Can you show CIFS/SMB pull working from anything other than a Qnap?
I have a nightly scheduled pull from a non-Qnap server using CIFS/SMB.

Since the targets for the OP were web servers in a DMZ, I thought that FTP maybe would be a better fit than CIFS/SMB. I've never tested FTP though but I have no reason to think that it would be less successful than CIFS/SMB.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

User avatar
storageman
Ask me anything
Posts: 5511
Joined: Thu Sep 22, 2011 10:57 pm

Re: Rsync Pull with QNAP

Post by storageman » Thu Feb 15, 2018 7:48 pm

P3R wrote:
storageman wrote:P3R, you can't use RTRR for pull off anything other than Qnap (he is talking about remote servers).
I know that but for completeness I listed all the protocols available for that particular use.
Can you show CIFS/SMB pull working from anything other than a Qnap?
I have a nightly scheduled pull from a non-Qnap server using CIFS/SMB.

Since the targets for the OP were web servers in a DMZ, I thought that FTP maybe would be a better fit than CIFS/SMB. I've never tested FTP though but I have no reason to think that it would be less successful than CIFS/SMB.


Can you show how you do that in HBS? I can't get it to work.
I can create a remote SMB connection but can't find a way to use as the source.

P3R
Guru
Posts: 12387
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Rsync Pull with QNAP

Post by P3R » Thu Feb 15, 2018 7:54 pm

Of course, see my screenshot in the thread and start the setup wizard from there. Come back with specific issues if there are any.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

User avatar
storageman
Ask me anything
Posts: 5511
Joined: Thu Sep 22, 2011 10:57 pm

Re: Rsync Pull with QNAP

Post by storageman » Thu Feb 15, 2018 8:05 pm

P3R wrote:Of course, see my screenshot in the thread and start the setup wizard from there. Come back with specific issues if there are any.

Haha, the arrow goes the other way, stupid me, should have just tried it! Cheers!

temp.png
You do not have the required permissions to view the files attached to this post.

P3R
Guru
Posts: 12387
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Rsync Pull with QNAP

Post by P3R » Thu Feb 15, 2018 9:09 pm

:D
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

qnixx
New here
Posts: 4
Joined: Wed Feb 14, 2018 11:39 pm

Re: Rsync Pull with QNAP

Post by qnixx » Fri Feb 16, 2018 12:33 am

P3R wrote:Does it have to be Rsync? Hybrid Backup Sync can do pull with RTRR, CIFS/SMB and FTP. Maybe FTP is possible instead of Rsync?


I would prefer Rsync because it does have advantages over a SMB file copy, especially when you are dealing with large amounts of data. I already have all my Windows servers being pulled with SMB which does work nicely so now I just want to get my linux boxes backed up. If needed I'll just do it through the command line but I'm nervous that the cronjob could get wiped out for some unforseen reason and my backups will silently fail.

P3R
Guru
Posts: 12387
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Rsync Pull with QNAP

Post by P3R » Fri Feb 16, 2018 1:08 am

qnixx wrote:I would prefer Rsync because it does have advantages over a SMB file copy, especially when you are dealing with large amounts of data.
Yes on slow connections. On LAN(-like) connections it usually doesn't matter that much.
If needed I'll just do it through the command line but I'm nervous that the cronjob could get wiped out for some unforseen reason and my backups will silently fail.
As far as I know cron jobs usually stick but I wouldn't bet my right hand on it.

In Hybrid Backup Sync you have proper logging and even configurable mail notifications when there's a problem.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

qnixx
New here
Posts: 4
Joined: Wed Feb 14, 2018 11:39 pm

Re: Rsync Pull with QNAP

Post by qnixx » Fri Feb 16, 2018 10:19 pm

P3R wrote:In Hybrid Backup Sync you have proper logging and even configurable mail notifications when there's a problem.


The importance of this cannot be understated.

User avatar
storageman
Ask me anything
Posts: 5511
Joined: Thu Sep 22, 2011 10:57 pm

Re: Rsync Pull with QNAP

Post by storageman » Fri Feb 16, 2018 11:05 pm

qnixx wrote:
P3R wrote:In Hybrid Backup Sync you have proper logging and even configurable mail notifications when there's a problem.


The importance of this cannot be understated.


True, but not that detailed.

zebulon2
Getting the hang of things
Posts: 55
Joined: Mon Jan 01, 2018 5:20 pm

Re: Rsync Pull with QNAP

Post by zebulon2 » Fri Feb 16, 2018 11:52 pm

Thanks for this discussion, I was wonderin about the same thing. Besides, is it possible to use Hybrid Backup Sync to backup the files from a non-rooted Android phone?

Post Reply

Return to “Backup & Restore”