RemoTe 2-way Sync Queries

Discussion on remote replication.
Post Reply
MarkH000
New here
Posts: 4
Joined: Tue Aug 27, 2019 12:58 pm

RemoTe 2-way Sync Queries

Post by MarkH000 » Sat Sep 28, 2019 12:14 pm

I have two new TS-253Be-4G QNAP servers (RAID-1), both currently sitting in a Sydney office. The plan is to move one to Melbourne and use both as file servers for the two offices.

Note that I am not all that familiar with QNAP.

Both QNAP systems currently have the same data on them with the same file structures etc.. Both were setup by copying from an old (and now decommissioned) QNAP box.

I have loaded HBS3 on both systems and both have the latest firmware.

My questions:

1. I would like to set them up as a 2-way sync so that files updated at one location are reflected at the other location. Update activity is fairly low though some of the files are large data files. What is the best QNAP software to do this with (there seems to be more than one lot of software that does this or similar)? This is so both offices have the latest copy of the data rather than for backup purposes.

2. I am very keen to keep it so that we can use the current copies as the initial setup and avoid an initial syncing of all the data (hence the initial copying of the data to both boxes). Is there a way to start the sync by using the existing data?

3. I would like to trial it in Sydney before sending one box off to Melbourne. Does this provide a good test?

4. Any other items of good advice or gotchas I should be aware of?

Thank you in advance for any assistance and advice.

P3R
Guru
Posts: 12191
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: RemoTe 2-way Sync Queries

Post by P3R » Sat Sep 28, 2019 6:03 pm

MarkH000 wrote:
Sat Sep 28, 2019 12:14 pm
4. Any other items of good advice or gotchas I should be aware of?
2-way sync sounds fantastic in theory but what happen when the same file have been concurrently updated by different people at each server?

You need a connection with good bandwidth between the sites and of course VPN is a basic requirement for adequate security.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

MarkH000
New here
Posts: 4
Joined: Tue Aug 27, 2019 12:58 pm

Re: RemoTe 2-way Sync Queries

Post by MarkH000 » Sat Sep 28, 2019 6:43 pm

Thank you P3R. That is an important consideration.

File update activity is fairly low (they are primarily new data files that are loaded in one office or the other). So while that is a risk, it is a very small risk in this circumstance by the nature of the usage of the files.

So what do you consider the best approach - RTRR? or something else?

P3R
Guru
Posts: 12191
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: RemoTe 2-way Sync Queries

Post by P3R » Sat Sep 28, 2019 7:33 pm

MarkH000 wrote:
Sat Sep 28, 2019 6:43 pm
So what do you consider the best approach - RTRR? or something else?
HBS/RTRR is the only one I've tried briefly and know of but since I'm not a 2-way sync fan I'm not really updated on any alternatives.

Be aware that HBS is based on jobs (can be periodically scheduled) and doesn't run continously so if going this route you would have to experiment to find a good compromise schedule frequency where the job have time to finish (depend on the total amount of data, connection speed and the size of the changed data) before running again and still run often enough for your requirements.
2. I am very keen to keep it so that we can use the current copies as the initial setup and avoid an initial syncing of all the data (hence the initial copying of the data to both boxes). Is there a way to start the sync by using the existing data?
Yes that should work with folder pairs like NAS1/Public to NAS2/Public.
3. I would like to trial it in Sydney before sending one box off to Melbourne. Does this provide a good test?
For testing the concept yes but things will change when going live over the remote connection. Maybe you could try to rate limit down to the remote connection speed? I don't use rate limit either.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

User avatar
storageman
Ask me anything
Posts: 5511
Joined: Thu Sep 22, 2011 10:57 pm

Re: RemoTe 2-way Sync Queries

Post by storageman » Mon Sep 30, 2019 7:16 pm

Rules are here
https://www.qnap.com/solution/hbs3/en/
1 way sync to remote can be set realtime

Semantix4Arnold
New here
Posts: 6
Joined: Fri Feb 15, 2019 4:10 pm

Re: RemoTe 2-way Sync Queries

Post by Semantix4Arnold » Sun Dec 08, 2019 1:17 am

I have the same problem, different context: 2 QNAP NASses (TS419PII with QTS 4.3.3 and TS453A with QTS 4.4.1). I want to remotely 2-way sync nas1/multimedia with nas2/multimedia and nas1/public with nas2/public. Frequency overnight.
Currently the NASses stand next to each other on the same LAN. I can't get it working from either side. I HBS can't do this, is there another APP around that can?

P3R
Guru
Posts: 12191
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: RemoTe 2-way Sync Queries

Post by P3R » Sun Dec 08, 2019 1:37 am

Semantix4Arnold wrote:
Sun Dec 08, 2019 1:17 am
Currently the NASses stand next to each other on the same LAN. I can't get it working from either side.
Then you probably do something wrong. I have 2-way sync working with an even older HBS-version on the remote site than the one you have (assuming you use the latest available) on the TS-419 P II so I'm sure that it must be possible to get it working.

If you provide information maybe we can help you solve the problem?
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

Semantix4Arnold
New here
Posts: 6
Joined: Fri Feb 15, 2019 4:10 pm

Re: RemoTe 2-way Sync Queries

Post by Semantix4Arnold » Sun Dec 08, 2019 2:36 am

P3R wrote:
Sun Dec 08, 2019 1:37 am
Semantix4Arnold wrote:
Sun Dec 08, 2019 1:17 am
Currently the NASses stand next to each other on the same LAN. I can't get it working from either side.
Then you probably do something wrong. I have 2-way sync working with an even older HBS-version on the remote site than the one you have (assuming you use the latest available) on the TS-419 P II so I'm sure that it must be possible to get it working.

If you provide information maybe we can help you solve the problem?
Where to start -
I have the same users authorized with admin rights.
HBS 2.1.190909 on TS-419 Pii; HBS 3.0.191202 on TS-453A
Both directories to be synchronized are visible on windows clients through 2 shares
there are two networkpaths available adapter 1 on TS-419Pii to adapter 2 on TS-453a (network 1) and/or adapter 1+3 on TS453a to adapter 2 on TS-419pii (network 2) all gigabit speed; network 1 has access to network 2 through NAT (not vice versa). Workstations are on network 2.
All services are enabled on all interfaces.
All admin users are enabled for SSH
What other relevant information is missing?
I tried to follow instruction movie on Youtube but systems do not exactly work like in the movie. systems seem to see eachother but apparently are not authorized appropriately?

P3R
Guru
Posts: 12191
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: RemoTe 2-way Sync Queries

Post by P3R » Sun Dec 08, 2019 4:22 am

That was a good start.

Did you create the job on the TS-453A? If not do that, the opposite would probably be more likely to have issues.

Network path used should be decided by what remote ip address you designate. Are they in the same ip network or are there routers and/or a VPN in the path as well?

What's the reason for the complicated multipath networking? Is this in a corporate network?

What error messages do you get?
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

Semantix4Arnold
New here
Posts: 6
Joined: Fri Feb 15, 2019 4:10 pm

Re: RemoTe 2-way Sync Queries

Post by Semantix4Arnold » Sun Dec 08, 2019 8:20 am

I tried in both directions and get essentially the same problems:
When I select 2-way sync, I cannot select Rsync but can only continue with RTRR which then fails to connect (message "failed to connect")
When I try 1-way sync, I can select Rsync but get error message "Wrong port or server not started" which is not true: server is started (I think) on port 873
I tried IP-addresses on both networks, same result. I probably am doing something wrong but didn't figure out what

P3R
Guru
Posts: 12191
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: RemoTe 2-way Sync Queries

Post by P3R » Sun Dec 08, 2019 6:54 pm

It could be either you configuring something wrong or network connectivity issues between the two Qnaps. Without more information supplied, It's not possible to even try to assist in finding out which it is.

Please contact Qnap support.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

User avatar
storageman
Ask me anything
Posts: 5511
Joined: Thu Sep 22, 2011 10:57 pm

Re: RemoTe 2-way Sync Queries

Post by storageman » Thu Dec 12, 2019 10:44 pm

U sure RSYNC service started on both boxes.
HB3 Services tab for one.

P3R
Guru
Posts: 12191
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: RemoTe 2-way Sync Queries

Post by P3R » Fri Dec 13, 2019 3:04 am

2-way sync isn't supported with Rsync (only RTRR) but of course the same apply, the RTRR server need to be manually enabled at both ends.

Edit: when I started to think more about this it wasn't as clear as I first thought. Sure enough, when I test it I see that 2-way sync work even with the RTRR server started only on the remote system from the one that run the job.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

Post Reply

Return to “Remote Replication/ Disaster Recovery”