RTRR not working between TS-639 and TS1079

Discussion on remote replication.
Post Reply
dulcifium
Been there, done that
Posts: 720
Joined: Fri Oct 23, 2009 6:52 pm

RTRR not working between TS-639 and TS1079

Post by dulcifium »

I am trying to get all files from TS-639 to TS-EC1079. About 12 TB. Both devices on a 1GbE switch on same subnet. Manual copying via a PC maxes out the GbE but as this is two ways on the same wire is limited to <50MB/s. sshing in and doing a manual copy via FTP maxes out the GbE too. However, none of the built-in methods work.

I'm trying to use Hybrid Backup Sync on the TS-EC1079.
  • I can't do it with the "Restore" option. I click Remote NAS then try to set up a new profile inputting the 639 IP and password. When I click Test it simply says "Failure". I suspect this is because there is no way to make the 639 "listen" on the RTRR port.
  • I can't do it by creating a Remote storage space. When I try to set up a new profile inputting the 639 IP and password. When I click Test it simply says "Failure". I suspect this is because there is no way to make the 639 "listen" on the RTRR port.
  • I can't do it with One Way Sync as this only allows local to remote and not vice versa.
  • I can't do it with Two Way Sync with CIFS. I input the 639 IP address and credentials and it says "Failure". I've tried both the admin account and other accounts that have access to the shares.
  • I can't do it with Two Way Sync with RTRR. I input the 639 IP address and credentials and it says "Failure". I've tried both the admin account and other accounts that have access to the shares.
  • I can't do it with ActiveSync with FTP. I just got this to work for the first time. Unfortunately transfer speeds 5-10MB/s and it doesn't seem to preserve timestamps which is essential for me :?
Then I tried to it from the TS-639 but this doesn't work either.
  • I can't do it with RTRR. I add a new replication job and choose RTRR and the TS-1079 IP and admin password. It won't connect. I have enabled RTRR server on the correct port and allowed all connections on the 1079 but there's no way to make the 639 see it for transfers.
  • I can't do it with FTP. Even though I can FTP correctly between the two via command line, when I try to go on 639: Remote Replication -> Local to Remote -> FTP and enter the exact same FTP settings, it says wrong username or password. I have enabled FTP on the remote 1079. (edit, fixed by using another username other than admin, however, although the test can achieve speeds of >100MB/s as can manual FTP, QNAP RTRR FTP goes at <5MB/s so it's still too slow to be usable for a full seed.)
Manual copying via a PC maxes out 1GbE as does manual FTP. Even the FTP test done by QNAP when setting up ActiveSync gives speeds of 100MB/s!

How on earth do I seed the 1079 at a reasonable rate without having to wait weeks? Right now it would be much quicker to get an external HDD and run two USB 3.0 copy operations rather than use my 1GbE.
QNAP TS-EC1079 Pro (4.3.4.0435) | 10 x 4 TB (WD WDC WD40EFRX firmware 82.00A82) | 32 TB RAID-6
QNAP TS-639 Pro (3.8.1 Build 20121205) | 6 x 3 TB (Toshiba DT01ACA300 firmware MX6OABB0) | 12 TB RAID-6
Both shared over single port GbE and wifi router to Linux, Android, Windows, macOS and NVIDIA Shield
P3R
Guru
Posts: 13190
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: RTRR not working between TS-639 and TS1079

Post by P3R »

dulcifium wrote:Manual copying via a PC maxes out the GbE but as this is two ways on the same wire is limited to <50MB/s.
GbE by default is full duplex so can transfer at full gigabit speed both up and down on the same cable at the same time. If you see less than wire speed, I'd think that's because of client performance limitations. Copying multiple small files is normally slower than a single large file.
I suspect this is because there is no way to make the 639 "listen" on the RTRR port.
Why? Have you enabled RTRR server on the correct port (in Backup Station, Backup Server, RTRR Server)?
I have enabled RTRR server on the correct port and allowed all connections on the 1079 but there's no way to make the 639 see it for transfers.
Backup Station is released software integrated in QTS while Hybrid Backup Sync is an add on still in beta testing.

When I recommended you to use RTRR to be used for this bulk transfer purpose I meant using Backup Station/RTRR on both sides as I don't recommend the use of beta software on anything than non-production lab NASes.

From your post it seems the respective RTRR-implementations in Backup Station and Hybrid Backup Sync may be incompatible but I have no idea as I don't use HBS. The other alternative is that your almost 5 year old and totally outdated TS-639 QTS firmware is an issue here.

Please contact Qnap support for assistance.
Last edited by P3R on Wed Aug 23, 2017 11:52 pm, edited 1 time in total.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
dulcifium
Been there, done that
Posts: 720
Joined: Fri Oct 23, 2009 6:52 pm

Re: RTRR not working between TS-639 and TS1079

Post by dulcifium »

Thanks for your reply.
P3R wrote:
dulcifium wrote:Manual copying via a PC maxes out the GbE but as this is two ways on the same wire is limited to <50MB/s.
GbE by default is full duplex so can transfer at full gigabit speed both up and down on the same cable at the same time. If you see less than wire speed, I'd think that's because of client performance limitations. Copying multiple small files is normally slower than a single large file.
Okay. Will move a PC to where the NAS are and try again. Unfortunately only have ac wifi easily accessible.
P3R wrote:
I suspect this is because there is no way to make the 639 "listen" on the RTRR port.
Why? Have you enabled RTRR server on the correct port (in Backup Station, Backup Server, RTRR Server)?
Aha! I was using the Backup tab the whole time. I never scrolled down far enough under Applications to find "Backup Sever". I have no "Backup Station" on the 639. I'm trying to test this now but clicking stop on the 1079 hasn't done anything after an hour.
P3R wrote:
I have enabled RTRR server on the correct port and allowed all connections on the 1079 but there's no way to make the 639 see it for transfers.
Backup Station is released software integrated in QTS while Hybrid Backup Sync is an add on still in beta testing.
There is no sign of any "Backup Station" on the 1079 either. If I type "Backup Station" into search and click on it, it simply opens up the Hybrid Sync thing. I tried to find out whether there was another Backup programme on the 1079 by googling and couldn't get any info either.
P3R wrote:When I recommended you to use RTRR to be used for this bulk transfer purpose I meant using Backup Station/RTRR on both sides as I don't recommend the use of beta software on anything than non-production lab NASes.

From your post it seems the respective RTRR-implementations in Backup Station and Hybrid Backup Sync may be incompatible but I have no idea as I don't use HBS. The other alternative is that your almost 5 year old and totally outdated TS-639 QTS firmware is an issue here.
As above, Hybrid claims to use RTRR. There is no Backup Station on either the 639 or 1079 nor any way of getting such an application that I can find.
QNAP TS-EC1079 Pro (4.3.4.0435) | 10 x 4 TB (WD WDC WD40EFRX firmware 82.00A82) | 32 TB RAID-6
QNAP TS-639 Pro (3.8.1 Build 20121205) | 6 x 3 TB (Toshiba DT01ACA300 firmware MX6OABB0) | 12 TB RAID-6
Both shared over single port GbE and wifi router to Linux, Android, Windows, macOS and NVIDIA Shield
dulcifium
Been there, done that
Posts: 720
Joined: Fri Oct 23, 2009 6:52 pm

Re: RTRR not working between TS-639 and TS1079

Post by dulcifium »

I have just enabled RTRR server on the 639 and now the 1079 can see it. When I test I get a speed of 80 MB/s. However, when I start the job, it says:

Code: Select all

Job Remote Restore 1 failed: Server is terminated by a request!Please check Network Access Protection in server setting page, or Incoming List page..
There is nothing in either server's settings about this and, as I said, the test works fine. (As an aside, I still can't cancel the FTP jobs). I'm at my wit's end with this! Been trying for over 12 hours now. They are both on the same network, 3rd party copy and paste would work fine, is there really no straightforward terminal command to initially seed the QNAP?
QNAP TS-EC1079 Pro (4.3.4.0435) | 10 x 4 TB (WD WDC WD40EFRX firmware 82.00A82) | 32 TB RAID-6
QNAP TS-639 Pro (3.8.1 Build 20121205) | 6 x 3 TB (Toshiba DT01ACA300 firmware MX6OABB0) | 12 TB RAID-6
Both shared over single port GbE and wifi router to Linux, Android, Windows, macOS and NVIDIA Shield
P3R
Guru
Posts: 13190
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: RTRR not working between TS-639 and TS1079

Post by P3R »

dulcifium wrote:Unfortunately only have ac wifi easily accessible.
WiFi, different from cabled gigabit networking, is only half duplex so there you will at best have half the speed in that same scenario. Even worse, WiFi is a medium shared between all clients so any other traffic from them will also lower the usable bandwidth.
I have no "Backup Station" on the 639.
Okay, I didn't think of that Backup Station wasn't around in such ancient firmware as the one you have in the TS-639. Backup Station, being a portal to all backup features, was probably introduced in QTS version 4.0 or 4.1.
There is no sign of any "Backup Station" on the 1079 either. If I type "Backup Station" into search and click on it, it simply opens up the Hybrid Sync thing. I tried to find out whether there was another Backup programme on the 1079 by googling and couldn't get any info either.
You could have looked in the documentation. Backup Station have by default been present in QTS firmware for many years.

However if I'm not wrong when you installed Hybrid Backup Sync, it at the same time removed (or at least disabled?) Backup Station. :cry:

I don't know how to get Backup Station back. Maybe uninstall Hybrid Backup Sync (if possible) and reapply the QTS firmware? Please ask Qnap support.
As above, Hybrid claims to use RTRR.
Absolutely but the implementation may be different from in Backup Station and the now even earlier RTRR you're using on the TS-639. I'm not saying it is but it's a possibility since you're having problems with them interacting.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
User avatar
Trexx
Ask me anything
Posts: 5393
Joined: Sat Oct 01, 2011 7:50 am
Location: Minnesota

Re: RTRR not working between TS-639 and TS1079

Post by Trexx »

You might try doing a manual update to the new 4.3.3.262 build as there were a couple RTRR related fixes in it.
Paul

Model: TS-877-1600 FW: 4.5.3.x
QTS (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
Data (HDD): [RAID-5] 6 x 3TB HGST DeskStar
VMs (SSD): [RAID-1] 2 x1TB SK Hynix Gold
Ext. (HDD): TR-004 [Raid-5] 4 x 4TB HGST Ultastor
RAM: Kingston HyperX Fury 64GB DDR4-2666
UPS: CP AVR1350

Model:TVS-673 32GB & TS-228a Offline[/color]
-----------------------------------------------------------------------------------------------------------------------------------------
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq
P3R
Guru
Posts: 13190
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: RTRR not working between TS-639 and TS1079

Post by P3R »

Trexx wrote:You might try doing a manual update to the new 4.3.3.262 build as there were a couple RTRR related fixes in it.
But wouldn't those fixes only be related to Backup Station/RTRR that's a part of QTS and not to Hybrid Backup Sync/RTRR that's in use here?
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
User avatar
Trexx
Ask me anything
Posts: 5393
Joined: Sat Oct 01, 2011 7:50 am
Location: Minnesota

Re: RTRR not working between TS-639 and TS1079

Post by Trexx »

P3R wrote:
Trexx wrote:You might try doing a manual update to the new 4.3.3.262 build as there were a couple RTRR related fixes in it.
But wouldn't those fixes only be related to Backup Station/RTRR that's a part of QTS and not to Hybrid Backup Sync/RTRR that's in use here?
You never know :)
Paul

Model: TS-877-1600 FW: 4.5.3.x
QTS (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
Data (HDD): [RAID-5] 6 x 3TB HGST DeskStar
VMs (SSD): [RAID-1] 2 x1TB SK Hynix Gold
Ext. (HDD): TR-004 [Raid-5] 4 x 4TB HGST Ultastor
RAM: Kingston HyperX Fury 64GB DDR4-2666
UPS: CP AVR1350

Model:TVS-673 32GB & TS-228a Offline[/color]
-----------------------------------------------------------------------------------------------------------------------------------------
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq
dulcifium
Been there, done that
Posts: 720
Joined: Fri Oct 23, 2009 6:52 pm

Re: RTRR not working between TS-639 and TS1079

Post by dulcifium »

Okay I've now updated the firmware and done some more investigation. Still a little unsure what's going on.

Doing manual FTP transfers from 639->1079 I get constant speeds of 100MB/s.
Using RTRR from 639->1079 I only get about 10MB/s.
Copy/pasting via a PC connected by 1GbE I get about 50MB/s.

I wish there were a way to use FTP to do transfers of multiple files...

Doing manual FTP transfers from 1079->639 I only get speeds of 10MB/s. CPU is maxing out about 20% so I don't think that's the bottleneck. In fact, look at this graph of FTP transfers:

https://www.dropbox.com/s/jmyhyomdm3zpy ... 1.png?dl=0

Any ideas why transfers into the 639 would be so slow?!
QNAP TS-EC1079 Pro (4.3.4.0435) | 10 x 4 TB (WD WDC WD40EFRX firmware 82.00A82) | 32 TB RAID-6
QNAP TS-639 Pro (3.8.1 Build 20121205) | 6 x 3 TB (Toshiba DT01ACA300 firmware MX6OABB0) | 12 TB RAID-6
Both shared over single port GbE and wifi router to Linux, Android, Windows, macOS and NVIDIA Shield
P3R
Guru
Posts: 13190
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: RTRR not working between TS-639 and TS1079

Post by P3R »

dulcifium wrote:Okay I've now updated the firmware and done some more investigation.
So now you're using what?
Using RTRR from 639->1079 I only get about 10MB/s.
Using what RTRR-configuration?
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
dulcifium
Been there, done that
Posts: 720
Joined: Fri Oct 23, 2009 6:52 pm

Re: RTRR not working between TS-639 and TS1079

Post by dulcifium »

P3R wrote:
dulcifium wrote:Okay I've now updated the firmware and done some more investigation.
So now you're using what?
The three protocols I mentioned. FTP, file copy via PC, RTRR.
Using RTRR from 639->1079 I only get about 10MB/s.
Using what RTRR-configuration?[/quote]

RTRR over FTP (test speed 90MB/s+) with jobs hosted on 1079.
QNAP TS-EC1079 Pro (4.3.4.0435) | 10 x 4 TB (WD WDC WD40EFRX firmware 82.00A82) | 32 TB RAID-6
QNAP TS-639 Pro (3.8.1 Build 20121205) | 6 x 3 TB (Toshiba DT01ACA300 firmware MX6OABB0) | 12 TB RAID-6
Both shared over single port GbE and wifi router to Linux, Android, Windows, macOS and NVIDIA Shield
P3R
Guru
Posts: 13190
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: RTRR not working between TS-639 and TS1079

Post by P3R »

dulcifium wrote:
P3R wrote:
dulcifium wrote:Okay I've now updated the firmware and done some more investigation.
So now you're using what?
The three protocols I mentioned. FTP, file copy via PC, RTRR.
You said that you updated the firmware so my question was to what firmware? Are you no longer using 5 year old firmware in the TS-639?
dulcifium wrote:
P3R wrote:
dulcifium wrote:Using RTRR from 639->1079 I only get about 10MB/s.
Using what RTRR-configuration?
RTRR over FTP (test speed 90MB/s+) with jobs hosted on 1079.
FTP as transport protocol in RTRR have limitations and it was added only for compatibility with non-Qnap units. With two Qnaps you should be using the RTRR protocol. If you can't get RTRR to work between two Qnaps, you have basic compatibility problems that in my opinion should be solved instead of chasing performance over a very odd connection that most likely very few have any experience with.

When I asked for the RTRR-configuration I meant what type of RTRR-job with what options enabled. But if you still use FTP as transport protocol, it doesn't really matter for me as I have no experience with FTP in RTRR.

Generally speaking, copying a single large file is always quicker than copying a bunch of small files.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
dulcifium
Been there, done that
Posts: 720
Joined: Fri Oct 23, 2009 6:52 pm

Re: RTRR not working between TS-639 and TS1079

Post by dulcifium »

P3R wrote:
dulcifium wrote:
P3R wrote:
dulcifium wrote:Okay I've now updated the firmware and done some more investigation.
So now you're using what?
The three protocols I mentioned. FTP, file copy via PC, RTRR.
You said that you updated the firmware so my question was to what firmware? Are you no longer using 5 year old firmware in the TS-639?
No, I updated the firmware in the 1079. I didn't dare update the firmware in the 639 as I've had nothing but issues with that. But I restarted it today and did an update. Now, after booting, and trying to log into admin, the admin panel has been loading for over an hour with nothing showing except "Loading"...
Last edited by dulcifium on Thu Aug 24, 2017 12:07 am, edited 2 times in total.
QNAP TS-EC1079 Pro (4.3.4.0435) | 10 x 4 TB (WD WDC WD40EFRX firmware 82.00A82) | 32 TB RAID-6
QNAP TS-639 Pro (3.8.1 Build 20121205) | 6 x 3 TB (Toshiba DT01ACA300 firmware MX6OABB0) | 12 TB RAID-6
Both shared over single port GbE and wifi router to Linux, Android, Windows, macOS and NVIDIA Shield
P3R
Guru
Posts: 13190
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: RTRR not working between TS-639 and TS1079

Post by P3R »

As I couldn't help with the odd combination you use I recommended already in my first post here, that you contact Qnap support for assistance. I guess you now have yet another issue for them. :cry:
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
Post Reply

Return to “Remote Replication/ Disaster Recovery”