Move large amount of data (NAS to NAS)

Backup, Restore, Netbak Replicator, Cloud Storage Services
Post Reply
ss911der
Starting out
Posts: 47
Joined: Sat Aug 07, 2010 7:53 pm

Move large amount of data (NAS to NAS)

Post by ss911der »

Hello,

I have a QNAP TS-870 Pro that has one specific share that I need to move to another NAS TVS-871. There's about 7 TB of data in this share (all large file sizes). I have an 8 port gigabit switch (not managed or smart) that I was hoping to put to good use or have ethernet wired directly between a few of the ethernet interfaces on the NASes (in addition to being wired to my router for control obviously) to maximize the transfer rate... Is that possible? I also read that file manager gui would be a decent way to do the transfer?
User avatar
Don
Guru
Posts: 12289
Joined: Thu Jan 03, 2008 4:56 am
Location: Long Island, New York

Re: Move large amount of data (NAS to NAS)

Post by Don »

Trunking the interfaces won't help you. It only helps many to one connections and not a one to one connection. Try using RTRR to transfer the files. The file manager gui will take too long.
Use the forum search feature before posting.

Use RAID and external backups. RAID will protect you from disk failure, keep your system running, and data accessible while the disk is replaced, and the RAID rebuilt. Backups will allow you to recover data that is lost or corrupted, or from system failure. One does not replace the other.

NAS: TVS-882BR | F/W: 5.0.1.2346 | 40GB | 2 x 1TB M.2 SATA RAID 1 (System/VMs) | 3 x 1TB M.2 NMVe QM2-4P-384A RAID 5 (cache) | 5 x 14TB Exos HDD RAID 6 (Data) | 1 x Blu-ray
NAS: TVS-h674 | F/W: 5.0.1.2376 | 16GB | 3 x 18TB RAID 5
Apps: DNSMasq, PLEX, iDrive, QVPN, QLMS, MP3fs, HBS3, Entware, DLstation, VS, +
P3R
Guru
Posts: 13192
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Move large amount of data (NAS to NAS)

Post by P3R »

I agree with Don.

If using a single copy instance only, Backup Station, RTRR offer the best combination of convenience and performance. In the RTRR-job at least the following options should be disabled for best performance:
Check file contents
Compress files during transmission
Detect sparse files
Delete extra files
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
ss911der
Starting out
Posts: 47
Joined: Sat Aug 07, 2010 7:53 pm

Re: Move large amount of data (NAS to NAS)

Post by ss911der »

Darn. Well I guess 18 hrs to transfer isn't too bad. Hopefully the cost of a 8 port 10 gbe switch will come down soon. Thanks for the reply!
Teddy73
New here
Posts: 5
Joined: Sun Jan 24, 2016 8:17 pm

Re: Move large amount of data (NAS to NAS)

Post by Teddy73 »

...
JUPPRAZA
New here
Posts: 5
Joined: Sat Jan 30, 2016 9:06 am

Re: Move large amount of data (NAS to NAS)

Post by JUPPRAZA »

I am having difficulties following the advice here.

I'm trying to pull data from my old nas (synology) using RTRR (single event rsync) function in my new TS-453a without success.

I'm using QTS 4.2.1, when in the "Configure Remote Host Settings" dialogue, there doesn't seem to be any variation of IP Address/Host Name and Password that connects to my device. I'd be grateful if someone could steer me in the correct direction - having selected the Synchronize/ Remote-folder-to-a-local-folder option, I've been using the following input details:

IP Address: 192.168.1.66 (this is the address of the source device)
Server type: RTRR
Port: as default
Password: the same password i used to enable the Backup Server/ RTRR Server


My questions are:

for the above IP Address, should this be that of the source (old nas containing the data) or the TS-453a required target?

when I test these settings, i receive a message that says: "Invalid port number or the service is disabled". when i change the port number to 22 (the default port on the synology) and retest, I get a different message that says "I need to enable this option to obtain a log!". can you advise where I should be setting the port number?

in the following dialogue, "Configure Multiple Folder Pairs". the Remote source folder doesn't display any of the old nas's folders and doesn't offer any selections! the Local destination folder does display a list of the folders in my QNAP. its obvious I've missed one or two setup steps somewhere, but can't see it???
P3R
Guru
Posts: 13192
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Move large amount of data (NAS to NAS)

Post by P3R »

JUPPRAZA wrote:I am having difficulties following the advice here.

I'm trying to pull data from my old nas (synology) using RTRR (single event rsync)...
As far as I know RTRR is Qnap proprietary and it isn't compatible with rsync so I'm not surprised that it doesn't work for you.

I can think of two options:
Use RTRR over FTP or CIFS/SMB instead
OR
Enable the rsync server on the Qnap and have the Synology push the data to the TS-453A.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
JUPPRAZA
New here
Posts: 5
Joined: Sat Jan 30, 2016 9:06 am

Re: Move large amount of data (NAS to NAS)

Post by JUPPRAZA »

Thank you P3R.

That's really odd, because the manual seems to imply that the remote server can be any rsync enabled device so I suspected it was a user error! I'll review the options you've suggested.

thanks again
P3R
Guru
Posts: 13192
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Move large amount of data (NAS to NAS)

Post by P3R »

Unfortunately Qnap have NAS to NAS, Rsync and RTRR on the same Remote Replication page in the documentation. To complicate things further, NAS to NAS and Rsync are under the same heading without any explanation of the difference... :roll:

In fact those three options are all different:
  • NAS to NAS - based on rsync but optimized for when both sides are Qnaps. Should really have been called Qnap to Qnap.
  • Rsync - Obviously based on rsync but when the destination is a a general non-Qnap rsync-server.
  • RTRR - An extremely flexible and feature rich alternative with three different transport protocols (RTRR, FTP and CIFS/SMB) and many different use cases. Features and possibilities may be limited if not using RTRR transport.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
portedaix
New here
Posts: 8
Joined: Mon May 20, 2013 1:09 am

Re: Move large amount of data (NAS to NAS)

Post by portedaix »

This thread is old, but the question asked is still valid. I tried several ways to copy as fast as possible, with a safe method, data from one NAS to another. RTRR is very safe but slow. FTP transfer with Winscp is not faster, and probably not very safe. Actually the fastest way I found is to simply use my windows PC and copy a shared folder from NAS1 to another shared folder from NAS2. I guess this is a safe method. To make sure everything is right, after copy completion, I intend to run RTRR from NAS1 to NAS2 with "check file content" option activated, or something like that if that is the correct translation into english.
PS : I event tried iscsi and virtual disk, which gives a good speed, but it is not possible to read the data from the target itself. iscsi is not intended for that usage.
Post Reply

Return to “Backup & Restore”