NAS to NAS Replication Slow

[JustADudeinIT]

Hi I have two sites connected via VPN that have Nas to Nas file level replication running to backup changes to a 256GB file share of photos each night at 6pm.
We had an internet outage for a few days and since then the job been running well.

Once the link came back online the job progress got stuck at 1%. Each night the job progress's by an extra 1% each evening and we are currently at 7%.
We have to stop the copy job manually in the morning as it consumes quite a bit of network resources.

All the files are still at the destination site, There is less than a GB of data that needs to be transferred.
Both devices are a TS-221 running firmware 4.4.2

Are there any logs we can access to get better visibility on the issue?
Any advice would be greatly appreciated.

[storageman]

Using what RTRR or RSYNC?
If the link is reliable I would be using RTRR.

[JustADudeinIT]

Thanks for the reply storageman,
RSYNC is enabled and the job is setup as a NAS to NAS file-level backup.
The purpose of the job is to produce a nightly offsite backup of the data over an ADSL connection.

RTRR is cable of real time replication (which we don't need) but I understand that we can set a schedule so that the sync runs out of hours. Is this the prefered method of replication?
Could you elaborate on why you would use RTRR?

[P3R]

Both devices are a TS-221...

What's the CPU load when running the backup?

What disk models are used?

...running firmware 4.4.2

As far as I know QTS 4.4.2 isn't released yet but perhaps you mean 4.2.4?

Are there any logs we can access to get better visibility on the issue?

There should be a log visible if you check the job within the application. I don't remember how though but look around and you should find it. I think there's should be some extended logging option to enable in there as well. Then again, as you abort the the job the log may not be that useful.

...over an ADSL connection.

What's the lowest of the sending side upload speed and receiving side download speed, in other words the bandwidth bottleneck?

RTRR is cable of real time replication (which we don't need)...

RTRR can run in many modes, real time being only one of them.

I always only use RTRR for my backup needs but never use real time mode as it's in my opinion a poor choice for backups.

Could you elaborate on why you would use RTRR?

RTRR is the most feature-rich and best supported backup protocol. It also put less strain on low end models with weaker CPUs. RTRR generally requires more bandwidth though so may not be the best over very slow connections.

[JustADudeinIT]

Thanks P3R, testing this out tonight.

Tested RTRR on a 50Mb small directory and it synced in about 30mins.
The full drive ran for most of the night and didn't get past 0%.

I'm guessing things have changed since 2013.
viewtopic.php?t=79886

"The only time I would prefer RTRR would be if (near) real time remote replication or ftp protocol is a requirement."

CPU Load during the NAS to NAS backup is about 30% but the Network Bandwidth is 60KB/s
Disk models are Seagate ST100VM002-1CT1SC23's
It's actually running 4.2.2
Your right, the NAS to NAS log doesn't show anything except rync: cancelled by user and while the job is running I am unable to view the logs.

[P3R]

I'm guessing things have changed since 2013.

Your guess is correct. Since years back I almost only use RTRR for backups.

What are the up and download bandwidth specification for the two internet connections in your VPN?

[JustADudeinIT]

Sorry for the late reply.
The Main location gets 6mbps down and 0.5mbps up.
The backup site has 28mbps down and 12mbps up.

The job has been running each night and is getting cancelled each morning we are at 40%.

[P3R]

...and 0.5mbps up.

Well this is an awful bottleneck to run a backup over.

Previously you said:

...but the Network Bandwidth is 60KB/s

So the internet connction is used up completely.

The job has been running each night and is getting cancelled each morning we are at 40%.

As long as you have that extremely low upload at the main site, backups over it will be painful. Be happy that it works at all and that it eventually will catch up.

If the upload can't be increased, I would strongly consider to complement the current backup with at least two external backup disks rotated and acrried back and forth, with one always kept off-site.

[JustADudeinIT]

Thanks P3R.
Yeah it's not great unfortunately and there isn't a faster link available in the area.

The majority of the data is on both devices already the job just needs to recognize that they are in sync. Once the two drives run a successful sync the amount of change data each night will pass within 60mins. The quickest solution would be to disconnect the NAS move it to the main center and sync it locally but home drives are also stored on this device and it would disrupt work for a couple of days.

Next time we hit the issue and loose the backup sync.
I will change the job to a RTRR and see if this is a more stable option.

[P3R]

Next time we hit the issue and loose the backup sync.
I will change the job to a RTRR and see if this is a more stable option.

Don't waste you time! RTRR is okay on LAN-like media, the faster the better, but Rsync is the only protocol that (barely) work with such very low bandwidth connections.

[storageman]

Until I hit a serious backup fail I'm not sure that is more detailed but thanks for clarifying.

As this guy found:
viewtopic.php?f=15&p=659399#p659399

[P3R]

@storageman, you replied to the wrong backup thread so I move here with you (this is the other thread). It's now very confusing...

Until I hit a serious backup fail I'm not sure that is more detailed but thanks for clarifying.

It most certainly is more detailed and does contain individual actions on files. If it's detailed enough for your taste I don't know but if you haven't even looked at it, that would be a great first step.

As this guy found:
viewtopic.php?f=15&p=659399#p659399

Well in that thread I became aware that apparently logging is different on (at least some) cloud backups (I don't use cloud for backup). I do know that the detailed downloadable log file is available on local and remote jobs, at least when using the RTRR and CIFS/SMB protocols.