Nas to Nas stop long running backup

Backup, Restore, Netbak Replicator, Cloud Storage Services
Post Reply
pollardd
Starting out
Posts: 29
Joined: Wed Jan 03, 2018 8:43 am

Nas to Nas stop long running backup

Post by pollardd »

Hi Guys,
I have just configured a NAS to NAS backup and that seems fairly straight forward. An initial test works well.
I see I can restrict the bandwidth at either end of the connection which is great.

I want my backups to run at night during hours when my users aren't using the internet connection.
If I get a long transfer that goes overnight and into business hours the next morning I want to end the process at a scheduled time say around 8:45am.

I can't see a way to end a long running backup.
Does anyone have some ideas?

Thanks
David
P3R
Guru
Posts: 13190
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Nas to Nas stop long running backup

Post by P3R »

No I know of way to do it and to schedule a shutdown of a running backup wouldn't be a solution anyway. It would lead to incomplete and therefore broken and unreliable backups. Backups are the most important thing you have to protect your data and if they don't complete in an expected time frame, THAT'S the problem that need to be fixed.

Rethink the backup strategy so that it works in accordance with your needs. Remove the bottleneck that's slowing the backup down or do smaller chunks of backup each night.

If the backup is very close to finish in time but often spills over into business hours (and that's a problem) a solution would be to get a decent firewall that can dynamically and scheduled throttle the specific connection down to an acceptable level (assuming there's enough bandwidth to at least still have the backup running during the day, even if it's slow).

If it's only a matter of an initial very large backup and that later incremental backups are expected to end in time, then 'seed' the backup unit with a complete backup either by bringing the units together on a local network or by physically moving an external disk with the data between the two NASes.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
pollardd
Starting out
Posts: 29
Joined: Wed Jan 03, 2018 8:43 am

Re: Nas to Nas stop long running backup

Post by pollardd »

Hi P3R,
I have a very similar process already working but the source end is an old windows machine that holds the onsite bacup and I need to pension off this machine. The destination is already a QNAP device. I do position the initial backups at the remote location manually with a USB disk to avoid transferring huge files.
Currently on the windows machine I use xcopy to send the files to the remote qnap device. It uses the archive bit to determine if the file has been successfully transfered or not. On the odd occasion where the transfer has not completed by the start of business I terminate the process automatically and the archive bit remains set, so the transfer can start again either at slower speed or the following evening.
During business hours I do a slow xcopy transfer that uses about 30% of the bandwidth and staff don't notice that.
I do have decent cisco routers that provide a VPN between sites but using this to limit bandwidth will be difficult. I suppose I could create a separate low speed tunnel for business hours transfers but I still face the problem of detecting and dealing with a fast transfer that is "going long". Creating a second vpn tunnel that has a variable speed based on time of day is going to be a big ask.

My incremental backups are usually smallish but as they are sector based it is difficult to tell when something will create a large change.

Thanks for your input, I'll have to keep my thinking cap on. I'm open to any ideas and I'll post again if I come up with a solution.

David
pollardd
Starting out
Posts: 29
Joined: Wed Jan 03, 2018 8:43 am

Re: Nas to Nas stop long running backup

Post by pollardd »

I solved my problem and just for in case it helps someone else here is what I did.
I had my network expert create a Access Control List (ACL) on both routers so it would rate limit traffic in both directions at the specified times.
It is only applied to the specific IP address of each NAS device.
The exact syntax of the ACL is quite specific to my network so no point in posting it here.
The result was basically during business hours the transfer is limited to 25% of the entire bandwidth.
There is no longer a need to cancel running file transfers as the speed is simply limited when the specified time is reached.

Thanks to P3R for his suggestion.
David
Post Reply

Return to “Backup & Restore”