Is Real Time Synchronization a good backup?

[CQDan]

Need some help here. I'm trying to decide which is the best scenario I should use to ensure my data is safe. I am also trying to cut cost.

I have a 6-bay TS-653b.

- DataVol1 consists of 3 x 8TB RAID5 or 4 x 8TB RAID10

Q1: Is Real Time Synchronization backing up DataVol1 to a single 16TB non-RAID volume (1 drive) within the NAS a good idea?

Q2: Would RTS stress the system much?

Q3: Which is better: RTS to a separate 16TB volume within the NAS or RTS to an external 16TB HDD via USB?


Thanks in advance.

[Mousetick]

Q1: Is Real Time Synchronization backing up DataVol1 to a single 16TB non-RAID volume (1 drive) within the NAS a good idea?

No. A good backup must be physically separate from the NAS.

Q2: Would RTS stress the system much?

N/A

Q3: Which is better: RTS to a separate 16TB volume within the NAS or RTS to an external 16TB HDD via USB?

Real Time Syncronization is not a good backup by itself no matter where the copy is stored. What if you accidentally delete an entire folder of precious files on the NAS, or some files get encrypted by ransomware? Thanks to RTS, the same folder is now deleted, and the same files are encrypted on the RTS mirror. How does that make your data safe?

RTS could be one component of a backup plan, but it can't be the whole plan. Furthermore since you're already using fault-tolerant RAID on the NAS, I see little benefit in using real-time synchronization.

What you need at the very least is a point-in-time (e.g. taken daily or weekly) full copy of your data stored on an external device that can easily be taken offline and transported. That should be the bare minimum basis for your backup plan.

To ensure the safety of your data, you should strongly consider making a second full copy, so that you end up with 3 copies of your data: 1) the original on the NAS, 2) the copy on an external device 3) another full copy on another external, off-site device (which can be in the cloud).

[CQDan]

Thanks for the quick response.


Those are good points. Forgot to take ransonware/viruses into consideration. Thanks

Currently my data has backup redundancy: 1) RAID5 on NAS, 2) external USB HDD and 3) Backblaze.

Eventually, I wanna cut BackBlaze as it cost $$$ That leaves #1 and #2 only, which are in the same house. I understand if the house burns down, I could lose everything (I am betting the odds of that happening to be NEVER ). Let's take off-site backup out of the picture for now.

The data on the external USB HDD is not up-to-date, hence the reason for RTS.

All those talk about 2-drive failure in a RAID1/5/10 had me worried. Although I am very skeptical the probability of 2 drives failing at the same time to be highly unlikely but not impossible and I am giving it the benefit of the doubt. RAID6 is out of the question. I had RAID6 running on my original 6 x 3TB array and it was slow as hell. Data transfer can dip to less than 100MB/s in an uncongested gigabit network. A few times I've seen it transfer at around 60 MB/s.

[Mousetick]

The chances of 2 or more drives failing simultaneously in a fault-tolerant RAID are quite slim. The probability increases significantly in large arrays containing many identical large capacity drives from the same production batch. These large arrays take a very long time to rebuild, and identical drives from the same production batch are more prone to fail within a given time window, so it is more likely that another drive may fail before the array rebuild is completed. But in your case unless you're extremely unlucky (for example, your NAS is striken by a power spike or lightning and all drives fry), only 1 drive will fail and you should have ample time to make a new point-in-time backup to copy all the modified data since your previous point-in-time backup.

Even if you have 2 drives failing simultaneously or in close succession and the RAID is toast, if we assume you backup your NAS daily, you would at most lose one day of changes. That wouldn't be a disaster, would it?

I do use real-time synchronization software myself, locally (ResilioSync, one copy on each computer + NAS on the LAN) for whatever project I'm currently working on, and also in the cloud for a very small critical dataset or stuff that needs to be shared with 3rd-parties (Dropbox). But that's for convenience and extra safety of no more than 1% of my total data, it's in complement of a standard comprehensive backup, it's not "the" backup. For my main backup I require multiple versions (last 30 days, 30 weeks, 6 months, 1 year historical retention), which synchronization doesn't provide.

But let's go back to the off-site backup for a minute Backup is like insurance. How much data can you afford to lose, and how much data is vital to keep, if your house burns down or is burglarized? You need to invest enough in off-site backup to cover what you can't afford to lose, like insurance. Maybe you don't need to backup your entire music or movie library to the cloud. On the other hand it would probably be a good idea to back up your personal essential documents to the cloud, which should be free or quite cheap because they're small, or to an external device located off-site.

[CQDan]

Thanks for the useful info, had me rethink what I failed to take into consideration previously.

The reason why I didn't consider off-site backup necessary is because the Asian city where I live in, home buglary is extremely rare. Leaving my front door wide open and neighbors wouldn't even stick their heads in, let alone come in (tried that a few times ). But now that you mentioned "burglarized", I just remembered I will be returning to America end of the year. So, yes, home invasion and burglary is back in the menu. LOL. Where I used to live in east-end Toronto, my burglar alarm was triggered once every few weeks. Frigging annoying! And not to mention power surges, the same east-end Toronto house just less than 20 mins away from the Pickering Nuclear Power Plant, very unstable power lines and often time no electricity for as long as 30 mins. Getting your hardware fried is not impossible.

Anyway, looks like off-site backup is here to stay. My current NAS backup to Backblaze B2 only covers 1TB of data. I cannot afford to backup the whole 16TB RAID volume although that would be ideal. To pull that off, I need to find a way to perform an automated backup/sync to the external HDD and then automatically backup that HDD to a Backblaze Personal Backup plan that allows unlimited data for a more affordable $60/year. HBS3 can't seem to be able to backup to a network mount/drive. Know of any other backup/sync app that can pull that off? Or another workaround?

[Mousetick]

I'm not sure I quite understand your idea for using Backblaze Personal Backup. Can you elaborate on how you intend it to work?

I don't use any QNAP software for backup, so can't speak about HBS... but if I'm not mistaken it allows for backing up to/from SMB network shares. You may want to search online for more information or ask your questions about that in a new topic on this forum. There are more knowledgeable users than I here who will likely answer.

I'll just offer some counter-arguments for now
- Backing up a backup is generally not a very good idea. If the source backup becomes bad for whatever reason, you end up with 2 bad backups, which is no better than no backup at all. Furthermore it can make restoration of the secondary backup more complicated and trouble-prone, depending on how the primary and secondary backups are stored. In your case it seems your primary backup will be a simple mirror sync of your files and folders, so restoration should only involve the secondary backup software.
- Your backup plan should be as automated, seamless and friction-less as possible. If the procedure is too complicated, requires manual steps, not only it becomes more error-prone, but you may also tire of it over time and stop following it.

[CQDan]

Backblaze Personal Backup only cost $60/year with unlimited data. However, that only applies to Personal Computers. It does not work for QNAP NAS or any other NASes. QNAP NAS needs to pay for Backblaze B2, which cost $0.005/GB or $5/TB. Back to Backblaze Personal, you can attach a USB HDD onto the PC and the Backblaze client can backup all data to your Backblaze Personal account.

For more info, check out this thread:
viewtopic.php?t=153544

My idea was to use a PC running 24/7, connect my 16TB USB HDD to it and share the USB HDD out. Have a QNAP backup app to backup to the USB network share drive. With the Backblaze client installed on the PC, the BB client will automatically backup whatever is on the USB HDD. Everything can be done automated....providing that you can find a QNAP backup app that that can backup to a network share? If no such app, then I have to use the hard way...manual labor: 1) QNAP backup to an external USB HDD, 2) then connect the USB HDD to the PC and let the BB client automatically backup whatever's on the USB HDD regardless how big it is. Manual steps sound simple but I am sure you know how long QNAP backup takes?? :O HBS3 Sync my current 8 TB data already takes over 12 hours long. Imagine 16TB!! HA!

As for your counter-argument HBS3 has versioning. You can set it to NOT overwrite previous backup but that would take up more space. Even Backblaze offer versioning as well:
https://help.backblaze.com/hc/en-us/art ... istory-FAQ
https://www.backblaze.com/b2/docs/file_versions.html

No worries there even if you backed up a Backup that had an accidentally deleted folder. And since BB Personal is UNLIMITED DATA, I wouldn't worry about versioning taking up too much disk space.

PS. No, HBS does not support backup to network shares. It's either Local NAS (including USB HDDs), Remote NAS (RTRR Server) or Clouds like Backblaze and Google Drive

[Mousetick]

* Are you absolutely sure that HBS doesn't support sync to/from Windows network shares? On this page https://www.qnap.com/en/software/hybrid-backup-sync, I'm reading that it supports "One-way sync" and "Active sync" (whatever that means) to/from CIFS/SMB server. But again I'm not a user nor an expert, so can't vouch for it.

* Are you sure that Backblaze will let you backup 16 TB from an external drive and nothing or very little from the PC it is running on? I'd bet a lot of people try to work around the NAS or server restriction and abuse the Personal Backup. I wouldn't be surprised if Backblaze had checks in place to limit what you can do. But if not, hey, all the better and good for you. Go for it

* On the matter of backing up a backup. It's fine to backup a mirror sync of individual files and folders, as it's essentially the same as backing up from the original disk. However if your primary backup is multi-versioned, or compressed, or encrypted or otherwise stored in a proprietary format that can only be restored by the primary backup software, it's a bad idea to make a secondary backup of that primary backup. Too many things can go wrong, including potential bugs in either primary or secondary backup software. The point of making multiple backups to different destinations is to diminish the risks of failure by spreading them and to increase the chances of a quick recovery, if you make a backup of a backup, you're conversely compounding the risks and decreasing the chances.

* Since you'll be making the PC running Backblaze Personal Backup available 24/7, why not make it the backup server then? Forget about HBS, and backup (or sync) the NAS shared folders from the PC to the external USB drive via SMB. So you'll have all your backup software and settings in one place. There are plenty of backup/sync tools available for Windows. If you're going to do a simple mirror sync and aren't afraid of tinkering a bit, I could recommend FastCopy, which is extremely fast, very reliable, and is free. It's not the most user-friendly tool (you can't have everything for free), but you can create jobs from the UI and then run those jobs from the command line in Windows Task Scheduler.

[CQDan]

* Are you absolutely sure that HBS doesn't support sync to/from Windows network shares? On this page https://www.qnap.com/en/software/hybrid-backup-sync, I'm reading that it supports "One-way sync" and "Active sync" (whatever that means) to/from CIFS/SMB server. But again I'm not a user nor an expert, so can't vouch for it.

That's what it claims but no can do, no such option. This thread also confirms it: viewtopic.php?t=154948

Maybe it used to support SMB but not anymore.

* Are you sure that Backblaze will let you backup 16 TB from an external drive and nothing or very little from the PC it is running on? I'd bet a lot of people try to work around the NAS or server restriction and abuse the Personal Backup. I wouldn't be surprised if Backblaze had checks in place to limit what you can do. But if not, hey, all the better and good for you. Go for it

Well, I am not sure. I haven't tried it yet. I did read somewhere (I think on reddit) that someone regularly backed up double-digit TB of media files. No problems he claims.

* Since you'll be making the PC running Backblaze Personal Backup available 24/7, why not make it the backup server then? Forget about HBS, and backup (or sync) the NAS shared folders from the PC to the external USB drive via SMB. So you'll have all your backup software and settings in one place. There are plenty of backup/sync tools available for Windows. If you're going to do a simple mirror sync and aren't afraid of tinkering a bit, I could recommend FastCopy, which is extremely fast, very reliable, and is free. It's not the most user-friendly tool (you can't have everything for free), but you can create jobs from the UI and then run those jobs from the command line in Windows Task Scheduler.

Thanks. I'll check that out.

[LMSMFT]

- Your backup plan should be as automated, seamless and friction-less as possible. If the procedure is too complicated, requires manual steps, not only it becomes more error-prone, but you may also tire of it over time and stop following it.

Would you mind sharing what you use for you backup solution? Like others, I am surprisedly and disappointingly finding that HBS simply does not work. I was using backup station to an external USB drive and swapping two drives out physically to my workplace once a month and was quite happy with that. Looking for a good solution. Thanks.

[Mousetick]

Would you mind sharing what you use for you backup solution? Like others, I am surprisedly and disappointingly finding that HBS simply does not work. I was using backup station to an external USB drive and swapping two drives out physically to my workplace once a month and was quite happy with that. Looking for a good solution. Thanks.

I don't mind at all. For backing up my NAS at home I use Duplicacy.

This is an open-source product that is sold with various licenses depending on usage: personal, or professional/commercial. There is also a free license to use the command-line version (without the GUI), which is what I use because I'm a cheapskate and for various other reasons not worth describing here.

It runs on Windows, macOS and Linux. There is QPKG for the GUI version available in QNAP's App Center: Duplicacy.

There is a community support forum https://forum.duplicacy.com/ where the developer(s) participates actively.

You can find more information at the links I provided, but the main features in a nutshell:
- Versioned incremental backups
- Inclusion/exclusion filters
- Encryption
- Compression
- Deduplication
- Backup from/to local or network drives
- Backup to cloud storage (most major backends are supported)
- Scheduling
- Configurable retention policy of old backups

I use it for backing up nightly the NAS to an external USB drive and a data subset to Backblaze B2 cloud storage.

Please note: I have zero experience using the GUI version, so can't speak about it at all, either good or bad. All I can say is that it looks nice and easy to use. Without a license you can try the GUI version for 30 days.

Would I recommend it? Yes, with a few caveats:
- As mentioned, I have no experience with the GUI.
- This is still a pretty "young" product (first released about 4 years ago) with a few rough edges here and there, although it's become quite stable in the last year or so.
- This is pretty much a one-man show, with a few contributors.
- The terminology used in the interfaces and documentation can be confusing coming from other storage and backup products.
- It's designed and optimized for versioned incremental backups. Not a good fit and overkill for 'sync' or 'mirror' types of backup.
On the plus side:
- The set of features provided is hard to find elsewhere at that price.
- The developer is very active and responsive.
- It is cross-platform and any backup can be restored anywhere (i.e. you don't need a working NAS to recover your files in case of a disaster for example). Restoration doesn't require a license.
- It's open-source.

You can always give it a try and see if it fits your needs. This user on the Duplicacy community support forum appears to be using the GUI version on a QNAP NAS. Perhaps you could get in touch with him to get a second opinion.

Disclaimer: I have no affiliation with the product or the developer. I'm just a user lambda.

[LMSMFT]

Thanks - it's is conveniently downloadable right from the QNAP AppCenter. I'll give it a try.

[P3R]

* Are you absolutely sure that HBS doesn't support sync to/from Windows network shares? On this page https://www.qnap.com/en/software/hybrid-backup-sync, I'm reading that it supports "One-way sync" and "Active sync" (whatever that means) to/from CIFS/SMB server.

HBS support sync over SMB the same as it always have. One-way sync is a simple push sync to the destination. The name Active sync however is a mystery, I never figured out what it means but if I say that it's a pull sync it should hopefully be clearer. So Active sync simply sync data in the opposite direction from a One-way sync.

[P3R]

That's what it claims but no can do, no such option. This thread also confirms it: viewtopic.php?t=154948

Remember that HBS stand for Hybrid Backup Sync so the same application support both syncs and backups with different features and different protocol support for each type.

My post in that thread only confirm that HBS doesn't support backups over SMB. The OP mention that sync work.

HBS only support backups over the Qnap proprietary RTRR protocol so versioning is limited to Qnap backup destination servers. Versioned backups are also supported to internal storage (which due to being internal is almost useless as a backup), externally attached storage and to (at least some) cloud services.

[LMSMFT]

You can always give it a try and see if it fits your needs. This user on the Duplicacy community support forum appears to be using the GUI version on a QNAP NAS. Perhaps you could get in touch with him to get a second opinion.

So I installed it, and man the GUI is not intuitive at all.

I ended up writing a shell script to use rsync, and I am just about there with it, doing some final tweaking of some non-critical behavior.

@Mousetick, if you don't mind, why did you decide to use the duplicacy CLI vs just going with the proven rsync? Does it do something better than rsync?