Two questions regarding HBS (Hybrid Backup Sync) from home user

[gedonis]

Sorry for my questions,, but I did not find answers. I have several questions regarding HBS (Hybrid Backup Sync)

1a) What exactly means Auto-backup option in Schedule page of Backup job?
1b) What exactly means Auto-backup option in Schedule page of One-way sync job?

2) What is the difference among One-way sync and Backup?

My situation - I have QNAP NAS TS-453b and I attached large external HDD via USB. Size of HDD is a little nit more than NAS. I want to copy all files from several NAS folders to HDD periodically (once per 1-2 weeks). I do not need versioning and history of files, HDD should contain the latest (actual) version of the file. To decrease noise, I plan to switch off HDD between copying of files, so the schedule will look like one day copying - two weeks silence. What should I choose - backup or one-way sync?

[tgsbn]

Schedule "Auto-backup" means the backup job is started when the corresponding USB disk is connected.
This only works if the disk has a unique label so that HBS3 can recognize it.
Also the disk labels must not contain blanks (space characters).

The difference between one-way sync and backup is that the former is intended for creating a copy that can be used by itself while the latter is intended for being accessed via the restore function of HBS3.
In practice:
* The destination of a sync job does not appear as a possible source in HBS3 restore.
* A backup job may put the data on the destination volume in container files only HBS3 can read, instead of copies of your files which you can directly use. (In my experience it does this only if you activate the Dedup option.)

[gedonis]

Thanks! But I have three extra-questions -
1) "while the latter is intended for being accessed via the restore function of HBS3." - does it mean that backup function has archiving features (like zip, rar, etc), and size of backup is less than size of one-way sync?
2) "via the restore function of HBS3." - can I restore on another NAS?
3) " it does this only if you activate the Dedup option." - If do NOT activate the Dedup function, difference between backup and one-way sync still is not clear to me...

[P3R]

1) "while the latter is intended for being accessed via the restore function of HBS3." - does it mean that backup function has archiving features (like zip, rar, etc), and size of backup is less than size of one-way sync?

QuDedup will always do deduplication of data and it will compress if the compression option is enabled. The files are .qdff, a Qnap file format.

If you don't use QuDedup, compression or client-side encryption, the backup will be normal plain files that can be accessed normally so no extraction tools or HBS restore feature is necessary to use.

2) "via the restore function of HBS3." - can I restore on another NAS?

Yes, and from Windows, Mac or Ubuntu as backup extract tools exist for all those platforms.

3) " it does this only if you activate the Dedup option." - If do NOT activate the Dedup function, difference between backup and one-way sync still is not clear to me...

The differences are that backup is limited to other Qnaps, cloud services and to directly connected disks. Backups are placed in a subdirectory named as the backup job and it support the very useful backup versioning, so you can select to restore from different generations of backups. Versioning is very useful if a file was mistakenly deleted (or encrypted by a ransomware) before the last backup was executed. If it's a backup to a supported destination, Backup is normally the better job type to use.

Sync support more (also non-Qnap) transport protocols and the destination will hold a plain copy of the destination. If it's a synchronization of data intended to be actively used also on the destination system, Sync is usually the preferred choice. Sync jobs can also be used as backups if the destination doesn't support a Backup job but the backup features (versioning, QuDedup, compression and client-side encryption) will of course be missing.

[gedonis]

...
Sync support more (also non-Qnap) transport protocols and the destination will hold a plain copy of the destination. If it's a synchronization of data intended to be actively used also on the destination system, Sync is usually the preferred choice. Sync jobs can also be used as backups if the destination doesn't support a Backup job but the backup features (versioning, QuDedup, compression and client-side encryption) will of course be missing.

Thanks a lot, now it is clear

[oversampling]

it support the very useful backup versioning, so you can select to restore from different generations of backups

Is backup versioning a kind of differential backup? I am looking for a way to shorten the backup time for 15TB of "semi-cold" data: would versioning backup help with this? Or the time to backup 2 almost equal versions would be the same?

[P3R]

Is backup versioning a kind of differential backup?

It is in the sense that only changed data use up additional storage capacity on the backup destination. It's isn't in the sense that each version is a complete backup so you only need to restore from one backup.

I am looking for a way to shorten the backup time for 15TB of "semi-cold" data: would versioning backup help with this? Or the time to backup 2 almost equal versions would be the same?

What takes the longest time is to compare the source data to the backup data and determine if data is new or have changed so that it need to be transferred again. Versioning is really only different in the way backup data is organzied and stored so can't really speed the comparison process up.

Backup times are mostly dependent on the performance of the backup destination used, the configuration options used in HBS and the amount of data. Those are the things you can change that will affect backup times and the configuration options used are usually the things to look at assuming the backup destination throughput isn't the bottleneck.

[oversampling]

Thank you P3R.
I can't really change the amount of data to backup, this leaves me with 2 options.
For hardware replacement I was waiting for the first reviews of some QNAP Hero system...kind of tired of holding my breath though...
As for the HBS options: my understanding was that the target/source comparison was more efficient under Rsync (=quicker file copy) than RTRR but Rsync don't encompass "pulling data" backup (which I wanted). Is this so?

[P3R]

I can't really change the amount of data to backup...

No but if some parts of it are more like an archive you could perhaps separate that part and backup that less frequently? Maybe even set it to read only and backup only once?

As for the HBS options: my understanding was that the target/source comparison was more efficient under Rsync (=quicker file copy) than RTRR...

I'd say that it isn't that simple but depend on different factors.

Speed also very much depend on the options used in RTRR. Have you tried tweaking them?

...but Rsync don't encompass "pulling data" backup (which I wanted). Is this so?

Correct. Rsync only support push not pull.

[oversampling]

No but if some parts of it are more like an archive you could perhaps separate that part and backup that less frequently? Maybe even set it to read only and backup only once?

That makes very much sense: I'll run separate backup jobs for colder and warmer data.

Speed also very much depend on the options used in RTRR. Have you tried tweaking them?

Yep, my latest job (RTRR one-way push sync) towards an NTFS external raid-5 box (USB3.0 connection) took 1d 23h 39' 28" for 14.82TB. No filters, active policies: exclude duplicate files, exclude system-generated temp files, do not take a snapshot. I'll try slightly different option sets on the next backups (including EXT4/NTFS and eSATA/USB3.0 comparisons, just out of curiosity)

Thank you again P3R for you suggestions.

[P3R]

Yep, my latest job (RTRR one-way push sync) towards an NTFS external raid-5 box (USB3.0 connection) took 1d 23h 39' 28" for 14.82TB. No filters, active policies: exclude duplicate files, exclude system-generated temp files, do not take a snapshot.

Disable Exclude duplicate files and I would expect it to become much faster. Maybe you don't even need to try anything else?

I'll try slightly different option sets on the next backups (including EXT4/NTFS...

Ext4 should be faster.

...and eSATA/USB3.0

When I tested many years ago eSATA was faster but later USB 3 chips probably have improved. My money would still be on eSATA though.

[oversampling]

Another test backup to eSATA/Ext4 (instead of USB3.0/NTFS) took 1d 16h 33' 45" for the same 14.82TB. Same options as first backup --> no filters, active policies: exclude duplicate files, exclude system-generated temp files, do not take a snapshot.
That is 7h 5' 43" faster (around 15% faster).
Following P3M suggestion I am going to perform another backup unchecking "exclude duplicate file": I will post here the result.

[oversampling]

Here is the result of the new trial.
Test backup to eSATA/Ext4 (instead of USB3.0/NTFS) took 1d 19h 47' 53" for the same 14.82TB. That's about 3h 15' slower than former backup
Just one variation (in bold) with respect to former backup settings--> no filters, active policies: DO NOT exclude duplicate files, exclude system-generated temp files, do not take a snapshot.

It seems that in my case it is useful to exclude duplicate file but probably YMMV depending on your connection speed and on how many duplicate files you have..