Syncing a Windows-SMB-share via App "Hybrid-Backup" extremely slow.

[F.One]

Hello,

Users can read / write via SMB and AFP to/from Qnap 1232XU with adequate speed (60-100Mbyte/s), TimeMachine is up and running fine so far.

Different servers and QNAP can copy files from/to each other via SMB with 110MByte/s.

But:
Syncing a Windows-SMB-share via App "Hybrid-Backup" is running with 100-200kByte/s (!)

So that just comparing the two directories with no changes during weekend gives:
"Sync statistic: Total number of the folders: 60203. Total number of the files: 651549. Total updated size: 0 byte. Average updating speed: N/A. Time elapsed: 5 hours 10 minutes 6 seconds"!
(from RTRR-Job0.log)

System-configuration is as follows:

Qnap: 1232XU
Software: 4.35.0760
4GB (Ressourcemonitor: "25%", 400GB usable"
CPU-load always below 50%, usually at 10-25%
4x hdd, 20TB avail. of 38TB
I/O wait always below 6%, usually at 0,5-1,5%
eth1: 1Gbit/s for administration only
eth3: 10Gbit/s for filetransfer
max & minimal CIFS/SMB version: 3


Windows Server 2016 as VM on HyperV-Host
Hardware:
IBM BladecenterH
HS23 7875 196GB RAM
IBM V7000 Storage via 8GBit/s FC
20 hdds with 10k rpm and automatic tiering to flash

Both connected to HP 5400zl with 10Gbit/s

Thank you.

PS:
From time to time additionally I get:
"ERROR: Job encountered a remote I/O error or no space left on the destination device." There is space enough and at Windows-Server side and on the Qnap there is no I/O error visible.

[F.One]

So sad, that nobody, including official Qnap-support-team will even try to answer.
I did not expect that this scenario is such a rare one...

[storageman]

Do you have "check contents" turned on? If so turn it off. Overkill!

[F.One]

It is turned OFF of course.

[storageman]

If you test with third-party app like Goodsync, Freefilesync, etc, what is the difference in times?

[F.One]

@storageman Thank you for the idea of using a 3rd party App.
But the aim was to NOT use a 3rd party tool otherwise I would prefer using rsync.

Meanwhile I pushed the plain SMB-speed up to 350-400MB/s with dedicated 10GBit/s uplink - but Hybrid-Backup is not impressed by this unfortunately. It takes 12h and much more to even "count" the differences on the source.
I do really think about defining an iSCSI-target on the QNAP for the Backup and virtualizing this via the good old IBM V7000 where the source resides. Then I could use good old an reliable real enterprise storage techniques from IBM (real-time mirroring w/ snapshots) instead of this Qnap basalt solution.

(Yes, I know, that I am not fair at this point, the Qnap was about 2k while a Vxxx from IBM is 20k€ and above...)

[storageman]

Hold on, first you show "(from RTRR-Job0.log)" now you're saying RSYNC, which is it???
Are you doing a mirror or a one way backup?
You really not very clear.

[F.One]

I did not expect that one could misunderstand but let´s clear things up:

I AM using RTRR/SMB and still trying to find a solution to fire this one up to at least normal 1GBits/s speed.

YOU did suggest to test a 3rd party app (which is a good suggestion!) - and I did state out that in this case I would prefer setting up RSYNC (which means a 3rd party app on the source server instead of "native" smb).
Thinking about other solutions the idea came to me to use the existing real enterprise storage solutions to virtualize the cheaper capacity of a Qnap there and get the benefit of have all the reliable controls from Big Blue.

In the meantime I was behind our storage cabinets. Unfortunately our Storvize V7000 has fibrechannel only so I just could use a single 1Gbit/s eth for iSCSI. Thats not to my taste, too. The others are V3700 w/o the feature "external storage virtualization".
Hm, I think I need a new V5030 with 16GBit/S fiber AND 10Gbit/s sfp+ ....

(And isn´t a "mirror" always a one way backup? )

[storageman]

You could try Qnap's Netbak Replicator.

[UndSimQ]

" I AM using RTRR/SMB and still trying to find a solution to fire this one up to at least normal 1GBits/s speed."

I would start with expectations , I am not sure yours are realistic. syncing tens of thousands of files near real time is hard enough for very powerful CPUs much less an 1.7 Ghz ARM quad you have in your QNAP. I run DoubleTake and its alternatives in various form for almost 18+ years and it takes a sizable system to do that.

where are the changes happening - on Windows server or on NAS? are the changes happening at both places (if yes, why?) ?
why not designate a single place where your changes happen and back that up. you already know that works and works well since you would be dealing with disk segments vs individual files.

[F.One]

@UndSimQ

One-Way Sync from Windows Fileserver to NAS. Daily Diff is about 100 files rarely more than some hundred megabytes.
Once per night, no realtime at all.
Expectation:
Initial copy files not with"Dual-ISDN-speed" of 128kBit/s via 1 GBit/s ethernet.

Mirroring for offsite-Backup to LTO8 and hav a nearline mountable copy of the 5TB volume.

[UndSimQ]

So if that is your requirement then why bother with sync at all, especially from a destination end running much weaker CPU? Why not a simple (robo)copy job (scheduled task) on Windows server to the remote destination?

[F.One]

CPU on Qnap is almost idle so this should not be the point.
x/robo-copy could work, too but is not that easy to monitor during everyday routine one has to read the logfile(s).
The every day task "check wether all backup jobs did well last night" is not loved by anybody so I prefer to view at a glance the state: OK or Fail. (there are a couple of jobs I have to look for...)

And funny you reminded me at DoubleTake - I almost forgot them since WindowsNT

[UndSimQ]

i think it would come down to re-defining the problem - you issue is not the infrastructure , it is the scope of the task.
your current setup is already doing synchronization of 4-5 files a second against remote source , every day (60k files 5 hour runtime reported), that is not bad.

either 'powers that be' (aka 'the business' or 'decision makers' , etc) are willing to live with it or they dont
if they don't want to wait then look at the files and 'archive' (move to read only, etc) files you know you do not want comparing, as that is nothing but wasted resources.
for the remainder of the files (current year, current quarter, whatever), leave sync on and your job performance would skyrocket.

if powers that be are unwilling to do this, then they de facto make decision to accept the performance.

comparing 60k files of which 1% may change daily is not efficient (aka stupid).


and for doubletake - I had to support environment where it did file replication down to dozens of webservers. nothing worked 100% at any point in time..

[F.One]

@UndSimQ: I guess some parts of your posting is more "opinion" than real experience.

like:
"...your current setup is already doing synchronization of 4-5 files a second against remote source , every day (60k files 5 hour runtime reported), that is not bad."

As I stated out:
1) The initial copy is done with some KILObytes per second when there is nothing to compare at all
2) It does not matter at all wether or not there are to be transferred 1Gig in thousands of small files or as a single big one
3) All resources are bored on both ends: cpu, disks (IO and throughput!), NICs

If you call a throughput of some kB/s under these circumstances "not bad" you are probably still on a PS/2 system with token ring. Otherwise it is just your opinion.

You do not know anything about the purpose, the structure and the usage of the source server - but you feel free to give advices to "archive" or even "better" call people stupid.
Sorry but I don´t like these kind of postings too much.

But just for the records:
I did set up a test job for a real small part of this server - and guess what? No "skyrocket" at all - bits still just trickle to the Qnap.


Anyway - thank you for your input.

The point that even QNAP support is unwilling / unable to answer my ticket for 2 weeks now is a sign to me to not use this solution (HybridBackup) anymore.
Maybe I will try another Synctoy or I can manage to virtualize that Qnap NAS via iSCSI behind the just ordered IBM V5030. Then I would live-migrate the source servers storage via fiberchannel to the v5030, too and can create real SAN snapshots.