Hybrid Backup RTRR impossibly slow

[philled]

I have recently started using HBS to backup files from a QNAP TS-410 to a QNAP TS-119. Having read the docco, HBS sounds great and just what I was looking for to schedule and store different versions of backups. I have approx 2TB of data to backup.

However, in practice the RTRR backups are so painfully slow that it's almost unusable. As an example, the initial backup of one person's folders from TS-410 to TS-119 takes 7.5 hrs to backup only 13GB of data. The average backup speed is approx 900KB/s. But when I do an RSYNC backup the speeds are much higher at approx 5.7MB/s. Both NAS devices have gigabit ethernet connections. These are the settings for the rync job:



So I could use the rsync protocol, but it's a sync, not a backup that retains versions histories. I really want to use rtrr but at these speeds it's not possible. Why is rtrr so slow compared to rsync, and what can I do to speed it up and make it usable?

[hallohome]

On the 2nd ov Nov. we made a ticket for this problem. Because is was very fast until, the "must have" update to HBS3. In the old backub station rtrr was extremly fast.
But Qnap never realy helped us until today. Only hints like "firmware update" "reset and restore" but it doesn't help. Now it took them almost 20 days to respond again in our ticket.
The have send it to the development in Taiwan. Lets hope they find out something.
I recommend you, open a ticket, if more people send in this problem it gets hopefully solved.
https://service.qnap.com/

[philled]

On the 2nd ov Nov. we made a ticket for this problem. Because is was very fast until, the "must have" update to HBS3. In the old backub station rtrr was extremly fast.
But Qnap never realy helped us until today. Only hints like "firmware update" "reset and restore" but it doesn't help. Now it took them almost 20 days to respond again in our ticket.
The have send it to the development in Taiwan. Lets hope they find out something.
I recommend you, open a ticket, if more people send in this problem it gets hopefully solved.
https://service.qnap.com/

Wow, that's really poor. I've also seen other similar posts that didn't get resolved. And over 200 people have read this with no solution offered so I guess everyone has the same problem that QNAP have implemented a backup solutio that's unusable. They ought to be onto that quick smart as backups are crucial to any storage solution.

I'll raise a ticket and see what they say. Probably nothing by the looks of things!

Meantime I've stopped using their unusable HFS app for backups and am using Rsnapshot instead. This achieves the same end result of incremental backups, but uses rsync so the performance is respectable.

[zmsh99]

I have had similar problems backing up from a TS-541+ to a TS410 using HBS 3.

I had 750GB in 1.5M files. The first 50% went reasonably quickly. Then it slowed to a crawl. Stopping the job and restarting would get a 100,000 done fairly fast, before slowing to a crawl. For the last 100,000 had to restart the job every few thousand in order to get a reasonable speed.

At a guess, I would say a log on the server is getting full, and somehow it is not purging old entries fast enough to allow the backup to continue at a sensible speed.

One thing I noted was the files it seemed to hang up on were very small byte sized files. The large ones didn't seem to be a problem.

Hopefully QNAP technical support/development can reproduce this and develop a fix.

[dolbyman]

if it's the destination RRTR, then don't bank on QNAP, no more feature or security updates for x19 devices starting today

[hweegink]

Same issue here. Backup from a TVS-471 to a TVS-472XT. Initial backup speed was at least 50 Mb/sec, processing 1.5Tb in a couple of hours.

But then it slowed down to a crawl (< 4 Kb/s). The last 10% of the files will take a full day this way.

I am 100% sure it is not the CPU load (<10%), memory usage (<30%) on either NAS and it is definitely
not the network congestion (using Gbit ethernet, dedicated).

Also, when I start another backup job (in parallel), the total speed is back to >50 Mb/sec instantly.

So it must be a bug in HBS. Hopefully the QNAP guys will fix it soon.

[hweegink]

Did a quick test. Split the same backup job in 3 parts, ran then one by one and they were all processed at a speed of approx. 110MB/s.

So it looks like the issue may have something to do with the large number of (small) files.

PS: in my previous post I mentioned Kb/s, Mb/s, Tb. That should have been KB/s, MB/s, TB. Oops.

[glennmac]

I have also found that a large number of small files creates problems. Unfortunately "large" is not all that large. I find that if you I the backup and run it again, I can get a bit further and eventually get it done. Something is definitely amiss.

[albumman]

On the 2nd ov Nov. we made a ticket for this problem. Because is was very fast until, the "must have" update to HBS3. In the old backub station rtrr was extremly fast.

Not sure if this works for newer models, but if HBS is removed the built-in Backup Station returns. I have no need for cloud based backups or other features of HBS on my old, but still running TS-212P.

[alfredborges]

I had this exact same problem - I think a suggestion either on this thread or similar on this board fixed this for me

I increased the size of the log file to the maximum 1024 mb

instead of the estimated 12 days, took less than 8 hours to complete a full backup of approx 4.5 TB to an attached USB drive

not a "guru" in this area, but I wonder if a backup with many small files causes log file bloat



hopefully, QNAP will address this

[alokeprasad]

Progress
I increased the log file to 1024 MB.

Now, the BU proceeds rapidly to 5% and then slows down to a crawl. The initial time to finish is 22 hrs (which is reasonable), and then grows steadily to many days.
At least HBS was able to calculate the job size

I'll try splitting the job into 2-3 parts..

[cacloss]

I am having the same issue. I had a 4TB USB drive that seemed to backup incremental files quickly. When I replaced the smaller external drive with a new larger one I started having this problem. I am backing up 3TB. At about 31% is slows to a crawl. Looking at the details it's backing up small image files when it starts to choke.

Is QNAP looking into this?

[LMSMFT]

[
Not sure if this works for newer models, but if HBS is removed the built-in Backup Station returns. I have no need for cloud based backups or other features of HBS on my old, but still running TS-212P.

Hmm - I tried this and Backup Station did not show up, nor do I see it in the app center to install. I did a reboot. Any other steps you did to get it?

[albumman]

[Not sure if this works for newer models, but if HBS is removed the built-in Backup Station returns. I have no need for cloud based backups or other features of HBS on my old, but still running TS-212P.

Hmm - I tried this and Backup Station did not show up, nor do I see it in the app center to install. I did a reboot. Any other steps you did to get it?

It probably depends on the firmware for your model. I remember trying HBS when it first came out some 4 or so years ago and was buggy then. All I did was uninstall it and magically Backup Station was there.

[Ron1963]

As of firmware 4.5.1 good old Backup Station is removed, as mentioned in the release notes. Unfortunately.