Hybrid Backup RTRR impossibly slow

Backup, Restore, Netbak Replicator, Cloud Storage Services
pekako
New here
Posts: 4
Joined: Sat Mar 06, 2021 2:40 am

Re: Hybrid Backup RTRR impossibly slow

Post by pekako »

I have the same problem.
The last backup ( 2 months ago ) was normal - 90 MB/sec on 1Gb ethernet between 2 qnap ( TS-251+ and backup to TS-231-P2 with QTS 4.5.1.1540 build 20210107 on both machines)
Last month i installed the update QTS 4.5.2.1566 Build 20210202 - i don't look to performance.
Now, 5 days ago i started a new backup and i see only 15-17MB/sec - very slow.
Today i updated both machines to the lates level 4.5.2.1594- the same problem - backup and restore only with 15-17MB/sec.
With putty and rsync between the machines i see up to 100MB/sec.
My envireonment is unchanged since one year.
wk170
Starting out
Posts: 21
Joined: Thu May 02, 2019 2:51 am

Re: Hybrid Backup RTRR impossibly slow

Post by wk170 »

Just installed the new HBS 3 version (15.1.0225) which was released yesterday.

Just ran my one-way sync job, unfortunately it is still very very slow, and unusable.
Looks like the issue still has not been fixed.
TS-453A + TR-004
TS-659 Pro II
pekako
New here
Posts: 4
Joined: Sat Mar 06, 2021 2:40 am

Re: Hybrid Backup RTRR impossibly slow

Post by pekako »

my problem, reported 4 days ago, is resolved.
now i see 90 ...100Mbyte/sec.
Resolution: on source machine ,i disabled "use ssl connection" in HBS3 --> storage places --> remote NAS --> edit job profile.
After the change, speed test shows 100MByte/sec and the running job showes also 95... 105MByte/sec.
I'm happy.
tobi_bw
New here
Posts: 4
Joined: Fri Oct 06, 2017 2:47 am

Re: Hybrid Backup RTRR impossibly slow

Post by tobi_bw »

I want to join in - I have this problem while syncing with HBS RTRR from my local QNAP to a USB3.1 connected external HDD on the sampe QNAP. So it is not some obscure network issue, it is literally the inability of a backup program to copy files at a decent speed. Indeed, stopping the job, changing the log file to maximum 1024 MB and restarting helps. What a shame.
autpgblds
New here
Posts: 3
Joined: Tue Feb 09, 2021 3:39 am

Re: Hybrid Backup RTRR impossibly slow

Post by autpgblds »

I just don't get it. Simply spawning a shell and rsync takes like 7 minutes to copy 1.2gb.
HBS 3 doing a "sync" job takes ... well who knows, I killed it after 46min, at 1.2MB/s

Why TF cant they just write a gui for building cron files for rsync? Then parse logs and populate into the gui's alert system.
There are literally 1000's of these posts on the web about how slow this pile of garbage app is.

At the end of the day, the app should take no longer than a simply rsync does. But its on accord of 100x slower.
Amoebia
Easy as a breeze
Posts: 253
Joined: Sat Jul 04, 2009 9:50 pm
Location: Georgia

Re: Hybrid Backup RTRR impossibly slow

Post by Amoebia »

Same here... :roll:
--------------------------------------------------------------------------------
4 * Seagate 3TB - Raid-10
--------------------------------------------------------------------------------
User avatar
Moogle Stiltzkin
Guru
Posts: 11448
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: Hybrid Backup RTRR impossibly slow

Post by Moogle Stiltzkin »

latest qts, latest hbs, ts-877

can someone explain what is going on with this >_>:

Image


my settings are as follows

- one way rtrr sync from a ts-877 to a tl-d400s connected via special cable directly to each other
- ssl enabled
- check file contents enabled
- remove additional files at destination location enabled
- do not take a snapshot enabled


yes i expect with some of these settings it may be slow, but not to this extent. at least my observation of the file transfer speed seems rather slow.

I checked the internal speed is showing 150-300 MB/s read when doing the check file content if not mistaken so that at least doesn't seem to be the bottleneck. Something else is going on but i can't explain these results o-O;


*update

ran using the same settings but for ts-877 to a ts-653a, all updated to latest, but it seems fine :' hm
Image


so basically in my example, the transfer to a ts-653a was way faster than a tl-d400s. i suspect there is a bug or something, cauz i can't explain the results :( I'll check the cable or something.
NAS
[Main Server] QNAP TS-877 (QTS) w. 4tb [ 3x HGST Deskstar NAS & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A (Truenas Core) w. 4x 2TB Samsung F3 (HD203WI) RaidZ1 ZFS + 8gb ddr3 Crucial
[^] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) 2x 4TB Seagate Ironwolf, Raid5
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-253D (Truenas Scale)
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Qotom Pfsense|100mbps FTTH | Win11, Ryzen 5600X Desktop (1x2tb Crucial P50 Plus M.2 SSD, 1x 8tb seagate Ironwolf,1x 4tb HGST Ultrastar 7K4000)


Resources
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin
Marc J
Getting the hang of things
Posts: 62
Joined: Wed Oct 15, 2008 1:36 am

Re: Hybrid Backup RTRR impossibly slow

Post by Marc J »

Moogle Stiltzkin wrote: Sun Mar 14, 2021 7:27 pmi suspect there is a bug or something, cauz i can't explain the results :( I'll check the cable or something.
Don't waste your time. I had two identical (new) external HDDs (USB3), tested with a 1Tb test file copied through File Station from NAS RAID to External HDD, swapping external disks, cables, and between front and rear USB3 ports on my TS-453D, and all gave the same results (approx 113MB/s). But when attempting any backup or sync job via HBS 3 it quickly slows to a crawl :(
User avatar
Moogle Stiltzkin
Guru
Posts: 11448
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: Hybrid Backup RTRR impossibly slow

Post by Moogle Stiltzkin »

Marc J wrote: Tue Mar 30, 2021 7:33 am
Moogle Stiltzkin wrote: Sun Mar 14, 2021 7:27 pmi suspect there is a bug or something, cauz i can't explain the results :( I'll check the cable or something.
Don't waste your time. I had two identical (new) external HDDs (USB3), tested with a 1Tb test file copied through File Station from NAS RAID to External HDD, swapping external disks, cables, and between front and rear USB3 ports on my TS-453D, and all gave the same results (approx 113MB/s). But when attempting any backup or sync job via HBS 3 it quickly slows to a crawl :(
not sure if i mentioned this, but i ran hbs in a few different scenarios

1. ts-877 to a tl-d400s
2. ts-877 to a ts-653a
3. ts-877 to a ts-509 pro


for 2 and 3, it seems to show the transfer speed as i normally would expect. but for 1, the transfer speeds looked awfully slow. i believe last time it worked just fine, but i don't know what changed either in hbs or qts, but it feels slower for some reason and hbs result looks slow at 1.5 MB/s transmission speed :'

i'll check again in a couple of months when i run my next backup to test again. i'll try reset the cables then.
NAS
[Main Server] QNAP TS-877 (QTS) w. 4tb [ 3x HGST Deskstar NAS & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A (Truenas Core) w. 4x 2TB Samsung F3 (HD203WI) RaidZ1 ZFS + 8gb ddr3 Crucial
[^] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) 2x 4TB Seagate Ironwolf, Raid5
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-253D (Truenas Scale)
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Qotom Pfsense|100mbps FTTH | Win11, Ryzen 5600X Desktop (1x2tb Crucial P50 Plus M.2 SSD, 1x 8tb seagate Ironwolf,1x 4tb HGST Ultrastar 7K4000)


Resources
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin
jlrx81
New here
Posts: 4
Joined: Thu Oct 15, 2015 11:12 pm

Re: Hybrid Backup RTRR impossibly slow

Post by jlrx81 »

Hello, we've been encountering a similar problem over the past two weeks.

We have a TS-EC880U,

Previously we've backed up 1.8TB to another volume on the device (before it is moved to external) and it takes 4 1/2 hours.

However over the last two weeks it is so slow as to never complete. It stuck at 23% with 8 days to go after several days, and now after getting to about 15% it drops to less than 10kbps transger rate.

Again, this is a volume on a local disk. I can copy files from it easily. It takes about 2 1/2 hours to copy a backup from the volume to an external drive. It should not be as slow as it's being now as to be completely unusable.
P3R
Guru
Posts: 13190
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Hybrid Backup RTRR impossibly slow

Post by P3R »

RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
User avatar
spikemixture
Been there, done that
Posts: 890
Joined: Wed Mar 07, 2018 11:04 pm
Location: 3rd World

Re: Hybrid Backup RTRR impossibly slow

Post by spikemixture »

To add to this discussion
I have 2 main back ups
1. Media ~30TB and mostly mp4
2. Data ~4tb - lots of small files.

I have tried SMB, FTP and RTRR from Qnap to Qnap AND Qnap to Synology.

All get the same results - Media (30TB)takes a few hours depending on changes.
Data (4TB) - extremely slow and takes about 15 hours using SMB , FTP and RTRR

The Data (4tb) job does not finish completely but appears near the end and gives an error message they tells me nothing.

To my non technical knowledge it appears to be trying to copy the whole 4TB everytime.(And not doing a very good job)
Qnap TS-1277 1700 (48gb RAM) 8x10TB WD White,- Raid5, 2x M.2 Crucial 1TB (Raid 1 VM),
2x SSD 860 EVO 500gb (Raid1 QTS), 2x SSD 860 EVO 250GB (Cache), 2x M.2 PCIe 970 500gb NVME (Raid1 Plex and Emby server)
GTX 1050 TI
Qnap TVS-1282 i7 (32GB RAM) 6x8TB WD White - JBOD, 2x M.2 Crucial 500gb (Raid1 VM),
2x SSD EVO 500gb (Raid1 QTS), 2x SSD EVO 250gb (Raid1 Cache), 2x M.2 PCIe Intel 512GB NVME (Raid1-Servers)
Synology -1817+ - DOA
Drobo 5n - 5x4TB Seagate, - Drobo Raid = 15TB
ProBox 8 Bay USB3 - 49TB mixed drives - JBOD
All software is updated asap.
I give my opinion from my experience i.e. I have (or had) that piece of equipment/software and used it! :roll:
P3R
Guru
Posts: 13190
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Hybrid Backup RTRR impossibly slow

Post by P3R »

What do you think about the HBS version 16.0 out now that is said to resolve the issue?
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
pekako
New here
Posts: 4
Joined: Sat Mar 06, 2021 2:40 am

Re: Hybrid Backup RTRR impossibly slow

Post by pekako »

Moogle Stiltzkin wrote: Sun Mar 14, 2021 7:27 pm latest qts, latest hbs, ts-877

can someone explain what is going on with this >_>:

Image


my settings are as follows

- one way rtrr sync from a ts-877 to a tl-d400s connected via special cable directly to each other
- ssl enabled
- check file contents enabled
- remove additional files at destination location enabled
- do not take a snapshot enabled


yes i expect with some of these settings it may be slow, but not to this extent. at least my observation of the file transfer speed seems rather slow.

I checked the internal speed is showing 150-300 MB/s read when doing the check file content if not mistaken so that at least doesn't seem to be the bottleneck. Something else is going on but i can't explain these results o-O;


*update

ran using the same settings but for ts-877 to a ts-653a, all updated to latest, but it seems fine :' hm
Image


so basically in my example, the transfer to a ts-653a was way faster than a tl-d400s. i suspect there is a bug or something, cauz i can't explain the results :( I'll check the cable or something.
please try to disable ssl.
this helped me.
after the change i have 90MegaByte/sec
P3R
Guru
Posts: 13190
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Hybrid Backup RTRR impossibly slow

Post by P3R »

Moogle Stiltzkin wrote: Sun Mar 14, 2021 7:27 pm - one way rtrr sync from a ts-877 to a tl-d400s connected via special cable directly to each other
- ssl enabled
Did you set it up as local or remote storage? Local would definitely be preferred and doing that I don't even understand how SSL could be enabled.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
Locked

Return to “Backup & Restore”