Very slow uploads to B2 using Hybrid Backup Sync
-
- Starting out
- Posts: 38
- Joined: Mon Aug 14, 2017 11:59 pm
Very slow uploads to B2 using Hybrid Backup Sync
I set up a job to do a one-way sync of my Photos folder on the QNAP to the Backblaze B2 bucket. It started off doing about 800Kbps-1.2Mpbs, but after a while slowed down to just a few Kbps. This is the initial sync, so the bucket didn't have any files. Transfers to B2 from my laptop are on the order of 1.2-1.5 Mbps. Doing an rsync of a large file from QNAP to a remote server is about the same. Any ideas what may be causing the very slow upload speeds in HBS?
-Andrei
-Andrei
-
- Know my way around
- Posts: 118
- Joined: Thu May 03, 2012 2:19 pm
Re: Very slow uploads to B2 using Hybrid Backup Sync
QNAP have confirmed there is a bug in HBS and to wait for a fix to come through.
John
John
TS-420 3 x 3TB Seagate SD3000DM001, 1 x 3TB WD Red RAID 5
- twiga
- Starting out
- Posts: 36
- Joined: Tue Dec 29, 2015 2:09 pm
Re: Very slow uploads to B2 using Hybrid Backup Sync
I am having the same problem. Any ETA on fix for HBS?
Model: TS-453 Pro + TS-653 Pro
-
- Know my way around
- Posts: 118
- Joined: Thu May 03, 2012 2:19 pm
Re: Very slow uploads to B2 using Hybrid Backup Sync
I've been told to wait and it will come through, so no idea.
TS-420 3 x 3TB Seagate SD3000DM001, 1 x 3TB WD Red RAID 5
-
- Getting the hang of things
- Posts: 93
- Joined: Mon Feb 12, 2018 9:26 pm
Re: Very slow uploads to B2 using Hybrid Backup Sync
So, the latest update didn't fix this problem, right? It's still totally slow here.
TVS-1282T3
CPU: Intel Core i7-7700 CPU @ 3.60GHz
Memory: 64 GB
2 x Samsung SSD 850 EVO M.2 1TB (M.2 SATA)
2 x Samsung SSD 860 EVO 2TB (SATA)
4 x WDC WD6002FFWX-68TZ4N0 (SATA) (6TB)
4 x Seagate ST12000VN0007-2GS116 (SATA) (12TB)
CPU: Intel Core i7-7700 CPU @ 3.60GHz
Memory: 64 GB
2 x Samsung SSD 850 EVO M.2 1TB (M.2 SATA)
2 x Samsung SSD 860 EVO 2TB (SATA)
4 x WDC WD6002FFWX-68TZ4N0 (SATA) (6TB)
4 x Seagate ST12000VN0007-2GS116 (SATA) (12TB)
-
- New here
- Posts: 5
- Joined: Thu Oct 11, 2018 10:51 pm
Re: Very slow uploads to B2 using Hybrid Backup Sync
Ugh I'm dealing with a QNAP tech guy right now with this. I have Broadband speeds of 800mbs and Backblaze is clocking me at over 300mbs upload to their site yet I have a measly 10mbs coming from the NAS TVS-463. I bought a fancy new Blue Wave Router, all new CAT 6 cabling added RAM in the NAS, RAID 5 with 4TB WD REDs and still slow as hell.
If I would have known QNAP was this slow I would have bought something else. They false advertised the whole Turbo NAS deal.
If I would have known QNAP was this slow I would have bought something else. They false advertised the whole Turbo NAS deal.
-
- Getting the hang of things
- Posts: 93
- Joined: Mon Feb 12, 2018 9:26 pm
Re: Very slow uploads to B2 using Hybrid Backup Sync
I'm now using qBackup, which is very fast, does an incremental backup and uses some hashed-block names to store things. So, no files in your bucket. Take a look, very good. The only drawback is, that the use a stored index. If something happens to it, before having a first complete backup you have to start over.
Or this here: https://github.com/gilbertchen/duplicacy currently some issues with B2 but I saw a ticket fixing it. Not checked yet, but if it works I would even prefer it over qBackup as it's not proprietary nor uses a central index.
The QNAP Hybrid Backup thing is OK but not really professional. So, it's way to slow and no intelligent incremental backups.
Or this here: https://github.com/gilbertchen/duplicacy currently some issues with B2 but I saw a ticket fixing it. Not checked yet, but if it works I would even prefer it over qBackup as it's not proprietary nor uses a central index.
The QNAP Hybrid Backup thing is OK but not really professional. So, it's way to slow and no intelligent incremental backups.
TVS-1282T3
CPU: Intel Core i7-7700 CPU @ 3.60GHz
Memory: 64 GB
2 x Samsung SSD 850 EVO M.2 1TB (M.2 SATA)
2 x Samsung SSD 860 EVO 2TB (SATA)
4 x WDC WD6002FFWX-68TZ4N0 (SATA) (6TB)
4 x Seagate ST12000VN0007-2GS116 (SATA) (12TB)
CPU: Intel Core i7-7700 CPU @ 3.60GHz
Memory: 64 GB
2 x Samsung SSD 850 EVO M.2 1TB (M.2 SATA)
2 x Samsung SSD 860 EVO 2TB (SATA)
4 x WDC WD6002FFWX-68TZ4N0 (SATA) (6TB)
4 x Seagate ST12000VN0007-2GS116 (SATA) (12TB)
-
- New here
- Posts: 5
- Joined: Thu Oct 11, 2018 10:51 pm
Re: Very slow uploads to B2 using Hybrid Backup Sync
Hmm thanks robert I'll check those out, it's been tough to find something that can handle the amount of TB I'm working with and they are all large video files. Definitely cannot wait for months to upload lol
-
- Getting the hang of things
- Posts: 93
- Joined: Mon Feb 12, 2018 9:26 pm
Re: Very slow uploads to B2 using Hybrid Backup Sync
I have a 1GB/s synchron line. So, I run two qBackup threads, each using 10 network threads. This gives me about 30MB/s upload to B2. Which is OK. I haven't played around with the parameters to see how far I could make it. I expect that 70-80MB/s should be possible.
TVS-1282T3
CPU: Intel Core i7-7700 CPU @ 3.60GHz
Memory: 64 GB
2 x Samsung SSD 850 EVO M.2 1TB (M.2 SATA)
2 x Samsung SSD 860 EVO 2TB (SATA)
4 x WDC WD6002FFWX-68TZ4N0 (SATA) (6TB)
4 x Seagate ST12000VN0007-2GS116 (SATA) (12TB)
CPU: Intel Core i7-7700 CPU @ 3.60GHz
Memory: 64 GB
2 x Samsung SSD 850 EVO M.2 1TB (M.2 SATA)
2 x Samsung SSD 860 EVO 2TB (SATA)
4 x WDC WD6002FFWX-68TZ4N0 (SATA) (6TB)
4 x Seagate ST12000VN0007-2GS116 (SATA) (12TB)
-
- New here
- Posts: 5
- Joined: Thu Oct 11, 2018 10:51 pm
Re: Very slow uploads to B2 using Hybrid Backup Sync
I'm thinking I'm an idiot because I assumed with 300mb/s upload speeds that I would be able to upload at those speeds! Lol Is that not the case with cloud service? I'm completely new to cloud.
-
- Getting the hang of things
- Posts: 93
- Joined: Mon Feb 12, 2018 9:26 pm
Re: Very slow uploads to B2 using Hybrid Backup Sync
Do you mean 300 Mega-Byte/s or Mega-Bit/s?
With a 1Gb (Giga-Bit) /s you will get a theoretical speed of about 100MB (Mega-Byte) /s. But there are many other factors like how quick is your CPU etc. So, Backblaze is very fast. You can really push at mostly max speed to them. I tested a couple of services and Backblaze was fastest for me.
With a 1Gb (Giga-Bit) /s you will get a theoretical speed of about 100MB (Mega-Byte) /s. But there are many other factors like how quick is your CPU etc. So, Backblaze is very fast. You can really push at mostly max speed to them. I tested a couple of services and Backblaze was fastest for me.
TVS-1282T3
CPU: Intel Core i7-7700 CPU @ 3.60GHz
Memory: 64 GB
2 x Samsung SSD 850 EVO M.2 1TB (M.2 SATA)
2 x Samsung SSD 860 EVO 2TB (SATA)
4 x WDC WD6002FFWX-68TZ4N0 (SATA) (6TB)
4 x Seagate ST12000VN0007-2GS116 (SATA) (12TB)
CPU: Intel Core i7-7700 CPU @ 3.60GHz
Memory: 64 GB
2 x Samsung SSD 850 EVO M.2 1TB (M.2 SATA)
2 x Samsung SSD 860 EVO 2TB (SATA)
4 x WDC WD6002FFWX-68TZ4N0 (SATA) (6TB)
4 x Seagate ST12000VN0007-2GS116 (SATA) (12TB)
-
- New here
- Posts: 5
- Joined: Thu Oct 11, 2018 10:51 pm
Re: Very slow uploads to B2 using Hybrid Backup Sync
I also thought that Mbits and Mb/s are the same thing, lol Guess I have some more to learn here. I just ran a backblaze speed test and it came out at
Download
807.0Mbit/s
Upload
304.7Mbit/s
Ping
61.5ms
Jitter
33.6ms
and QNap Hybrid is currently uploading at 14MB/s right now. So maybe I'm not that far off?
Download
807.0Mbit/s
Upload
304.7Mbit/s
Ping
61.5ms
Jitter
33.6ms
and QNap Hybrid is currently uploading at 14MB/s right now. So maybe I'm not that far off?
-
- Getting the hang of things
- Posts: 93
- Joined: Mon Feb 12, 2018 9:26 pm
Re: Very slow uploads to B2 using Hybrid Backup Sync
You are far off... it should run with about 50MB/s (Mega-Byte). Hybrid backup is horribly slow and inefficient.
TVS-1282T3
CPU: Intel Core i7-7700 CPU @ 3.60GHz
Memory: 64 GB
2 x Samsung SSD 850 EVO M.2 1TB (M.2 SATA)
2 x Samsung SSD 860 EVO 2TB (SATA)
4 x WDC WD6002FFWX-68TZ4N0 (SATA) (6TB)
4 x Seagate ST12000VN0007-2GS116 (SATA) (12TB)
CPU: Intel Core i7-7700 CPU @ 3.60GHz
Memory: 64 GB
2 x Samsung SSD 850 EVO M.2 1TB (M.2 SATA)
2 x Samsung SSD 860 EVO 2TB (SATA)
4 x WDC WD6002FFWX-68TZ4N0 (SATA) (6TB)
4 x Seagate ST12000VN0007-2GS116 (SATA) (12TB)
-
- First post
- Posts: 1
- Joined: Sun May 05, 2019 4:48 am
Re: Very slow uploads to B2 using Hybrid Backup Sync
Has anyone seen a fix for this?
I'm trying to upload to B2, and I'm seeing ~150kbps on my 1000Mb down/250Mb up connection.
I'm on a TVS-473e with 32GB ram, and (4) 12TB drives, running in raid 10.
Thanks,
Mike
I'm trying to upload to B2, and I'm seeing ~150kbps on my 1000Mb down/250Mb up connection.
I'm on a TVS-473e with 32GB ram, and (4) 12TB drives, running in raid 10.
Thanks,
Mike
- Moogle Stiltzkin
- Guru
- Posts: 11445
- Joined: Thu Dec 04, 2008 12:21 am
- Location: Around the world....
- Contact:
Re: Very slow uploads to B2 using Hybrid Backup Sync
the new HBS MAY help (pending QTS 4.4.x release for our qnap models )miketeachesclass wrote: ↑Sat May 18, 2019 1:45 am Has anyone seen a fix for this?
I'm trying to upload to B2, and I'm seeing ~150kbps on my 1000Mb down/250Mb up connection.
I'm on a TVS-473e with 32GB ram, and (4) 12TB drives, running in raid 10.
Thanks,
Mike
https://www.youtube.com/watch?v=KeQyptqo33M
https://www.youtube.com/watch?v=xsSPnBbuDNU
few things they are adding
- TCP BBR
https://www.youtube.com/watch?v=PkWleEfaGAU
- QuDedup
First thing to do is troubleshoot your own internet connection to see if that is the issue to begin with
https://www.speedtest.net/
https://www.backblaze.com/speedtest/
NAS
[Main Server] QNAP TS-877 (QTS) w. 4tb [ 3x HGST Deskstar NAS & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A (Truenas Core) w. 4x 2TB Samsung F3 (HD203WI) RaidZ1 ZFS + 8gb ddr3 Crucial
[^] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) 2x 4TB Seagate Ironwolf, Raid5
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-253D (Truenas Scale)
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1
Network
Qotom Pfsense|100mbps FTTH | Win11, Ryzen 5600X Desktop (1x2tb Crucial P50 Plus M.2 SSD, 1x 8tb seagate Ironwolf,1x 4tb HGST Ultrastar 7K4000)
Resources
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin
[Main Server] QNAP TS-877 (QTS) w. 4tb [ 3x HGST Deskstar NAS & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A (Truenas Core) w. 4x 2TB Samsung F3 (HD203WI) RaidZ1 ZFS + 8gb ddr3 Crucial
[^] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) 2x 4TB Seagate Ironwolf, Raid5
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-253D (Truenas Scale)
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1
Network
Qotom Pfsense|100mbps FTTH | Win11, Ryzen 5600X Desktop (1x2tb Crucial P50 Plus M.2 SSD, 1x 8tb seagate Ironwolf,1x 4tb HGST Ultrastar 7K4000)
Resources
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin