Very slow uploads to B2 using Hybrid Backup Sync

Backup, Restore, Netbak Replicator, Cloud Storage Services
Post Reply
andreiz
New here
Posts: 4
Joined: Mon Aug 14, 2017 11:59 pm

Very slow uploads to B2 using Hybrid Backup Sync

Post by andreiz » Sat Sep 01, 2018 6:17 am

I set up a job to do a one-way sync of my Photos folder on the QNAP to the Backblaze B2 bucket. It started off doing about 800Kbps-1.2Mpbs, but after a while slowed down to just a few Kbps. This is the initial sync, so the bucket didn't have any files. Transfers to B2 from my laptop are on the order of 1.2-1.5 Mbps. Doing an rsync of a large file from QNAP to a remote server is about the same. Any ideas what may be causing the very slow upload speeds in HBS?

-Andrei

johnvick
Know my way around
Posts: 104
Joined: Thu May 03, 2012 2:19 pm

Re: Very slow uploads to B2 using Hybrid Backup Sync

Post by johnvick » Sat Sep 01, 2018 7:24 am

QNAP have confirmed there is a bug in HBS and to wait for a fix to come through.

John
TS-420 3 x 3TB Seagate SD3000DM001, 1 x 3TB WD Red RAID 5, FW 4.3.3.0789
TS-212 2 x 3TB Seagate SD3000DM001, FW 4.3.3.0789

User avatar
twiga
Starting out
Posts: 26
Joined: Tue Dec 29, 2015 2:09 pm

Re: Very slow uploads to B2 using Hybrid Backup Sync

Post by twiga » Sun Sep 02, 2018 12:25 am

I am having the same problem. Any ETA on fix for HBS?
Model: TS-653 Pro
Ram: 2G
4x WD Red 4TB
QTS 4.2.1
Raid 5

johnvick
Know my way around
Posts: 104
Joined: Thu May 03, 2012 2:19 pm

Re: Very slow uploads to B2 using Hybrid Backup Sync

Post by johnvick » Sun Sep 02, 2018 7:19 am

I've been told to wait and it will come through, so no idea.
TS-420 3 x 3TB Seagate SD3000DM001, 1 x 3TB WD Red RAID 5, FW 4.3.3.0789
TS-212 2 x 3TB Seagate SD3000DM001, FW 4.3.3.0789

robert_m_muench
Getting the hang of things
Posts: 82
Joined: Mon Feb 12, 2018 9:26 pm

Re: Very slow uploads to B2 using Hybrid Backup Sync

Post by robert_m_muench » Sun Sep 09, 2018 9:15 pm

So, the latest update didn't fix this problem, right? It's still totally slow here.
TVS-1282T3
CPU: Intel Core i7-7700 CPU @ 3.60GHz
Memory: 64 GB
2 x Samsung SSD 850 EVO M.2 1TB (M.2 SATA)
2 x Samsung SSD 860 EVO 2TB (SATA)
4 x WDC WD6002FFWX-68TZ4N0 (SATA) (6TB)
4 x Seagate ST12000VN0007-2GS116 (SATA) (12TB)

Hobbitual
New here
Posts: 5
Joined: Thu Oct 11, 2018 10:51 pm

Re: Very slow uploads to B2 using Hybrid Backup Sync

Post by Hobbitual » Thu Oct 11, 2018 11:01 pm

Ugh I'm dealing with a QNAP tech guy right now with this. I have Broadband speeds of 800mbs and Backblaze is clocking me at over 300mbs upload to their site yet I have a measly 10mbs coming from the NAS TVS-463. I bought a fancy new Blue Wave Router, all new CAT 6 cabling added RAM in the NAS, RAID 5 with 4TB WD REDs and still slow as **.

If I would have known QNAP was this slow I would have bought something else. They false advertised the whole Turbo NAS deal.

robert_m_muench
Getting the hang of things
Posts: 82
Joined: Mon Feb 12, 2018 9:26 pm

Re: Very slow uploads to B2 using Hybrid Backup Sync

Post by robert_m_muench » Fri Oct 12, 2018 12:18 am

I'm now using qBackup, which is very fast, does an incremental backup and uses some hashed-block names to store things. So, no files in your bucket. Take a look, very good. The only drawback is, that the use a stored index. If something happens to it, before having a first complete backup you have to start over.

Or this here: https://github.com/gilbertchen/duplicacy currently some issues with B2 but I saw a ticket fixing it. Not checked yet, but if it works I would even prefer it over qBackup as it's not proprietary nor uses a central index.

The QNAP Hybrid Backup thing is OK but not really professional. So, it's way to slow and no intelligent incremental backups.
TVS-1282T3
CPU: Intel Core i7-7700 CPU @ 3.60GHz
Memory: 64 GB
2 x Samsung SSD 850 EVO M.2 1TB (M.2 SATA)
2 x Samsung SSD 860 EVO 2TB (SATA)
4 x WDC WD6002FFWX-68TZ4N0 (SATA) (6TB)
4 x Seagate ST12000VN0007-2GS116 (SATA) (12TB)

Hobbitual
New here
Posts: 5
Joined: Thu Oct 11, 2018 10:51 pm

Re: Very slow uploads to B2 using Hybrid Backup Sync

Post by Hobbitual » Fri Oct 12, 2018 12:56 am

Hmm thanks robert I'll check those out, it's been tough to find something that can handle the amount of TB I'm working with and they are all large video files. Definitely cannot wait for months to upload lol

robert_m_muench
Getting the hang of things
Posts: 82
Joined: Mon Feb 12, 2018 9:26 pm

Re: Very slow uploads to B2 using Hybrid Backup Sync

Post by robert_m_muench » Fri Oct 12, 2018 1:42 am

I have a 1GB/s synchron line. So, I run two qBackup threads, each using 10 network threads. This gives me about 30MB/s upload to B2. Which is OK. I haven't played around with the parameters to see how far I could make it. I expect that 70-80MB/s should be possible.
TVS-1282T3
CPU: Intel Core i7-7700 CPU @ 3.60GHz
Memory: 64 GB
2 x Samsung SSD 850 EVO M.2 1TB (M.2 SATA)
2 x Samsung SSD 860 EVO 2TB (SATA)
4 x WDC WD6002FFWX-68TZ4N0 (SATA) (6TB)
4 x Seagate ST12000VN0007-2GS116 (SATA) (12TB)

Hobbitual
New here
Posts: 5
Joined: Thu Oct 11, 2018 10:51 pm

Re: Very slow uploads to B2 using Hybrid Backup Sync

Post by Hobbitual » Fri Oct 12, 2018 6:00 am

I'm thinking I'm an idiot because I assumed with 300mb/s upload speeds that I would be able to upload at those speeds! Lol Is that not the case with cloud service? I'm completely new to cloud.

robert_m_muench
Getting the hang of things
Posts: 82
Joined: Mon Feb 12, 2018 9:26 pm

Re: Very slow uploads to B2 using Hybrid Backup Sync

Post by robert_m_muench » Fri Oct 12, 2018 3:28 pm

Do you mean 300 Mega-Byte/s or Mega-Bit/s?

With a 1Gb (Giga-Bit) /s you will get a theoretical speed of about 100MB (Mega-Byte) /s. But there are many other factors like how quick is your CPU etc. So, Backblaze is very fast. You can really push at mostly max speed to them. I tested a couple of services and Backblaze was fastest for me.
TVS-1282T3
CPU: Intel Core i7-7700 CPU @ 3.60GHz
Memory: 64 GB
2 x Samsung SSD 850 EVO M.2 1TB (M.2 SATA)
2 x Samsung SSD 860 EVO 2TB (SATA)
4 x WDC WD6002FFWX-68TZ4N0 (SATA) (6TB)
4 x Seagate ST12000VN0007-2GS116 (SATA) (12TB)

Hobbitual
New here
Posts: 5
Joined: Thu Oct 11, 2018 10:51 pm

Re: Very slow uploads to B2 using Hybrid Backup Sync

Post by Hobbitual » Sat Oct 13, 2018 12:12 am

I also thought that Mbits and Mb/s are the same thing, lol Guess I have some more to learn here. I just ran a backblaze speed test and it came out at
Download
807.0Mbit/s
Upload
304.7Mbit/s
Ping
61.5ms
Jitter
33.6ms

and QNap Hybrid is currently uploading at 14MB/s right now. So maybe I'm not that far off?

robert_m_muench
Getting the hang of things
Posts: 82
Joined: Mon Feb 12, 2018 9:26 pm

Re: Very slow uploads to B2 using Hybrid Backup Sync

Post by robert_m_muench » Sat Oct 13, 2018 10:44 pm

You are far off... it should run with about 50MB/s (Mega-Byte). Hybrid backup is horribly slow and inefficient.
TVS-1282T3
CPU: Intel Core i7-7700 CPU @ 3.60GHz
Memory: 64 GB
2 x Samsung SSD 850 EVO M.2 1TB (M.2 SATA)
2 x Samsung SSD 860 EVO 2TB (SATA)
4 x WDC WD6002FFWX-68TZ4N0 (SATA) (6TB)
4 x Seagate ST12000VN0007-2GS116 (SATA) (12TB)

miketeachesclass
First post
Posts: 1
Joined: Sun May 05, 2019 4:48 am

Re: Very slow uploads to B2 using Hybrid Backup Sync

Post by miketeachesclass » Sat May 18, 2019 1:45 am

Has anyone seen a fix for this?

I'm trying to upload to B2, and I'm seeing ~150kbps on my 1000Mb down/250Mb up connection.

I'm on a TVS-473e with 32GB ram, and (4) 12TB drives, running in raid 10.

Thanks,
Mike

User avatar
Moogle Stiltzkin
Ask me anything
Posts: 7125
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: Very slow uploads to B2 using Hybrid Backup Sync

Post by Moogle Stiltzkin » Sat May 18, 2019 3:37 am

miketeachesclass wrote:
Sat May 18, 2019 1:45 am
Has anyone seen a fix for this?

I'm trying to upload to B2, and I'm seeing ~150kbps on my 1000Mb down/250Mb up connection.

I'm on a TVS-473e with 32GB ram, and (4) 12TB drives, running in raid 10.

Thanks,
Mike
the new HBS MAY help :' (pending QTS 4.4.x release for our qnap models :( )

https://www.youtube.com/watch?v=KeQyptqo33M

https://www.youtube.com/watch?v=xsSPnBbuDNU

few things they are adding
- TCP BBR
https://www.youtube.com/watch?v=PkWleEfaGAU

- QuDedup



First thing to do is troubleshoot your own internet connection to see if that is the issue to begin with
https://www.speedtest.net/
https://www.backblaze.com/speedtest/
NAS
[Main Server] QNAP TS-877 w. 4 HDD total x 4tb [ 3x HGST Deskstar NAS (HDN724040ALE640) & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 Samsung 850 Evo raid1 + 16gb ddr4 Kingston Hyperx Fury + QWA-AC2600 wireless adapter addon.
[Backup] QNAP TS-653A w. 5x 2TB Samsung EcoGreen F3 (HD203WI) EXT4 Raid5
[Backup] QNAP TS-659 Pro
[Backup] QNAP TS-509 Pro w. 4x 1TB Western Digital RE3 (WD1002FBYS) EXT4 Raid5
[Backup] QNAP TS-228
[Backup] QNAP TS-128 w. 1x 1TB Western Digital RE3 (WD1002FBYS)

Network
Asus AC68U Router|100mbps dl/50mbps ul FTTH Internet | Windows 10 Enterprise, Water Cooled PC-Intel i7 920 Ivy bridge desktop (1x 512gb Samsung 850 Pro SSD + 1x 4tb HGST Ultrastar 7K4000)


Guides and articles by me
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review


https://www.patreon.com/mooglestiltzkin

Post Reply

Return to “Backup & Restore”