HBS3 Backup is slow

Backup, Restore, Netbak Replicator, Cloud Storage Services
thilan
New here
Posts: 8
Joined: Sat Feb 08, 2020 1:45 am

HBS3 Backup is slow

Post by thilan »

Dear All

i am using a Qnap at my office
Capture1.PNG
it have 2 volumes like this
Capture2.PNG
i am using oneway sync to back up my data as here
Capture3.PNG
backup nas is this
Capture4.PNG
so my backup process is two slow
Capture5.PNG
Capture6.PNG
as in the images

how i can solve this
please asist
thank you
You do not have the required permissions to view the files attached to this post.
P3R
Guru
Posts: 13192
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: HBS3 Backup is slow

Post by P3R »

I assume you use the RTRR protocol?

Untick the Exclude duplicate files policy option.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
User avatar
Moogle Stiltzkin
Guru
Posts: 11448
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: HBS3 Backup is slow

Post by Moogle Stiltzkin »

you have to elaborate more on your hbs settings. what did you set in it?

there are a few things that can affect the performance. like

- check comparison
- ssl encryption
- probably some others

also is this being done over the internet using vpn? or is this in the local lan?
NAS
[Main Server] QNAP TS-877 (QTS) w. 4tb [ 3x HGST Deskstar NAS & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A (Truenas Core) w. 4x 2TB Samsung F3 (HD203WI) RaidZ1 ZFS + 8gb ddr3 Crucial
[^] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) 2x 4TB Seagate Ironwolf, Raid5
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-253D (Truenas Scale)
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Qotom Pfsense|100mbps FTTH | Win11, Ryzen 5600X Desktop (1x2tb Crucial P50 Plus M.2 SSD, 1x 8tb seagate Ironwolf,1x 4tb HGST Ultrastar 7K4000)


Resources
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin
thilan
New here
Posts: 8
Joined: Sat Feb 08, 2020 1:45 am

Re: HBS3 Backup is slow

Post by thilan »

Yes
both RTRR and Rsync are enabled
rttt.PNG
rsync.PNG
do i have to use only RTRR ?
or Rsync ?
only
You do not have the required permissions to view the files attached to this post.
thilan
New here
Posts: 8
Joined: Sat Feb 08, 2020 1:45 am

Re: HBS3 Backup is slow

Post by thilan »

Moogle Stiltzkin wrote: Sat Feb 08, 2020 2:24 am you have to elaborate more on your hbs settings. what did you set in it?

there are a few things that can affect the performance. like

- check comparison
- ssl encryption
- probably some others

also is this being done over the internet using vpn? or is this in the local lan?
yes
both nas are in same lan
behind a fortigate firewall with all ports and service are allowed too
User avatar
Moogle Stiltzkin
Guru
Posts: 11448
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: HBS3 Backup is slow

Post by Moogle Stiltzkin »

well if it's on the same lan, you could always go for 10gbe networking to improve the speeds
https://blog.qnap.com/10gbe-walkthrough-en/

otherwise you are stuck on 1gbe networking? then the only solution is to tweak your HBS settings.
NAS
[Main Server] QNAP TS-877 (QTS) w. 4tb [ 3x HGST Deskstar NAS & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A (Truenas Core) w. 4x 2TB Samsung F3 (HD203WI) RaidZ1 ZFS + 8gb ddr3 Crucial
[^] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) 2x 4TB Seagate Ironwolf, Raid5
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-253D (Truenas Scale)
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Qotom Pfsense|100mbps FTTH | Win11, Ryzen 5600X Desktop (1x2tb Crucial P50 Plus M.2 SSD, 1x 8tb seagate Ironwolf,1x 4tb HGST Ultrastar 7K4000)


Resources
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin
P3R
Guru
Posts: 13192
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: HBS3 Backup is slow

Post by P3R »

That's the server part and that server (is that on the destination NAS, 172.20.10.95?) accept both RTRR and Rsync. But it's the actual sync job that determines what protocol is used.

With RTRR (normally the preferred protocol when on a LAN) the Exclude duplicate files option is normally best unticked. With Rsync it's the opposite and having it enabled is usually better.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
thilan
New here
Posts: 8
Joined: Sat Feb 08, 2020 1:45 am

Re: HBS3 Backup is slow

Post by thilan »

Moogle Stiltzkin wrote: Sat Feb 08, 2020 2:39 am well if it's on the same lan, you could always go for 10gbe networking to improve the speeds
https://blog.qnap.com/10gbe-walkthrough-en/

otherwise you are stuck on 1gbe networking? then the only solution is to tweak your HBS settings.
do i have to use the device as in the blog ?
P3R
Guru
Posts: 13192
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: HBS3 Backup is slow

Post by P3R »

thilan wrote: Sat Feb 08, 2020 2:46 am do i have to use the device as in the blog ?
Your TS-451+ doesn't support 10 GbE so could we please at least first concentrate on what you have now and how we can make that to work the best it can?
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
thilan
New here
Posts: 8
Joined: Sat Feb 08, 2020 1:45 am

Re: HBS3 Backup is slow

Post by thilan »

P3R wrote: Sat Feb 08, 2020 2:51 am
thilan wrote: Sat Feb 08, 2020 2:46 am do i have to use the device as in the blog ?
Your TS-451+ doesn't support 10 GbE so could we please at least first concentrate on what you have now and how we can make that to work the best it can?
Yes Sir
i am fed up with these Qnap nas now
as one time that ts-ec880u fails and lost all valuable photos in our organization
we faced to lot of difficulties
that why we try this backup solution now it is also getting fails
P3R
Guru
Posts: 13192
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: HBS3 Backup is slow

Post by P3R »

thilan wrote: Sat Feb 08, 2020 2:57 am as one time that ts-ec880u fails and lost all valuable photos in our organization
No. It wasn't the failing NAS that lost all valuable photos, it was the mistake of the administrator to not keep backups on independent systems of the important data that caused the data loss. I'm sorry if that offend you but that's how it is.
...that why we try this backup solution now it is also getting fails
As far as I can tell it's not failing but it takes around 6 hours to synchronize the roughly 1.7 million files you have there with several tens of thousand files being updated each time. We need to analyze that more to see if it's normal and expected but is it the 6 hour execution time that you feel is the big problem? If it is, why is it a problem if it run during the night (when I assume the systems are not used for work)?

What protocol did you use in your sync job?
If you don't know, then disable Rsync on the destination server and see if the job still start. if it does, you're running RTRR and can keep the Rsync server on the destination disabled (unless it's needed for something else?).

Also do you have the Exclude duplicate files policy option enabled or disabled?
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
User avatar
Moogle Stiltzkin
Guru
Posts: 11448
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: HBS3 Backup is slow

Post by Moogle Stiltzkin »

please read this in regards to backups :}
https://www.reddit.com/r/qnap/comments/ ... _a_backup/
NAS
[Main Server] QNAP TS-877 (QTS) w. 4tb [ 3x HGST Deskstar NAS & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A (Truenas Core) w. 4x 2TB Samsung F3 (HD203WI) RaidZ1 ZFS + 8gb ddr3 Crucial
[^] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) 2x 4TB Seagate Ironwolf, Raid5
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-253D (Truenas Scale)
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Qotom Pfsense|100mbps FTTH | Win11, Ryzen 5600X Desktop (1x2tb Crucial P50 Plus M.2 SSD, 1x 8tb seagate Ironwolf,1x 4tb HGST Ultrastar 7K4000)


Resources
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin
thilan
New here
Posts: 8
Joined: Sat Feb 08, 2020 1:45 am

Re: HBS3 Backup is slow

Post by thilan »

Moogle Stiltzkin wrote: Sat Feb 08, 2020 5:38 am please read this in regards to backups :}
https://www.reddit.com/r/qnap/comments/ ... _a_backup/
thanks a lot sir
thilan
New here
Posts: 8
Joined: Sat Feb 08, 2020 1:45 am

Re: HBS3 Backup is slow

Post by thilan »

P3R wrote: Sat Feb 08, 2020 3:57 am
thilan wrote: Sat Feb 08, 2020 2:57 am as one time that ts-ec880u fails and lost all valuable photos in our organization
No. It wasn't the failing NAS that lost all valuable photos, it was the mistake of the administrator to not keep backups on independent systems of the important data that caused the data loss. I'm sorry if that offend you but that's how it is.
...that why we try this backup solution now it is also getting fails
As far as I can tell it's not failing but it takes around 6 hours to synchronize the roughly 1.7 million files you have there with several tens of thousand files being updated each time. We need to analyze that more to see if it's normal and expected but is it the 6 hour execution time that you feel is the big problem? If it is, why is it a problem if it run during the night (when I assume the systems are not used for work)?

What protocol did you use in your sync job?
If you don't know, then disable Rsync on the destination server and see if the job still start. if it does, you're running RTRR and can keep the Rsync server on the destination disabled (unless it's needed for something else?).

Also do you have the Exclude duplicate files policy option enabled or disabled?
thanks a lot sir
thilan
New here
Posts: 8
Joined: Sat Feb 08, 2020 1:45 am

Re: HBS3 Backup is slow

Post by thilan »

1111.JPG
According to my Case
as both nas in same lan
what is the best back up solution do i have to use to backup my volume1?
while the backup runs
users add files to the volume also
as this allows in a live photo-sharing system

HBS3 Backup Or sync ?
if i used a sync will the active sync help me to lower the time ?
You do not have the required permissions to view the files attached to this post.
Post Reply

Return to “Backup & Restore”