[HOWTO] How to increase raid rebuild speed

Discussion on setting up QNAP NAS products.
Locked
User avatar
TeeDub
Getting the hang of things
Posts: 97
Joined: Fri Oct 11, 2013 7:48 am

Re: [HOWTO] How to increase raid rebuild speed

Post by TeeDub » Tue Oct 22, 2013 11:32 am

Just a question

If you are going to be running the NAS for years what does it matter if it takes a day or two to sync drives on initial set up. Once complete it's all good.

My TS-421 with 4x3 TB reds, raid 5, took about 24 hrs to sync.

Seems to me this is a complete waste of effort.

Cheers

TW
TS-421
Version 4.0.2
4x3TB WD Reds RAID 5

User avatar
Don
Guru
Posts: 11970
Joined: Thu Jan 03, 2008 4:56 am
Location: Long Island, New York
Contact:

Re: [HOWTO] How to increase raid rebuild speed

Post by Don » Tue Oct 22, 2013 12:12 pm

Wouldn't you want the rebuid to go faster when you replace a failed drive?
Read the Online Manuals and use the forum search feature before posting.

It is a recommended to use RAID and have external backups. RAID will protect you from disk failure, keep your system running, and data accessible while the disk is replaced and the RAID rebuilt. Backups will allow you to recover data that is lost or corrupted, or from system failure. One does not replace the other.

Submit bugs and feature requests to QNAP via their Helpdesk app.

NAS: TVS-882BR | F/W: 4.3.6.0895 | 40GB | 2 x M.2 SATA RAID 1 (System/VMs) | 4 x M.2 NMVe QM2-4P-384A RAID 5 (Cache) | 5 x 4TB HDD RAID 6 (Data) | 1 x Blu-ray
NAS: TVS-663 | F/W: 4.4.1.1086 | 16GB | 2 x M.2 NMVe QM2-2P RAID 1 (Cache) | 4 x 4TB RAID 5
Apps: Boinc, Squid, DNSMasq, PLEX, iDrive, QVPN, QLMS, MP3fs, HBS, Entware, DLstation, +others

User avatar
schumaku
Guru
Posts: 43673
Joined: Mon Jan 21, 2008 4:41 pm
Location: Kloten (Zurich), Switzerland -- Skype: schumaku
Contact:

Re: [HOWTO] How to increase raid rebuild speed

Post by schumaku » Tue Oct 22, 2013 2:28 pm

TeeDub wrote:Just a question ... TW
Missing the [Like] and [+1] button in the forum 8)

User avatar
TeeDub
Getting the hang of things
Posts: 97
Joined: Fri Oct 11, 2013 7:48 am

Re: [HOWTO] How to increase raid rebuild speed

Post by TeeDub » Fri Oct 25, 2013 10:14 am

Don wrote:Wouldn't you want the rebuid to go faster when you replace a failed drive?


Well I could just replace the drive and let the NAS do what is designed to and get on with my day. Or...spend a bunch of time trying to get it run quicker.

I would rather just get on with my day.

Cheers

TW
TS-421
Version 4.0.2
4x3TB WD Reds RAID 5

User avatar
TeeDub
Getting the hang of things
Posts: 97
Joined: Fri Oct 11, 2013 7:48 am

Re: [HOWTO] How to increase raid rebuild speed

Post by TeeDub » Fri Oct 25, 2013 10:15 am

schumaku wrote:
TeeDub wrote:Just a question ... TW
Missing the [Like] and [+1] button in the forum 8)


LOL

Thanks mate

TW
TS-421
Version 4.0.2
4x3TB WD Reds RAID 5

tomas.kocanda
Getting the hang of things
Posts: 74
Joined: Wed Nov 27, 2013 8:09 am

Re: [HOWTO] How to increase raid rebuild speed

Post by tomas.kocanda » Wed Nov 27, 2013 9:59 am

Very interesting thread...
I´m using TS-459 Pro II with 4x 3TB Seagate Barracudas ST300DM001 in RAID5 and due to problems with those drives, decided to buy WD40EFRX what should be supported. Anyway, now I´m rebuilding one Seagate what failed few days ago (had spare one) and cannot get speed over 60000K/s.
Just curious, how some users get twice this speed with simmilar QNAP and drives?
What else to try? I did min and max limits and chache is by default I think. All services down. No users, no activity over network.
What should be maximum real speed with such configuration?

User avatar
marshalleq
Easy as a breeze
Posts: 302
Joined: Mon Jul 23, 2012 2:44 pm
Contact:

Re: [HOWTO] How to increase raid rebuild speed

Post by marshalleq » Wed Nov 27, 2013 2:06 pm

I think your speed is pretty good. If you're replacing your drive (so maintaining RAID 5 with 3 disks not adding to 4 disks) it should be relatively conisistent. For the second option you probably have it slower at the start and then speed up half way through. confirm with cat that your settings stuck.

In addition number of drives will have a hit I expect, in raid 5 3 disk it calculates 50% of data by utilising the CPU, in 4 disk it would use CPU to calculate 33% of the data and so on (I assume) - I expect this makes a difference. Looking at my results compared to the previous persons getting more speed I see they have 5 disks and I have 4 I expect that to mean 25% CPU. Also the disabling of onboard services will make a difference and my NAS being an L series has limited SATA speeds...
TS669-L (3GB RAM) fw 4.3.2.0050 Build 20170104 with 6x4TB Seagate ST4000VN000-1H41SC43 2x2TB Hitachi HDS723020BLA642 2x8TB ST8000VN0022-2EL112
QNAP UX-500P External 5 Bay USB3 Expansion Cabinet
Raspberry Pi 1,2 & 3
Mac Mini
Apple TV 3/4
Windows
Sabayon Linux
Ubuntu Linux
Android
Docker
Ryzen 1800x custom system

Web
https://www.tech-knowhow.com

Twitter
@Marshalleq
@TechKn0wH0w

tomas.kocanda
Getting the hang of things
Posts: 74
Joined: Wed Nov 27, 2013 8:09 am

Re: [HOWTO] How to increase raid rebuild speed

Post by tomas.kocanda » Thu Nov 28, 2013 7:48 am

Thanks marshalleq.
I´m rebuilding array with new disks (3TB -> 4TB) and with new settings from the thread I got speed around 71000K/s and processing time less then 10 hours per drive, what is very good. Thanks!

kbk00
Know my way around
Posts: 196
Joined: Mon Dec 02, 2013 12:28 pm

Re: [HOWTO] How to increase raid rebuild speed

Post by kbk00 » Thu Dec 05, 2013 7:07 am

Adding another data point to the list:

ts-870 (celeron version)
default 2G of RAM with about 25% utilized
adding 1 x 4tb WD Re (WD4000F9YZ) to an existing RAID6 of 6 current drives of the same type.

default

    CPU @ ~30%
    [>....................] reshape = 1.4% (57772096/3897063616) finish=1531.4min speed=41781K/sec

min set to 170000
    CPU @ ~32%
    [>....................] reshape = 2.8% (109609216/3897063616) finish=1391.8min speed=43352K/sec

max set to 250000
    CPU @ ~30%
    [>....................] reshape = 3.3% (128887396/3897063616) finish=1378.2min speed=43567K/sec


Essentially no change as the speeds bounce around by about +-7% or so. I'm disappointed that I got zero impact and it's a little hard for me to understand. My disks are fast and from what I can see, the bottleneck isn't apparent. Proc looks more than good and the disks should be able to handle over twice as much traffic. I can't see if the spindles are thrashing but I assume that they aren't but I don't know of any tool to look that deep into the system (I'm a windows admin after all :)). Cache size adjustments did nothing for throughput yet it increased memory (as expected and stated in the thread earlier) but also processor. After setting the cache to 32M, my proc util went up about 50%.

Side note: I'm confused by the benefit of raising the min setting. Are most of your NAS units under contention currently? How I would read the minimum setting would come into play if the system is idle, the max setting would be the limiting factor and raising the floor (min) would affect the rebuild or reshape process when there's competing requests on the hardware. Can anyone enlighten me?

User avatar
forkless
Experience counts
Posts: 1921
Joined: Mon Nov 23, 2009 6:52 am
Location: The Netherlands

Re: [HOWTO] How to increase raid rebuild speed

Post by forkless » Thu Dec 05, 2013 7:22 am

I've had good result with increasing the rebuild speed on several occassions, however people should realize it comes with a severe penalty to the system performance. I had several services die on my when using this method in the past.

User avatar
Perlbo76
Know my way around
Posts: 132
Joined: Thu Feb 16, 2012 2:05 am

Re: [HOWTO] How to increase raid rebuild speed

Post by Perlbo76 » Sat Jan 04, 2014 10:12 pm

Hi ... it is no problem for me, how long rebuild needs... but i add to my 3 Disk 3TB one 3TB Disk. Until the process is finished it seems, that i have no access to my data. Is that normal ?

    Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] [multipath]
    md0 : active raid5 sdd3[3] sda3[0] sdc3[2] sdb3[1]
    5857395072 blocks super 1.0 level 5, 64k chunk, algorithm 2 [4/4] [UUUU]
    [>....................] reshape = 3.3% (96973056/2928697536) finish=1543.3min speed=30578K/sec

    md256 : active raid1 sdd2[3](S) sdc2[2](S) sdb2[1] sda2[0]
    530112 blocks super 1.0 [2/2] [UU]
    bitmap: 0/1 pages [0KB], 65536KB chunk

    md13 : active raid1 sdd4[3] sda4[0] sdc4[2] sdb4[1]
    458880 blocks [4/4] [UUUU]
    bitmap: 0/57 pages [0KB], 4KB chunk

    md9 : active raid1 sdd1[3] sda1[0] sdc1[2] sdb1[1]
    530048 blocks [4/4] [UUUU]
    bitmap: 0/65 pages [0KB], 4KB chunk

    unused devices: <none>
Qnap TVS-873 16GB Ram

User avatar
marshalleq
Easy as a breeze
Posts: 302
Joined: Mon Jul 23, 2012 2:44 pm
Contact:

Re: [HOWTO] How to increase raid rebuild speed

Post by marshalleq » Sun Jan 05, 2014 3:57 pm

I think with a reshape it is yes. but not a rebuild.
TS669-L (3GB RAM) fw 4.3.2.0050 Build 20170104 with 6x4TB Seagate ST4000VN000-1H41SC43 2x2TB Hitachi HDS723020BLA642 2x8TB ST8000VN0022-2EL112
QNAP UX-500P External 5 Bay USB3 Expansion Cabinet
Raspberry Pi 1,2 & 3
Mac Mini
Apple TV 3/4
Windows
Sabayon Linux
Ubuntu Linux
Android
Docker
Ryzen 1800x custom system

Web
https://www.tech-knowhow.com

Twitter
@Marshalleq
@TechKn0wH0w

User avatar
marshalleq
Easy as a breeze
Posts: 302
Joined: Mon Jul 23, 2012 2:44 pm
Contact:

Re: [HOWTO] How to increase raid rebuild speed

Post by marshalleq » Sun Jan 05, 2014 3:58 pm

Now you want to speed it up though right? ;) I don't think the speed up will affect the reshape, but it won't hurt to do it anyway.
TS669-L (3GB RAM) fw 4.3.2.0050 Build 20170104 with 6x4TB Seagate ST4000VN000-1H41SC43 2x2TB Hitachi HDS723020BLA642 2x8TB ST8000VN0022-2EL112
QNAP UX-500P External 5 Bay USB3 Expansion Cabinet
Raspberry Pi 1,2 & 3
Mac Mini
Apple TV 3/4
Windows
Sabayon Linux
Ubuntu Linux
Android
Docker
Ryzen 1800x custom system

Web
https://www.tech-knowhow.com

Twitter
@Marshalleq
@TechKn0wH0w

User avatar
Perlbo76
Know my way around
Posts: 132
Joined: Thu Feb 16, 2012 2:05 am

Re: [HOWTO] How to increase raid rebuild speed

Post by Perlbo76 » Thu Jan 09, 2014 2:24 am

Thanks for the informations ... especially the information about available Data on rebuild....
Qnap TVS-873 16GB Ram

StardustOne
Know my way around
Posts: 117
Joined: Sat Sep 15, 2007 5:18 am

Re: [HOWTO] How to increase raid rebuild speed

Post by StardustOne » Sun Jan 19, 2014 1:08 am

What can be done using Putty and a few commands must also be possible to implement with a Menu function. It is getting so much on my nerves that a rebuild or even a sync takes days on a QNAP NAS, please, you engineers out there, address this asap and make it possible to select the rebuild speed in a menu.

I was able to set the speed with putty and it also worked, although, everyone is recommending to put this in something called sh and I am no linux expert nor am I good enough at Unix. So why can't we have a simple solution for this issue?

A menu function where I can decide how fast I want to rebuild, why isn't this not implemented with the QNAP firmware long time ago. It is pretty much pointless naming these NAS turbo when you as a user can't even control such basic features as rebuild speed to cut down on rebuilds.

Is there a step by step instruction how this sh thing works? I may be able to follow this, although it has to be written for dummies that do not know much about Linux but can follow exact, correct and detailed instructions that are not full of surprises and work 100% if you follow them correctly.

If someone knows how to provide to have this update applied at startup, please let us know. I am not interested in killing my NAS speed, but I am very much interested in cutting down this silly multiple day reconstruction time because some clever or not so clever engineer thought I can't decide on my own how fast I want it to rebuild or migrate or expand.

Locked

Return to “Turbo Station Installation & Setup”