Thanks for sharing![color=#000000]P3R[/color] wrote:My assumption would instead be that your very fast CPU can easily manage this task already without changing this setting and that your disks are the performance bottlenecks.[color=#000000]Ifti[/color] wrote:Same here - adjusted mine to 200000 and my CPU doesn't go over 2%, so I can only assume its not working!!This thread was started in a time when the fastest NAS CPUs had someting like 1/15th or 1/20th of the processing capacity your CPU have.I have a TVS-871T (i7 model)Looks decent to me. What's wrong with it?[~] # egrep speed /proc/mdstat
[=>...................] resync = 9.1% (534795552/5850567168) finish=639.6min speed=138511K/sec
[HOWTO] How to increase raid rebuild speed
-
- New here
- Posts: 2
- Joined: Thu Oct 20, 2016 10:53 pm
Re: [HOWTO] How to increase raid rebuild speed
Last edited by jad albert on Fri Oct 21, 2016 8:05 pm, edited 1 time in total.
- Trexx
- Ask me anything
- Posts: 5393
- Joined: Sat Oct 01, 2011 7:50 am
- Location: Minnesota
Re: [HOWTO] How to increase raid rebuild speed
Just a note: For migrations from Raid1-Raid5, the first 50% of the rebuild will tend to be slow, not sure specially was going on behind the scenes,but once it hits the 50%+ your throughput should increase significantly. In my case, it was about 3-4x of that it was between 0-50%.
Paul
Model: TS-877-1600 FW: 4.5.3.x
QTS (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
Data (HDD): [RAID-5] 6 x 3TB HGST DeskStar
VMs (SSD): [RAID-1] 2 x1TB SK Hynix Gold
Ext. (HDD): TR-004 [Raid-5] 4 x 4TB HGST Ultastor
RAM: Kingston HyperX Fury 64GB DDR4-2666
UPS: CP AVR1350
Model:TVS-673 32GB & TS-228a Offline[/color]
-----------------------------------------------------------------------------------------------------------------------------------------
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq
Model: TS-877-1600 FW: 4.5.3.x
QTS (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
Data (HDD): [RAID-5] 6 x 3TB HGST DeskStar
VMs (SSD): [RAID-1] 2 x1TB SK Hynix Gold
Ext. (HDD): TR-004 [Raid-5] 4 x 4TB HGST Ultastor
RAM: Kingston HyperX Fury 64GB DDR4-2666
UPS: CP AVR1350
Model:TVS-673 32GB & TS-228a Offline[/color]
-----------------------------------------------------------------------------------------------------------------------------------------
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq
- avvidme
- Know my way around
- Posts: 185
- Joined: Fri Jan 16, 2009 10:36 am
- Contact:
Re: [HOWTO] How to increase raid rebuild speed
Hi everyone, I'm doing a Raid 5 to 6 migration on a TS-853 Pro on new WD 4TB drives (7 in total).
Here's the status:
[~] # cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] [multipath]
md1 : active raid6 sdf3[6] sdg3[0] sdd3[5] sdc3[4] sdb3[3] sda3[2] sdh3[1]
19485318080 blocks super 1.0 level 6, 64k chunk, algorithm 18 [7/6] [UUUUUU_]
[====>................] reshape = 21.0% (820684800/3897063616) finish=16520.9min speed=3103K/sec
No matter ANY setting I use such as
echo 800000 >/proc/sys/dev/raid/speed_limit_max
echo 400000 >/proc/sys/dev/raid/speed_limit_min
And cache size up to 32k. The speed of 3103k just won't change, and the CPU utilization is next to nothing.
After 4 days, I still have about 10 days to go!!!!!
HELP! is there ANY other way to increase the speed of this migration???
Yes I have Bitmap on, but is it safe to turn it off during the migration?
Help!
Here's the status:
[~] # cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] [multipath]
md1 : active raid6 sdf3[6] sdg3[0] sdd3[5] sdc3[4] sdb3[3] sda3[2] sdh3[1]
19485318080 blocks super 1.0 level 6, 64k chunk, algorithm 18 [7/6] [UUUUUU_]
[====>................] reshape = 21.0% (820684800/3897063616) finish=16520.9min speed=3103K/sec
No matter ANY setting I use such as
echo 800000 >/proc/sys/dev/raid/speed_limit_max
echo 400000 >/proc/sys/dev/raid/speed_limit_min
And cache size up to 32k. The speed of 3103k just won't change, and the CPU utilization is next to nothing.
After 4 days, I still have about 10 days to go!!!!!
HELP! is there ANY other way to increase the speed of this migration???
Yes I have Bitmap on, but is it safe to turn it off during the migration?
Help!
_______________________________________________________________________
Checkout http://www.digitalgatehouse.com for more of my ramblings!
Checkout http://www.digitalgatehouse.com for more of my ramblings!
- Spider99
- Experience counts
- Posts: 1951
- Joined: Fri Oct 21, 2011 11:14 pm
- Location: UK
Re: [HOWTO] How to increase raid rebuild speed
missing disk is affecting rebuild/migration?
Tim
TS-853A(16GB): - 4.3.4.0483 - Static volume - Raid5 - 8 x 4TB HGST Deskstar NAS
Windows Server + StableBit Drivepool and Scanner ~115 TB Backup Server
TS-412 & TS-459 Pro II: Retired
Clients: 3 x Windows 10 Pro(64bit)
TS-853A(16GB): - 4.3.4.0483 - Static volume - Raid5 - 8 x 4TB HGST Deskstar NAS
Windows Server + StableBit Drivepool and Scanner ~115 TB Backup Server
TS-412 & TS-459 Pro II: Retired
Clients: 3 x Windows 10 Pro(64bit)
-
- Guru
- Posts: 13192
- Joined: Sat Dec 29, 2007 1:39 am
- Location: Stockholm, Sweden (UTC+01:00)
Re: [HOWTO] How to increase raid rebuild speed
I wouldn't worry about the low CPU utilization. Today's CPUs are much faster so are unlikely to be a bottleneck. It's usually the disks that's the limitation.avvidme wrote:The speed of 3103k just won't change, and the CPU utilization is next to nothing.
Do you have anything else active that access the disks? Active clients in the network and/or active services/apps on the NAS? Virtualization, torrent downloading or anything else?
That prediction isn't necessarily accurate.After 4 days, I still have about 10 days to go!!!!!
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!
A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.
All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.
All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
- avvidme
- Know my way around
- Posts: 185
- Joined: Fri Jan 16, 2009 10:36 am
- Contact:
Re: [HOWTO] How to increase raid rebuild speed
Thanks for your reply;
Do you have anything else active that access the disks? Active clients in the network and/or active services/apps on the NAS? Virtualization, torrent downloading or anything else?
Nothing, I shut off all applications before I started the migration. Bare bones services running.
Do you have anything else active that access the disks? Active clients in the network and/or active services/apps on the NAS? Virtualization, torrent downloading or anything else?
That prediction isn't necessarily accurate.[/quote]After 4 days, I still have about 10 days to go!!!!!
Nothing, I shut off all applications before I started the migration. Bare bones services running.
_______________________________________________________________________
Checkout http://www.digitalgatehouse.com for more of my ramblings!
Checkout http://www.digitalgatehouse.com for more of my ramblings!
- avvidme
- Know my way around
- Posts: 185
- Joined: Fri Jan 16, 2009 10:36 am
- Contact:
Re: [HOWTO] How to increase raid rebuild speed
No missing disk, it's an online spare.Spider99 wrote:missing disk is affecting rebuild/migration?
J
_______________________________________________________________________
Checkout http://www.digitalgatehouse.com for more of my ramblings!
Checkout http://www.digitalgatehouse.com for more of my ramblings!
-
- Guru
- Posts: 13192
- Joined: Sat Dec 29, 2007 1:39 am
- Location: Stockholm, Sweden (UTC+01:00)
Re: [HOWTO] How to increase raid rebuild speed
In the manual for the older cat1 models you can find this:avvidme wrote:No missing disk, it's an online spare.
Note: A hot spare drive must be removed from the disk volume before executing the following action:
• Online RAID capacity expansion
• Online RAID level migration
• Adding a hard drive member to a RAID 5, RAID 6 or RAID 10 volume
I would have expected the same to apply to cat2 as well but I don't know.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!
A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.
All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.
All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
- avvidme
- Know my way around
- Posts: 185
- Joined: Fri Jan 16, 2009 10:36 am
- Contact:
Re: [HOWTO] How to increase raid rebuild speed
Hi, sorry I should've been more specific - the drive is physically in the slot, but it's NOT configured for anything yet. It'll become an online spare once the RAID 6 migration is done.P3R wrote:In the manual for the older cat1 models you can find this:avvidme wrote:No missing disk, it's an online spare.
Note: A hot spare drive must be removed from the disk volume before executing the following action:
• Online RAID capacity expansion
• Online RAID level migration
• Adding a hard drive member to a RAID 5, RAID 6 or RAID 10 volume
I would have expected the same to apply to cat2 as well but I don't know.
I'm just blown away that I'm only getting 3100k from an 853. I even set the min/max to very high numbers (800000, etc) and no effect.
Just stumped here.
_______________________________________________________________________
Checkout http://www.digitalgatehouse.com for more of my ramblings!
Checkout http://www.digitalgatehouse.com for more of my ramblings!
-
- Guru
- Posts: 13192
- Joined: Sat Dec 29, 2007 1:39 am
- Location: Stockholm, Sweden (UTC+01:00)
Re: [HOWTO] How to increase raid rebuild speed
If a car doesn't have the performance to reach the speed limit, a raised limit won't make it go faster. The question is why your 853 moves at Trabant speed?avvidme wrote:I even set the min/max to very high numbers (800000, etc) and no effect.
A bad disk slowing things down maybe? You could check the SMART data on all disks to see if that gives you any hint.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!
A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.
All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.
All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
- avvidme
- Know my way around
- Posts: 185
- Joined: Fri Jan 16, 2009 10:36 am
- Contact:
Re: [HOWTO] How to increase raid rebuild speed
Smart status looks perfect across all of them. They are the WD RED 4TB WD40EFRX drives, SATA3. Normal NAS performance (read/write) has been just fine under RAID 5.P3R wrote: A bad disk slowing things down maybe? You could check the SMART data on all disks to see if that gives you any hint.
But I decided to migrate to 6 just for better data protection. It's been 5 days running non-stop so far, I'm starting to believe it'll be another 9 days as estimated.
I guess there's no other speed adjustments outside of min/max/cache?
Thanks again
J
_______________________________________________________________________
Checkout http://www.digitalgatehouse.com for more of my ramblings!
Checkout http://www.digitalgatehouse.com for more of my ramblings!
- avvidme
- Know my way around
- Posts: 185
- Joined: Fri Jan 16, 2009 10:36 am
- Contact:
Re: [HOWTO] How to increase raid rebuild speed
Unfortunately, I have bitmap on right now which I'm sure isn't helping. Is it possible to turn it off WHILE the migration is running?
Doesn't seem to be an option in the GUI right now since a number of menu items aren't available during migration. If it's safe to do, is there a command line option?
Thanks again
Doesn't seem to be an option in the GUI right now since a number of menu items aren't available during migration. If it's safe to do, is there a command line option?
Thanks again
_______________________________________________________________________
Checkout http://www.digitalgatehouse.com for more of my ramblings!
Checkout http://www.digitalgatehouse.com for more of my ramblings!
- Spider99
- Experience counts
- Posts: 1951
- Joined: Fri Oct 21, 2011 11:14 pm
- Location: UK
Re: [HOWTO] How to increase raid rebuild speed
you are casting about in the dark - bitmap helps in raid rebuild
you have a bad disk or some other config issue thats slowing the reshape - if your array was "full" before the start that will have a detrimental effect
when i built my 8 disk raid 5 - new no data it took a day and a half on my 853 to sync - with speeding it up - you are reshaping - a lot more complicated and significant change and hence slower process - actually i have a hunch the speed up options have little or no effect on a reshape
I also suspect once it gets past 50% and /or the data it will speed up a lot
you have a bad disk or some other config issue thats slowing the reshape - if your array was "full" before the start that will have a detrimental effect
when i built my 8 disk raid 5 - new no data it took a day and a half on my 853 to sync - with speeding it up - you are reshaping - a lot more complicated and significant change and hence slower process - actually i have a hunch the speed up options have little or no effect on a reshape
I also suspect once it gets past 50% and /or the data it will speed up a lot
Tim
TS-853A(16GB): - 4.3.4.0483 - Static volume - Raid5 - 8 x 4TB HGST Deskstar NAS
Windows Server + StableBit Drivepool and Scanner ~115 TB Backup Server
TS-412 & TS-459 Pro II: Retired
Clients: 3 x Windows 10 Pro(64bit)
TS-853A(16GB): - 4.3.4.0483 - Static volume - Raid5 - 8 x 4TB HGST Deskstar NAS
Windows Server + StableBit Drivepool and Scanner ~115 TB Backup Server
TS-412 & TS-459 Pro II: Retired
Clients: 3 x Windows 10 Pro(64bit)
-
- Know my way around
- Posts: 224
- Joined: Thu Nov 26, 2009 4:28 am
Re: [HOWTO] How to increase raid rebuild speed
Code: Select all
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] [multipath]
md1 : active raid5 sdd3[11] sdc3[10] sdb3[9] sda3[8] sdf3[7] sde3[6]
39020358080 blocks super 1.0 level 5, 64k chunk, algorithm 2 [6/6] [UUUUUU]
[============>........] resync = 60.7% (4741593912/7804071616) finish=602.8min speed=84663K/sec
md256 : active raid1 sdd2[7](S) sdc2[6] sdb2[2] sda2[3](S) sdf2[4](S) sde2[5](S)
530112 blocks super 1.0 [2/2] [UU]
bitmap: 0/1 pages [0KB], 65536KB chunk
md13 : active raid1 sdd4[29] sdc4[28] sdb4[27] sda4[26] sdf4[25] sde4[24]
458880 blocks super 1.0 [24/6] [UUUUUU__________________]
bitmap: 1/1 pages [4KB], 65536KB chunk
md9 : active raid1 sdd1[29] sdc1[28] sdb1[27] sda1[26] sdf1[25] sde1[24]
530112 blocks super 1.0 [24/6] [UUUUUU__________________]
bitmap: 1/1 pages [4KB], 65536KB chunk
TVS-672N h5.0.1.2248 build 20221215
i7-8700T | 64GB DDR4 (2 x M471A4G43MB1-CTD) | 2 x Samsung 970 Evo Plus 1.0TB | 6 x 18TB HGST HC550
- avvidme
- Know my way around
- Posts: 185
- Joined: Fri Jan 16, 2009 10:36 am
- Contact:
Re: [HOWTO] How to increase raid rebuild speed
I wasn't just expanding the raid, I was also converting from Raid 5 to 6.chodaboy19 wrote:Upgrading from 6 x 4TB to 6 x 8TB, I set the min to 85000 and it's working nicely!
J
_______________________________________________________________________
Checkout http://www.digitalgatehouse.com for more of my ramblings!
Checkout http://www.digitalgatehouse.com for more of my ramblings!