saitou187 wrote:Now, the question is why so long for the expansion on such small drives? If i were to build a raid5 array from scratch using those 4 drives it would have taken maybe 2 hours at most. Any insight would be greatly appreciated
As I don't know the inner workings for the RAID, I can only speculate, but I can add some timings of my own ;
Started with a Single Disk, same size - In fact all disks are the same (Seagate ST3250310AS 3.AA - 232.89 GB)
Performed basic installation on single disk, configured NAS and versious services, set up defaults shares etc.
Step 1 - Expanded to 3 disk RAID 5 - Total Time 2 Hours;
This was performed in 2 steps, preparing the new disks took about 30 minutes, expanding the raid about 90 minutes more. The interesting thing is that this was shown as one operation, so the %age advanced faster for the first 50% (30 minutes) than the second 50% (90 minutes).
Tip: Progress % seems to represent activity rather than time
Added data to the disk to to fill about half of the now available 450Gig - this was at the maximum network speed so only indicated the NAS can match my network.
Step 2 - Expanded 3 Disk RAID 5 to 5 Disk Raid 5 - Total Time 40 Hours;
Again the % was deceptive, for the first 24 hours or so it maintained 1% per hour and only three drives were running in sync.
Sometime after this - I do sleep you know - the fourth drive started to run in sync and the % started to advance faster than the hours. 36 hours in it was at 50%. The 40 hours total was based on the logs as it finished when I was at work.
So to summarise;
Single disk to 3 disk RAID 5 (minimum data) - 2 hours
3 Disk RAID 5 to 5 Disk RAID 5 (half capacity) - 40 hours
As to why? Safety!
RAID has one purpose - to protect your data in the event of drive problems - so any RAID operation has to follow this principal with the possibility that it could happen at any time, and at any point in the operations.
Migrating from single disk to 3 disk RAID 5
-------------------------------------------
Single disk has no protection so the emphasis of this operation is to protect the data quickly, the best way to do this would be to mirror it initially and calculate parity as a secondary step - Like I said this is an ideal scenario as I don't know the RAID system used here.
I'm not giving a lesson on RAID here so won't go into the fiddly about what's going on, but suffice to say for most of the operation it will be reading from 1 drive and writing to all three.
Every bit of the drive has to be processed irrespective of the amound of data, as even empty sectors need parity - though raid systems usually have shortcuts for this.
Migrating from 3 Disk to 5 Disk RAID 5
----------------------------------------
RAID operations don't get much more complex than this, this is probably the most time consuming operation as it involves a lot of work. The same is true for four, six or eight disks - redistribution of a Raid 5 set is Slooooow. But not as slow as RAID 6
Remember a drive failure at any time must be recoverable so before any data can be overwritten or erased from one drive, there must be a safe copy elsewhere.
Because we're adding drive, space is not the issue it's moving that space around that takes the time.
I would guess that the system maintains the 3 disk RAID until as late as possible in the process as the part built 5 DISK is not yet fully recoverable.
So all it has to do is read data from three disks, discard the old parity, recalculate the new parity and write it back to 5 disks right?
Well no, because it can't overwrite the source data so it has to put it somewhere else. The data will never be written back to the same part of the disks as it came from as a power failure during this operation would lose data. It can't put it on the new drives temporarily as this data would not offer the protection that RAID is supposed to guarantee.
So it can only put it on an empty part of the existing 3 disks.
The best way to picture this - though not the way it physically happens is think for the disks as cylinders (the typical RAID diagram), it squashes all the (3 Disk) data down to the bottom of the three disks and starts to build the 5 disk raid at the top.
Physically it's different matter as "squashing data down" really isn't feasible when it comes to hard disks.
In addition, the fact that my rebuild sync'd 4 disks first suggests a 2 step rebuild, but the speed of the 2nd step (10 hrs) compared to the first (30 hrs) suggests it didn't migrate via a 4 disk raid in the true sense but possibly some intemediate internal format.
And this is the reason for the time, it has to keep shuffling data around like a giant tetris game until everything is where it should be and at no time must there be unprotected data anywhere.
It's a bigger job than you might imagine.
But Wait, There's more
------------------------
The RAID migration is a big job to start with, but on a live NAS you have the added compication that the data is still in use.
The system isn't just having to migrate static data, it's migrating a live operational file system which could be changing.
So it has to calculate each read and write based on floating data.
That MP3 you're listening to could be migrating across raid sets as you listen
All in all it's doing a lot more work than most people realise.
