TS-639 - RAID5 Add Hard Drive Failed

[Chod]

Hi All,

Hoping someone could help.

I was in the process of adding my 6th and final Hard Drive to my Raid5 when at some point through the night it failed mysteriously. I assume it was unable to unmount, i did try to kill as many processes as i could. Before this I had successfully "added a hard drive" from 3 -> 4 and then 4 -> 5, but this final step 5 -> 6 failed.

Whats worse is that the system looks as if it has actually added the drive to the RAID5 but not showing the full capacity (since it failed at some point). So the RAID5 is healthy but only showing 5 drives worth of RAID5 capacity.

I am using dm001s across the board except for one drive which is being replaced under warranty. The new drive is a dm001.

Here are the logs and things:

2012-07-03
255
01:08:00 System 127.0.0.1 localhost [RAID5 Disk Volume: Drive 1 2 3 4 5 6] Expanding Raid Device failed.

df /dev/md0
Filesystem Size Used Available Use% Mounted on
/dev/md0 7.2T 3.6T 3.6T 50% /share/MD0_DATA

mdadm -D /dev/md0
/dev/md0:
Version : 00.90.03
Creation Time : Wed Mar 28 12:26:49 2012
Raid Level : raid5
Array Size : 9759727040 (9307.60 GiB 9993.96 GB)
Used Dev Size : 1951945408 (1861.52 GiB 1998.79 GB)
Raid Devices : 6
Total Devices : 6
Preferred Minor : 0
Persistence : Superblock is persistent

Update Time : Tue Jul 3 22:47:48 2012
State : clean
Active Devices : 6
Working Devices : 6
Failed Devices : 0
Spare Devices : 0

Layout : left-symmetric
Chunk Size : 64K

UUID : 5e2435c5:d023175c:9ac3b640:8ec069c1
Events : 0.2435579

Number Major Minor RaidDevice State
0 8 3 0 active sync /dev/sda3
1 8 35 1 active sync /dev/sdc3
2 8 19 2 active sync /dev/sdb3
3 8 51 3 active sync /dev/sdd3
4 8 67 4 active sync /dev/sde3
5 8 83 5 active sync /dev/sdf3

cat /etc/mtab
/proc /proc proc rw 0 0
none /dev/pts devpts rw,gid=5,mode=620 0 0
sysfs /sys sysfs rw 0 0
tmpfs /tmp tmpfs rw,size=64M 0 0
none /proc/bus/usb usbfs rw 0 0
/dev/sda4 /mnt/ext ext3 rw 0 0
/dev/md9 /mnt/HDA_ROOT ext3 rw 0 0
none /sys/kernel/config configfs rw 0 0
/dev/md0 /share/MD0_DATA ext4 rw,usrjquota=aquota.user,jqfmt=vfsv0,user_xattr,data=ordered,delalloc,noacl 0 0
tmpfs /.eaccelerator.tmp tmpfs rw,size=32M 0 0


Any suggestions are greatly appreciated! Should i shutdown and pull out the 6th Drive?

[Chod]

One further thing...

The log entry directly before the "failed expansion" was this:

2012-07-03
254
01:04:00 System 127.0.0.1 localhost [RAID5 Disk Volume: Drive 1 2 3 4 5 6] Rebuilding completed.

So it looks like it completed whatever it was doing, but had trouble with the resizing? How can i reattempt the resizing part?

[Chod]

Just wanted to post that i solved the issue.

It had successfully added the 6th drive, all i needed to do was to expand. You go to Raid management, click expand and it will automatically show you the size of the new volume and you hit go. It only took a few short hours to complete.