RAID Recovery failed: Not enough devices to start the array

Questions about SNMP, Power, System, Logs, disk, & RAID.
Post Reply
ochropyra
New here
Posts: 4
Joined: Fri Apr 29, 2011 3:40 pm

RAID Recovery failed: Not enough devices to start the array

Post by ochropyra »

My TS-410 originally had two WD20EARS in a Raid 1. (Back then the Disk were still on the recommended list)

I recently added two additional WD20EARS (new hardware revision apparently) and migrated to a Raid 5.
Soon after the Migration finished successfully one of the harddrives went into a S.M.A.R.T. Status "Abnormal".
Since that usually means that the harddisk will fail soon, I replaced it.

However the RAID refuses to rebuild. If i try to recover it i get an error message:

Code: Select all

[RAID5 Disk Volume: Drive 1 2 3 4] RAID Recovery failed: Not enough devices to start the array.
I did a full SMART and Bad Sector check on the Harddisk that showed no errors.
Searching the forums I only found one apparently similar case. However without a solution other than "QNAP Support did some magic trick" and 'apparently the disk had data on it'

Anyone got a decent idea how to fix my Raid again? (Hopefully without loosing my Data)

regards

Peter

Additional Info:

Code: Select all

[~] # cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4] [linear] [raid0] [raid10]
md0 : inactive sda3[0] sdb3[4](S) sdc3[1]
      5855836800 blocks

md4 : active raid1 sdd2[2](S) sdc2[3](S) sdb2[1] sda2[0]
      530048 blocks [2/2] [UU]

md13 : active raid1 sda4[0] sdd4[3] sdc4[2] sdb4[1]
      458880 blocks [4/4] [UUUU]
      bitmap: 0/57 pages [0KB], 4KB chunk

md9 : active raid1 sda1[0] sdd1[3] sdc1[2] sdb1[1]
      530048 blocks [4/4] [UUUU]
      bitmap: 3/65 pages [12KB], 4KB chunk

unused devices: <none>
[~] #

Code: Select all

dmesg
tered for level 6
md: raid5 personality registered for level 5
md: raid4 personality registered for level 4
TCP cubic registered
NET: Registered protocol family 17
rtc kw_rtc: setting system clock to 2011-04-28 19:34:28 UTC (1304019268)
usb 1-1: new high speed USB device using ehci_marvell and address 2
sdd1 sdd2 sdd3 sdd4
sd 5:0:0:0: [sdd] Sector size 0 reported, assuming 512.
sda1 sda2 sda3 sda4
sd 5:0:0:0: [sdd] Attached SCSI disk
sdb1 sdb2 sdb3 sdb4
sd 2:0:0:0: [sda] Sector size 0 reported, assuming 512.
sd 3:0:0:0: [sdb] Sector size 0 reported, assuming 512.
sd 2:0:0:0: [sda] Attached SCSI disk
sd 3:0:0:0: [sdb] Attached SCSI disk
md: Waiting for all devices to be available before autodetect
md: If you don't use raid, use raid=noautodetect
md: Autodetecting RAID arrays.
md: Scanned 0 and added 0 devices.
md: autorun ...
md: ... autorun DONE.
RAMDISK: gzip image found at block 0
hub 1-1:1.0: USB hub found
hub 1-1:1.0: 4 ports detected
VFS: Mounted root (ext2 filesystem) on device 1:0.
Freeing init memory: 136K
kjournald starting.  Commit interval 5 seconds
EXT3-fs (sda1): mounted filesystem with writeback data mode
kjournald starting.  Commit interval 5 seconds
EXT3-fs (sdb1): mounted filesystem with writeback data mode
kjournald starting.  Commit interval 5 seconds
EXT3-fs (sdc1): mounted filesystem with writeback data mode
kjournald starting.  Commit interval 5 seconds
EXT3-fs (sdd1): mounted filesystem with writeback data mode
md: md9 stopped.
md: bind<sdb1>
md: bind<sdc1>
md: bind<sdd1>
md: bind<sda1>
raid1: raid set md9 active with 4 out of 4 mirrors
md9: bitmap initialized from disk: read 5/5 pages, set 2 bits
created bitmap (65 pages) for device md9
md9: detected capacity change from 0 to 542769152
md9: unknown partition table
kjournald starting.  Commit interval 5 seconds
EXT3-fs (md9): using internal journal
EXT3-fs (md9): mounted filesystem with writeback data mode
md: md13 stopped.
md: bind<sdb4>
md: bind<sdc4>
md: bind<sdd4>
md: bind<sda4>
raid1: raid set md13 active with 4 out of 4 mirrors
md13: bitmap initialized from disk: read 4/4 pages, set 2 bits
created bitmap (57 pages) for device md13
md13: detected capacity change from 0 to 469893120
md13: unknown partition table
kjournald starting.  Commit interval 5 seconds
EXT3-fs (md9): using internal journal
EXT3-fs (md9): mounted filesystem with writeback data mode
kjournald starting.  Commit interval 5 seconds
EXT3-fs (md9): using internal journal
EXT3-fs (md9): mounted filesystem with writeback data mode
kjournald starting.  Commit interval 5 seconds
EXT3-fs (md13): using internal journal
EXT3-fs (md13): mounted filesystem with writeback data mode
fuse init (API version 7.13)
NET: Registered protocol family 5
md: linear personality registered for level -1
md: raid0 personality registered for level 0
md: raid10 personality registered for level 10
device-mapper: ioctl: 4.16.0-ioctl (2009-11-05) initialised: dm-devel@redhat.com
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
usbcore: registered new interface driver usblp
sysinfo: Module loaded.
ufsd: module license 'Commercial product' taints kernel.
Disabling lock debugging due to kernel taint
ufsd: driver U82 (Mar 31 2011 02:47:23)  LBD=OFF with ioctl loaded at bf18a000
NTFS read/write support included
Hfs+/HfsX read/write support included
Slow work thread pool: Starting up
Slow work thread pool: Ready
lib80211: common routines for IEEE802.11 drivers
lib80211_crypt: registered algorithm 'NULL'
cfg80211: Calling CRDA to update world regulatory domain
Enable WDT...
usbcore: registered new interface driver snd-usb-audio
usbcore: registered new interface driver snd-usb-caiaq
eth0: link down
eth0: started
eth1: started
eth0: stopped
eth0: mac address changed
eth0: started
eth1: stopped
eth1: mac address changed
eth1: started
kjournald starting.  Commit interval 5 seconds
EXT3-fs (md9): using internal journal
EXT3-fs (md9): mounted filesystem with writeback data mode
eth0: link up, full duplex, speed 1 Gbps
md: bind<sda2>
raid1: raid set md4 active with 1 out of 1 mirrors
md4: detected capacity change from 0 to 542769152
md4: unknown partition table
Adding 530040k swap on /dev/md4.  Priority:-1 extents:1 across:530040k 
md: bind<sdb2>
RAID1 conf printout:
--- wd:1 rd:2
disk 0, wo:0, o:1, dev:sda2
disk 1, wo:1, o:1, dev:sdb2
md: recovery of RAID array md4
md: minimum _guaranteed_  speed: 5000 KB/sec/disk.
md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for recovery.
md: using 128k window, over a total of 530048 blocks.
md: bind<sdc2>
md: bind<sdd2>
active port 0 :139
active port 1 :445
active port 2 :20
md: md4: recovery done.
RAID1 conf printout:
--- wd:2 rd:2
disk 0, wo:0, o:1, dev:sda2
disk 1, wo:0, o:1, dev:sdb2
md: md0 stopped.
md: md0 stopped.
md: md0 stopped.
md: md0 stopped.
md: bind<sdc3>
md: bind<sdb3>
md: bind<sda3>
raid5: device sda3 operational as raid disk 0
raid5: device sdc3 operational as raid disk 1
raid5: allocated 67488kB for md0
0: w=1 pa=0 pr=4 m=1 a=2 r=4 op1=0 op2=0
1: w=2 pa=0 pr=4 m=1 a=2 r=4 op1=0 op2=0
raid5: not enough operational devices for md0 (2/4 failed)
RAID5 conf printout:
--- rd:4 wd:2
disk 0, o:1, dev:sda3
disk 1, o:1, dev:sdc3
raid5: failed to run raid set md0
md: pers->run() failed ...
md: md0 stopped.
md: unbind<sda3>
md: export_rdev(sda3)
md: unbind<sdb3>
md: export_rdev(sdb3)
md: unbind<sdc3>
md: export_rdev(sdc3)
md: md0 stopped.
md: md0 stopped.
md: bind<sdc3>
md: bind<sdb3>
md: bind<sda3>
raid5: device sda3 operational as raid disk 0
raid5: device sdc3 operational as raid disk 1
raid5: allocated 67488kB for md0
0: w=1 pa=0 pr=4 m=1 a=2 r=4 op1=0 op2=0
1: w=2 pa=0 pr=4 m=1 a=2 r=4 op1=0 op2=0
raid5: not enough operational devices for md0 (2/4 failed)
RAID5 conf printout:
--- rd:4 wd:2
disk 0, o:1, dev:sda3
disk 1, o:1, dev:sdc3
raid5: failed to run raid set md0
md: pers->run() failed ...
eth1: stopped
eth0: stopped
eth1: started
eth0: link down
eth0: started
eth0: link up, full duplex, speed 1 Gbps
eth1: stopped
eth1: started
eth0: link down
eth0: stopped
eth0: started
eth0: link up, full duplex, speed 1 Gbps
active port 0 :139
active port 1 :445
active port 2 :20
rule type=2, num=0
Loading iSCSI transport class v2.0-871.
iscsi: registered transport (tcp)
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=1
rule type=2, num=0
active port 0 :139
active port 1 :445
active port 2 :20
Wake up disk 0 immediately.
Wake up disk 3 immediately.
Wake up disk 2 in 9 seconds.
Wake up disk 1 in 9 seconds.
Wake up disk 2 immediately.
Wake up disk 0 immediately.
Wake up disk 1 in 9 seconds.
Wake up disk 0 immediately.
Wake up disk 2 immediately.
Wake up disk 1 in 9 seconds.
Wake up disk 0 immediately.
Wake up disk 3 immediately.
Wake up disk 2 in 9 seconds.
Wake up disk 1 in 9 seconds.
Wake up disk 0 immediately.
Wake up disk 3 immediately.
Wake up disk 2 in 9 seconds.
Wake up disk 1 in 9 seconds.
[~] # fdisk -l

Disk /dev/sda: 2000.3 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          66      530141   83  Linux
/dev/sda2              67         132      530142   82  Linux swap / Solaris
/dev/sda3             133      243138  1951945693   83  Linux
/dev/sda4          243139      243200      498012   83  Linux

Disk /dev/sda4: 469 MB, 469893120 bytes
2 heads, 4 sectors/track, 114720 cylinders
Units = cylinders of 8 * 512 = 4096 bytes

Disk /dev/sda4 doesn't contain a valid partition table

Disk /dev/sdb: 2000.3 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1          66      530141   83  Linux
/dev/sdb2              67         132      530142   82  Linux swap / Solaris
/dev/sdb3             133      243138  1951945693   83  Linux
/dev/sdb4          243139      243200      498012   83  Linux

Disk /dev/sdd: 2000.3 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1               1          66      530125   83  Linux
/dev/sdd2              67         132      530142   83  Linux
/dev/sdd3             133      243138  1951945693   83  Linux
/dev/sdd4          243139      243200      498012   83  Linux

Disk /dev/sdc: 2000.3 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1          66      530125   83  Linux
/dev/sdc2              67         132      530142   83  Linux
/dev/sdc3             133      243138  1951945693   83  Linux
/dev/sdc4          243139      243200      498012   83  Linux

Disk /dev/md9: 542 MB, 542769152 bytes
2 heads, 4 sectors/track, 132512 cylinders
Units = cylinders of 8 * 512 = 4096 bytes

Disk /dev/md9 doesn't contain a valid partition table

Disk /dev/md4: 542 MB, 542769152 bytes
2 heads, 4 sectors/track, 132512 cylinders
Units = cylinders of 8 * 512 = 4096 bytes

Disk /dev/md4 doesn't contain a valid partition table
[~] # exit
logout
ochropyra
New here
Posts: 4
Joined: Fri Apr 29, 2011 3:40 pm

Re: RAID Recovery failed: Not enough devices to start the ar

Post by ochropyra »

Anyone?
userx
New here
Posts: 5
Joined: Thu Mar 31, 2011 5:15 pm

Re: RAID Recovery failed: Not enough devices to start the ar

Post by userx »

I have the same problem. Did you solve it?
ochropyra
New here
Posts: 4
Joined: Fri Apr 29, 2011 3:40 pm

Re: RAID Recovery failed: Not enough devices to start the ar

Post by ochropyra »

My Solution was restoring a two month old partial Backup.
Thankfully the data wasn't particularly valuable apart fron reripping most of my DVDs.
The main reason for my problem was that the UI reported disk4 as faulty but the software raid expected disk3 as faulty.

On a different note for me it was a huge susses to switch back to two raid1.
MUCH better performance.
userx
New here
Posts: 5
Joined: Thu Mar 31, 2011 5:15 pm

Re: RAID Recovery failed: Not enough devices to start the ar

Post by userx »

Great...
That is definitely not an option for me since I dont have any backup... :|
ochropyra
New here
Posts: 4
Joined: Fri Apr 29, 2011 3:40 pm

Re: RAID Recovery failed: Not enough devices to start the ar

Post by ochropyra »

You could try attaching the hard drives to a Linux box

And then use mdadm (man mdadm) to scan all drives and attempt to rebuild the array.
(expect waittimes of days rather than hours)
Fairly good chance of getting it working.
( however I'd pull a bitcopy before trying anything if the data is of any value. "Man dd")

Doing these on the qnap will be a pain in the **. Linux box highly recommended.
Post Reply

Return to “System & Disk Volume Management”