Trying to recover from a degraded RAID, SMART errors and problems rebuilding with new HDD

[steenbras]

A couple of days ago, 2 HDDs in my 4-disk RAID 5 configured TS-453A showed SMART errors, and the storage pool went into a read-only state. I was away for a few days and ordered some replacement disks and when I got back I thought it would be a simple job to swap out the problem disks with new ones and the RAID would rebuild.

Unfortunately, swapping out the disks didn't really do anything - actually only started with one (Disk 1) and the NAS picked it up, but didn't do any rebuilding. It showed me that the disk was a member of the pool, and nothing else. There was no option to add it as a spare disk (that was greyed out), so I tried a recover, and then it marked the disk as purple and said the slot was already part of a storage pool and that I should create another pool. I returned the faulty disk to the NAS and ran a recover there, hoping to try again.

Still, the same thing happened when I put the new disk in - picks it up, flags it as good, but does not rebuild the NAS. I happened on this forum post (viewtopic.php?t=142179) and tried - perhaps stupidly - the following command :

Code: Select all

"/etc/init.d/init_lvm.sh"

This gave the following output - along with a segmentation fault.

Code: Select all

/etc/init.d/init_lvm.sh
Changing old config name...
Reinitialing...
Detect disk(8, 48)...
dev_count ++ = 0Detect disk(8, 16)...
dev_count ++ = 1Detect disk(253, 0)...
ignore non-root enclosure disk(253, 0).
Detect disk(8, 64)...
ignore non-root enclosure disk(8, 64).
Detect disk(8, 32)...
dev_count ++ = 2Detect disk(8, 0)...
dev_count ++ = 3Detect disk(8, 48)...
Detect disk(8, 16)...
Detect disk(253, 0)...
ignore non-root enclosure disk(253, 0).
Detect disk(8, 64)...
ignore non-root enclosure disk(8, 64).
Detect disk(8, 32)...
Detect disk(8, 0)...
sys_startup_p2:got called count = -1
/etc/init.d/init_lvm.sh: line 12: 22972 Segmentation fault      /sbin/storage_util --sys_startup_p2
Done

I then panicked, and tried a reboot from the gui. The NAS didn't reboot, but seems to have reset itself, I can no longer log in with the admin account and my password - I presume it's gone back to some default pwd but I can't login with admin/admin and I'm now terrified I've lost everything. Is there anything I can do to try to recover? Any help from wiser souls gladly received...

[dolbyman]

Raid5 can only survive one busted disk..you have two..you need to start from scratch and restore from backups

[steenbras]

The way I read the warnings on the disk was that they had been put into read-only mode to prevent an issue in case the disks failed completely - is there any way that I can at least attempt recovery?

[dolbyman]

depends on the smart errors

read
write
etc

if you have no backups but really important data..I would start looking for a recovery company (and stop fiddling around to not make it worse)

re-initializeing of the LVM is done if you have errors/problems of the LVM layer..you have issues with the underlying block storage ..so that was a big mistake

[dosborne]

If you still have access to the data, i.e. the share is still online but in read-only mode, copy it to a network location or USB drive or *SOMETHING* ASAP. Don't be farting around with commands, replacing drives, playing with the configuration etc. Get a backup NOW!

[steenbras]

Thanks for the contributions all - sounds like I'm up the proverbial without a paddle

Any suggestions on how to go down the recovery route? I contacted one firm via a search and had a call with them and as soon as I mentioned 4 x 10TB configuration they basically said it was economically unfeasible - £800 investigation and around £3000-4000 recovery. Does that sound correct?

I know this is my own fault - I should realise a NAS is not a backup, and I shouldn't have done the lvm init. But I have some files with massive sentimental value on there and losing them would be, well, not good.

[dolbyman]

well the other option would be not much cheaper

- Do dd copies of all drives to new ones (if you can copy them)
- Assemble the RAID with the new drives(you might need special tools)
- Read the threads about how to assamble the LVM that the NAS uses in Linux (not easy...even more so if it is already messed up)
- Copy off the data to yet another destination that can hold that much data

At this point the cost of that recovery company might not be too bad anymore

[ritters]

... and as soon as I mentioned 4 x 10TB configuration they basically said it was economically unfeasible - £800 investigation and around £3000-4000 recovery. Does that sound correct?

Yes, sounds quite ok, it's in the normal rates for such services.

[steenbras]

I had to post an update here - I tried to find a service to help me recover this data, and a Google Search found me a company that appeared to have amazing TrustPilot reviews. So I sent my device off to them and then discovered other posts online telling quite worrying tales about that company running scams and exorbitant recovery charges.

Sure enough, after they received my NAS, I was contacted and told that the recovery would be simple and would "only" cost around £1500. But hours later they contacted me to claim the drives were really struggling, that the recovery was going to be problematic, and that the opportunity for recovery was disappearing. And that the cost would be much higher than initially quoted for. I took the opportunity to bail and had the NAS returned.

I then found FreeDataRecovery. He was able to remotely connect and reassemble the RAID. Super impressed. Thank you Stephen - I have my lost data back. And at a very reasonable price.

[soficha]

Hello steenbras, reading your thread and having as well issues with my raid 5. I am looking for a data rescue specialist. Could you tell me more about FreeDataRecovery please? Is it a company or?
Many thanks

[steenbras]

Hi soficha - all I can suggest is get in touch with Stephen (the link is in my post)- he was able to help me.

[kingbacana]

paste the link again, its not in ur post, I dont see it I only see FreeDataRecovery. Soficha is right.

[PeterT1959]

Try https://www.freedatarecovery.us/

Sent from my Pixel 4a using Tapatalk

[soficha]

Many thanks!