TS-251+ dead, one RAID1 disk recovered

Discussion on remote replication.
Post Reply
ay200
New here
Posts: 7
Joined: Tue Apr 17, 2018 11:40 pm

TS-251+ dead, one RAID1 disk recovered

Post by ay200 »

Getting roped in to a bad situation, it seems.

(1) During a routine power-off to relocate the NAS, something happened. Not sure what. A power surge, maybe. Anyway, one drive in the RAID1 array is dead, and the TS-251+ refuses to power on further.

(2) The firmware is reported as being whatever comes with the TS-251+. No update.

(3) The external hard drive which was serving as backup was either misconfigured or caught up in the same event. No data can be recovered in that way.

(4) Connecting the drives via SATA to an independent machine revealed that one of the RAID1 drives was dead as a doornail. It refused to be recognized on multiple SATA ports with known good cables.

(5) Currently cloning the storage portion of the intact drive to a disk of identical size with DD. Its about 4TB, so it will be a little while.

(6) My MDADM-fu is basically limited to setting up new raid arrays and having it 'just work'. I'm cloning with DD because I have no idea what I'm doing and am properly terrified.

(7) MDADM cannot automatically mount raid arrays created by QNAP, apparently? It creates an MD for each RAID1 partition, but you cannot actually mount those.

(8) The people that own this thing have not properly budgeted for catastrophic data recovery. However, at this point, I'd be happy to pay someone to fix this problem for me.

So, if anyone here could help me answer three questions, I'd surely appreciate it:

(A) Can a single drive from a QNAP RAID1 created with the default firmware that comes with a TS-251+ be recovered with Linux?

(B) If mounting in Linux is not possible, what other solutions exist?

(C) Are hardware failures as described above (except the external HDD, which was probably misconfigured) a regular occurrence, and is there a subsequent NAS purchase I could make to mitigate those risks? Yes, I know that all of this wouldn't be a problem at all if the external HDD had been properly configured.

Thanks for your time, everyone!

Edit: And I -think- I've figured out why the system failed. Not a random manufacturing error. As I was shucking the surviving drive from its plastic cradle, I noticed one of the screws securing it was missing. I gave the dead NAS a little shake... and heard a rattle. :( Screw your stuff in tightish, people.
Last edited by ay200 on Fri Sep 07, 2018 10:21 pm, edited 1 time in total.
User avatar
dolbyman
Guru
Posts: 35249
Joined: Sat Feb 12, 2011 2:11 am
Location: Vancouver BC , Canada

Re: TS-251+ dead, one RAID1 disk recovered

Post by dolbyman »

stick the drive in a new qnap unit (check the migration guide) and it should work

mounting it in a linux machine is hard...there is not only the raid ..but also lvm on top of it
ay200
New here
Posts: 7
Joined: Tue Apr 17, 2018 11:40 pm

Re: TS-251+ dead, one RAID1 disk recovered

Post by ay200 »

Thanks for the reply, dolbyman!

Buying an identical NAS. Having it overnighted. Here's hoping this is resolved by tomorrow.

Assuming the migration guide you were referring to is either https://docs.qnap.com/nas/4.1/Home/en/i ... ration.htm or https://www.qnap.com/en/nas-migration/? ... nation=X51 , I'm forced to assume the x51+ series falls under x51 series entries?

Another possible problem is drive order. I only have the one drive, and this is Raid1, so I'm not sure if drive order is relevant. MDADM indicated the surviving drive was 0th in order, which I took to mean it would go in drive bay 1. Does that sound correct? More to the point, is there any risk to placing the surviving drive, by itself, in an unpowered, new TS-251+, and then powering it on? If so, I guess I'd need to pursue data recovery by other means, LVM notwithstanding (probably have to pay someone). Just super paranoid about writing anything to this drive that results in messed up raid or partition information.
User avatar
dolbyman
Guru
Posts: 35249
Joined: Sat Feb 12, 2011 2:11 am
Location: Vancouver BC , Canada

Re: TS-251+ dead, one RAID1 disk recovered

Post by dolbyman »

that manual is age old(F/W 4.1 .. and the wrong cat for the device chosen (251+ is CAT2 not CAT1/home)

here is the correct manual page for a current firmware
http://docs.qnap.com/nas/4.3/cat2/en/in ... ration.htm
User avatar
Don
Guru
Posts: 12289
Joined: Thu Jan 03, 2008 4:56 am
Location: Long Island, New York

Re: TS-251+ dead, one RAID1 disk recovered

Post by Don »

(2) The firmware is reported as being whatever comes with the TS-251+. No update.
This is hardly helpful. Know one knows what version of firmware came with your unit.
Use the forum search feature before posting.

Use RAID and external backups. RAID will protect you from disk failure, keep your system running, and data accessible while the disk is replaced, and the RAID rebuilt. Backups will allow you to recover data that is lost or corrupted, or from system failure. One does not replace the other.

NAS: TVS-882BR | F/W: 5.0.1.2346 | 40GB | 2 x 1TB M.2 SATA RAID 1 (System/VMs) | 3 x 1TB M.2 NMVe QM2-4P-384A RAID 5 (cache) | 5 x 14TB Exos HDD RAID 6 (Data) | 1 x Blu-ray
NAS: TVS-h674 | F/W: 5.0.1.2376 | 16GB | 3 x 18TB RAID 5
Apps: DNSMasq, PLEX, iDrive, QVPN, QLMS, MP3fs, HBS3, Entware, DLstation, VS, +
ay200
New here
Posts: 7
Joined: Tue Apr 17, 2018 11:40 pm

Re: TS-251+ dead, one RAID1 disk recovered

Post by ay200 »

If you know a way of figuring that out without being able to power on the device, I'd be happy to do that. Does information about QNAP NAS firmware show up in partition or RAID headers? The information MDADM would be able to return?
P3R
Guru
Posts: 13192
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: TS-251+ dead, one RAID1 disk recovered

Post by P3R »

The QTS version isn't all that important now. Install the disk in the new unit, power it up and hope for the best is probably your best option now.

Then when you have it running again, you can check the firmware version and after that find the correct documentation by looking in the right column on this page.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
ay200
New here
Posts: 7
Joined: Tue Apr 17, 2018 11:40 pm

Re: TS-251+ dead, one RAID1 disk recovered

Post by ay200 »

That's the plan.

Well. I'm cloning the surviving drive w/ dd. The cloned drive will inserted.
ay200
New here
Posts: 7
Joined: Tue Apr 17, 2018 11:40 pm

Re: TS-251+ dead, one RAID1 disk recovered

Post by ay200 »

Update time!

Got an identical NAS today. Inserted cloned drive, and got a message saying a drive with a QNAP signature was detected, but a factory reset was required. I did that, and it didn't work.

So I initialized the NAS on an SSD drive I had, and hooked the RAID1 drive up again. All of its partitions were recognized, but the big partitions with the relevant data were listed as being of an "unknown" type. Not great, but at least it works.

I was also able to determine that the old NAS, with which the RAID1 was created, was running firmware version 4.3.4.0597, which was released at about the same time the NAS was purchased.

So I update the new NAS to 4.3.4.0597, put in the drive, and... no instant win. As though I could be so lucky.

HOWEVER, the system appears to be recognizing a degraded RAID is involved:
Image

The cloned drive is recognized and the NAS appears to be assembling md of the correct size:
Image

And look! There's an unmount 'legacy' volume the exact correct size for the data!
Image
Image

I don't want to get their hopes up, but it seems like recovery is... well, a great deal more attainable than it was two days ago.

However, there are several clear problems. The first is that the NAS doesn't appear to be treating this as an actual degraded RAID. The manual indicates I should see some red-light blinking if there is a degraded RAID involved. That has yet to appear. Second, page 92 of the firmware manual indicates that the 'NAS Host' in the last picture should be identified as a RAID, and that the "Manage" button should be available when it is not.

I've noticed that maybe half a minute after the NAS loads and I can access the control panel through my web browser, it beeps, and at the same time a notification appears stating that "[RAID1 Disk Volume Host Drive: 1] Volume is unmounted."

Does anyone have any suggestions about how to get the RAID mounted now? is it time to run MDADM --assemble?

Edit: And by morning light, the next step seems obvious. I submitted a helpdesk ticket.
guki49
First post
Posts: 1
Joined: Sat Dec 15, 2018 10:38 pm

Re: TS-251+ dead, one RAID1 disk recovered

Post by guki49 »

This might help others who find themselves in a similar situation.

I just did a test to access the contents of one RAID1 disk in order to be prepared for a NAS harware desaster.

The RAID1 was configured as a static volume on two disks and one disk was taken from a switched-off TS-431P2 running QTS 4.3.4.0435.
By exactly following the instruction given in
http://blog.marcelbrouwers.nl/2016/04/2 ... on-ubuntu/
it is quite straight forward to connect the RAID1 disk taken from a TS-431P2 to a PC running linux (mine was Ubuntu 12.04.3).

After a few command lines at the PC console it is possible to get read access to the data files stored on the RAID1 disk in the largest lvm2 partition found on the disk.
Only at the mount command I got one error message "unknown file system linux_raid_member", but despite that it allowed data access.

So a failure of my NAS station hardware alone would have only a minor severity as long as at least one disk of the RAID1 will remain intact, this data can be recovered quickly.
A failure which affects all RAID disks would be more severe as it would require a time consuming restoration of several TB from the off-line backup media .
User avatar
dolbyman
Guru
Posts: 35249
Joined: Sat Feb 12, 2011 2:11 am
Location: Vancouver BC , Canada

Re: TS-251+ dead, one RAID1 disk recovered

Post by dolbyman »

it won't be that easy with thick or thin volumes..so always do external backups
P3R
Guru
Posts: 13192
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: TS-251+ dead, one RAID1 disk recovered

Post by P3R »

@guki49,
Thank you for a nice first post in the forum. It may help some users here but if data recovery is needed, it's a sign that a proper backup routine to an independent external system (which in it's simplest form can be a external USB-connected disk) haven't been in place. External backups are mandatory for data that have any value!
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
Post Reply

Return to “Remote Replication/ Disaster Recovery”