TS-410U Disk crashed, no longer access, HELP...

Discussion on remote replication.
Post Reply
Nick57
Starting out
Posts: 31
Joined: Mon Jul 05, 2010 12:12 am

TS-410U Disk crashed, no longer access, HELP...

Post by Nick57 »

Hi,

My brand new TS-410U died only after 5 days of working as a backup server ( nightly backups )
For sure 1 Disk is 'kaput' now I need to know the correct way to get my backups of this system.

-) Is it possible to recover data from the other 3 disk in the mean time while waiting for the new disk, is so how?
I just can't access the system, not with the browser or with any other means.

-) System is configured as raid 6, how to proceed in correct way to get the system back in working state when spare disk arrives?

Is it possible to use any other disk with less space, in mean time to rebuild system?
Will the system downgrade or will it break even more when I put in a smaller disk, like a 500GB or 1TB?

Thanks for any help.
TS-410U firmware 3.7.1 Build 20120615
4 x WD2002FYPS 2TB, RAID 5, 3 drives + 1 Spare.
User avatar
schumaku
Guru
Posts: 43579
Joined: Mon Jan 21, 2008 4:41 pm
Location: Kloten (Zurich), Switzerland -- Skype: schumaku
Contact:

Re: TS-410U Disk crashed, no longer access, HELP...

Post by schumaku »

Hoi Nick,

A loss of a single HDD in a RAID6 (or RAID5) should not break the RAID. With a four HDD RAID6, only two disks are required to hold the data.

How do you know one disk is broken?

What is the exact status of your NAS now?

You should be able to continue the NAS operations with just three healthy disks, and plug the new disk once replaced. Therefore, there is no need for an "emergency" replacement - which cannot be a smaller HDD anyway.

-Kurt.
Nick57
Starting out
Posts: 31
Joined: Mon Jul 05, 2010 12:12 am

Re: TS-410U Disk crashed, no longer access, HELP...

Post by Nick57 »

Hi Kurt,

Thanks, I had to collect the NAS from the DC, I couldn't access it all , the backups on the servers and te zen servers were still trying to connect to it of course. That and the drive failure ( or did the drive fail because of the many connections ) made it just impossible to do anything. At home I found out that i couldn't do anything with it as well, not even a reset of 3 seconds or the famous 10 seconds, nothing helped. Turning it off, taking drives out, power on again and putting drives back one by one made it clear that drive 2 was broke, the green led was flashing continues for nearly 2 days, First I thought it was trying to rebuild the raid, but after 2 days i gave up on that idea, 51 hours later this should have been done, right? drives are 2TB each.

I took the drive out and tried it in one of my PCs, no response!

So, at the moment the situation is that I think when my replacement disk comes it will rebuild the raid6 set and I will be able to access my data? There is nothing else at the moment I can do, I notice there was an update for the bios v3.3.1 so I hope it will make it more reliable... because i had problems to create the nfs iso share for the zen servers.

I have been reading up about disaster recovery, but nothing in there for my TS-410.... :-(

Status: it is turned off here at my desk and I feel hopeless... is there anything I can get it back with only 3 drives in it?

I can't believe this, I Just turned it on and noticed that it comes back and I can access it ( with only 3 drives in it) is that because it has been turned of for at least 16 hours or longer?
TS-410U firmware 3.7.1 Build 20120615
4 x WD2002FYPS 2TB, RAID 5, 3 drives + 1 Spare.
User avatar
schumaku
Guru
Posts: 43579
Joined: Mon Jan 21, 2008 4:41 pm
Location: Kloten (Zurich), Switzerland -- Skype: schumaku
Contact:

Re: TS-410U Disk crashed, no longer access, HELP...

Post by schumaku »

Nick57 wrote:Status: it is turned off here at my desk and I feel hopeless... is there anything I can get it back with only 3 drives in it? I can't believe this, I Just turned it on and noticed that it comes back and I can access it ( with only 3 drives in it) ...
This is the way it is supposed to be, as mentioned - only two healthy (from your total four) drives are required to bring the RAID6. Downtime and some sleep is not required. Is your Data still intact?

Might happen, a broken HDD prohibit from a correct operations continuity or a reboot, and must be removed from its slot. The price of the inexpensive SATA technology I'm afraid.
Nick57
Starting out
Posts: 31
Joined: Mon Jul 05, 2010 12:12 am

Re: TS-410U Disk crashed, no longer access, HELP...

Post by Nick57 »

Kurt, good news, I am able to connect to the nas with my laptop and just to be sure I am retrieving some important data from it.

I am totally surprised about this, as I don't understand a few things, because it is SATA it is 'normal' that it halts completely?
So it stopped abrupt after the harddisk broke and that is down to SATA behaviour??

It doesn't click in my brains :-) raid 6, drive broke, system halted, a drive in the car turned for an hour, reset, nothing helped, only disk2 green led flashing... Then after being turned of after so many hours it comes back, it smiles at me and says what you want, I am still alive...

Phwww

Any advice for what i can do after I have my most important data?
Shall I update the firmware to the latest version, or wait till replacement drive comes?
Do you think I should try to put the 'broke' disk back and see if it was in the middle of a scan or rebuild?

2010-07-31 10:59 Drive 2 might have bad blocks. Please do a "Bad Blocks Scan" on the drive.
2010-07-31 10:57 [Raid6 Disk Volume: Drive 1 2 3 4] Recovering journal.
TS-410U firmware 3.7.1 Build 20120615
4 x WD2002FYPS 2TB, RAID 5, 3 drives + 1 Spare.
User avatar
schumaku
Guru
Posts: 43579
Joined: Mon Jan 21, 2008 4:41 pm
Location: Kloten (Zurich), Switzerland -- Skype: schumaku
Contact:

Re: TS-410U Disk crashed, no longer access, HELP...

Post by schumaku »

Have seen defective SATA drives "hung" and stop all operations on the same controller several times, but never with the electrically very similar SAS attached drives. Duonot ask me, why...I have lost almost all my hair already.

If you want to experiment, I'd suggest this:

1. Connect the potentially defective drive to a different system, and erase all partitions.
2. Wait until the RAID6 is rebuilt, after the "hard" shutdown certainly required.
3. Hot-plug the suspect defective unit back to the internal slot, and see...

The drive will be re-partitioned again, the main data partition integrated again in the RAID6.

Take the opportunity to take copies of your backup files; this will create some I/O operations on the NAS. Later, create a dummy test share, and put some load to the system. Depending of the distance, you can consider to re-install it in the DC - and make another tour in case the drive goes mad again.
Nick57
Starting out
Posts: 31
Joined: Mon Jul 05, 2010 12:12 am

Re: TS-410U Disk crashed, no longer access, HELP...

Post by Nick57 »

Thanks very much for your help, much appreciated !

Disk is really broke, can't get it to work in any other system.
Need to wait till replacement comes, in mean time I have retrieved my backups.

Question, is the Firmware stored on the harddisks?
TS-410U firmware 3.7.1 Build 20120615
4 x WD2002FYPS 2TB, RAID 5, 3 drives + 1 Spare.
User avatar
schumaku
Guru
Posts: 43579
Joined: Mon Jan 21, 2008 4:41 pm
Location: Kloten (Zurich), Switzerland -- Skype: schumaku
Contact:

Re: TS-410U Disk crashed, no longer access, HELP...

Post by schumaku »

Depending on the models, the installation split between the firmware installed in a FLASH/DOM and on the HDD partitions is different. In any case, the partition table scheme is similar - this is required also for the new drive portability.

On the replacement of a single disk, the admin has not to take care about.
Nick57
Starting out
Posts: 31
Joined: Mon Jul 05, 2010 12:12 am

Re: TS-410U Disk crashed, no longer access, HELP...

Post by Nick57 »

Thanks, so it should be safe to update the firmware now, and put the new disk in later, right?

Is it better to put the new disk in while the system is up and running or is it better to power it off and put disk in?

Sorry to ask so much, and again many thanks for all your help, much appreciated :-))
TS-410U firmware 3.7.1 Build 20120615
4 x WD2002FYPS 2TB, RAID 5, 3 drives + 1 Spare.
User avatar
schumaku
Guru
Posts: 43579
Joined: Mon Jan 21, 2008 4:41 pm
Location: Kloten (Zurich), Switzerland -- Skype: schumaku
Contact:

Re: TS-410U Disk crashed, no longer access, HELP...

Post by schumaku »

Yes, update the firmware now (with the degraded RAID) or wait until it's back does not make a difference.

Overall, the risk of hot plugging in a fully operational NAS is certainly higher then cold plugging: In case anything does mad while hot plugging (i.e. power supply, ...), you have unhappy users, a crashed file system, RAID requires rebuild, applications require restarting, ... Under the line, it's more an operational question: 24*7 environments tend to take the risk.

Technically, it does not matter - the NAS will pick up the replacement disk and set in place accordingly.
Nick57
Starting out
Posts: 31
Joined: Mon Jul 05, 2010 12:12 am

Re: TS-410U Disk crashed, no longer access, HELP...

Post by Nick57 »

I have updated the firmware and it seems to be even a bit faster, I will turn it off when putting the disk in better safe then sorry.
Thank you ever so much for all your help.
TS-410U firmware 3.7.1 Build 20120615
4 x WD2002FYPS 2TB, RAID 5, 3 drives + 1 Spare.
Post Reply

Return to “Remote Replication/ Disaster Recovery”