TS-451 Failing Drive Bay (Update / Possible Fix)

[RobbieWilkes]

Hello All,

My TS-451 (4 bay) started reporting drive errors on bay 3. Strangely, the drive itself was not reporting errors, but the NAS would just start randomly failing.

My setup (which I understand is NOT optimal) is as follows:

Bay 1 is configured as my DataVol (home, public, downloads, installs, etc.)
Bays 2 - 4 are spanned as my Media volume (as the name implies)

Unfortunately, I did not take advantage of any of the RAID, functions, opting instead for the JBOD idea for maximum space.

Anyway, I pulled the disk from drive 3, and tested externally, and got no failures, so that pointed me to the probability of an issue with the bay itself. To that end, I popped the DataVol drive out of bay 1 and then inserted the 3 drives, from my Media span, into bays 1, 2 & 4 (skipping the suspected bay 3). Sure enough, I just ran a test on the drive, previously in bay 3, and no errors.

Of course, the problem is, with the drives in 3 different bays, my Media volume is not being recognized. My hope is to try and pull all my media off, onto a large external USB drive, and then figure out where to go from there, but I'm not sure how to proceed.

So, is there a way to tell the NAS that the Media volume now sits in bays 1, 2 & 4, so that I can get my data backed off?

Thanks for any insight anyone can offer.

Robbie Wilkes

[dolbyman]

so you used a risky spanning jbod but did not care about backups..oooff

cat2 devices should not care about the disk position when disks are swapped while the NAS is off

you could also be seeing the onset of the lpc degradation issue (check the forum for plenty of topics about it)

[dosborne]

What firmware version are you running? There seem to be a few posts about false drive errors reported in the 5.0.0 stream.

[RobbieWilkes]

What firmware version are you running? There seem to be a few posts about false drive or nay errors reported in the 5.0.0 stream.

Thanks so much for your response. Firmware version is 5.0.0.1986.

I'll take a look for some of those threads.

[RobbieWilkes]

As a follow-up on this... I reached out to QNAP support on my issue, with my, almost, 7 year old TS-451. I was hoping they might be able to offer some suggestions on how I could re-join my 3 drives, in different bays (1,2 & 4), bypassing my failing 3rd bay. Honestly, I didn't think I would hear anything, since it wasn't under any type of warranty.

Instead, I was contacted by a QNAP tech named Paul, who remoted into my laptop via TeamViewer, connected to the NAS, booted into SSH without the hard drive option, and proceeded to issue magical strings of linux commands at lightning speed. After a good bit of trial and error, he was finally able to get my Media volume stitched back together.

He then tried copying to my external USB drive, but it failed. Undeterred, he had me connect my external drive to my laptop, downloaded WinSCP, connected to my NAS, and kicked off the copy of my files. Thankfully I paid enough attention to the steps involved, so that, after he disconnected, I was able to move the drive over to my HTPC, install WinSCP and resume the process, because it took over 3 days to complete the transfer.

I was absolutely blown away by how much time Paul spent assisting me to resolve a problem of my own making. He went above and beyond, and it is greatly appreciated.

Sadly, bay 3 in my TS-451 is still not working, confirmed by placing a brand new drive in the bay. When I first put it in, it saw it, and allowed me to kick off a test, but then started giving the same errors that it did on my other drive, and now doesn't see it at all.

I reached out to QNAP, to see if they would repair it, or send me the parts to do so (I know, I know, I've seen here that they won't, but took my shot), but, sadly, it's a no-go on both fronts.

To try and eliminate the possibility of an issue related to firmware (since my initial problem did occur VERY SOON after the last update), I downgraded the BIOS, to the last 4.? revision, but no luck. I've now updated to the latest 5.? version, and am in the same boat.

I also I pulled it completely apart, cleaned the contacts and reseated everything, but, other than good housekeeping, it was, otherwise, fruitless.

Anyone have any other suggestions, that might point to it NOT being an actual hardware problem?

Either way, given the high degree of customer service and the quality of the hardware, I am a big QNAP fan.

[RobbieWilkes]

Wanted to come back here and give another update, since things are actually looking up. Fingers crossed.

Since my last post, almost 6 months ago, my TS-451, formally the center of my household media universe, had been relegated to just a download box, from which I would then copy over to the external USB drive I used for backup. My plan, when I had time, was to pull it apart and do further investigation on the LPC degradation issue mentioned above, using the steps outlined HERE viewtopic.php?t=157459 though my aging eyes and failing dexterity in my dominant hand, doesn't really lend itself to those types of things anymore.

Don't get me wrong, several times between then and now, usually after applying updates, I have plugged the drives back in, hoping that it would be fixed, but nope... bay 3 was STILL reporting as failing. Last night was one of those occasions. After applying the most recent update, and restarting, I pushed in the drives in bays 2-4, and crossed my fingers. Nope. Bay 3 still reporting as bad.

This time, however, I decided to come back and read through some of the other threads, to see if I could find any other clues. Honestly, I still didn't believe it was a sudden hardware failure, because it started happening within 3 days of an update, after working FLAWLESSLY since 2017, and avoiding updates since one previously screwed it up before.

The first thing I happened upon, that I thought would be worth trying, was to disable the Smart Fan, and set to manual mode, as described HERE - viewtopic.php?p=802858#p802891 In the TS-451, you only have 3 manual options, Low / Medium / High. High is pretty loud, so I decided to start with Medium. Once I set it to Medium, and restarted, it was STILL freakin' beeping and flashing a red light, so I didn't think there was any type of change. But, then, when I went into the dashboard, my drive was there. It was still reporting an error, but it was there.

The error it was showing on the drive was "One or more unrecoverable read/write errors have been detected". I first ran tests on the drive, both Rapid & Complete, and, no errors were found, but the error remained. A search took me to this post HERE - viewtopic.php?t=162576#p799667

Well, I already had WinSCP loaded, just needed to refresh my memory on how to use it. Then, once I got connected over to my NAS, and followed the directions to the file that was being referenced, /mnt/HDA_ROOT/.conf I couldn't find it. Ended up having to use WinSCP's search function, and there it was. I hit Edit, and scrolled through the file. Sure enough, just as cameo had predicted back in Oct of 2021, there was a line that began with "pd_err_wwn_500" . Following his advice, I ejected the drive from bay 3 and waited until it was showing as removed. Then I deleted the offending line, saved the file, and then pushed the drive back in.

NO MORE ERROR!!!

Right now it has been up and going, without error, for about 7 hours, the last 6 of which it has been running bad block checks on drives 2, 3 & 4... again with no errors.

I'm going to head to bed soon, and let the tests finish up. When I get up in the morning, I'm going to see what the Stress Test under Help Desk \ Diagnostic Tool does.

I'll come back up provide additional updates later.

[RobbieWilkes]

For those that are interested, and possibly looking for solutions... so far so good. Bad block checks and stress test completed without any errors, and my NAS has been up and running for well over a day now.

Now I just need to figure out how best to configure the drives this time. The last time I made the big mistake of spanning a volume, MEDIA, across drives 2,3 & 4, so, when the NAS decided to lose its mind, it was only with the genius assistance of Paul, from QNAP (detailed in a previous post above), that my data wasn't completely lost. Well, at least what wasn't backed off on DVDs would have been. Still, while it wouldn't have been the end of the world, it would have still been painful.

So, I'm weighing the pros & cons of how best to proceed. Should I just make each drive (5tb / 6tb / 6tb) its own volume, and then split my data, based on type, across the 3, setting up backups to an external drive? Or, should I go with a RAID 5 configuration, which essentially eats one of my 6tb drives for "data protection"?

While the first option will be less "convenient", having to deal with multiple volumes, the difference in available space is fairly significant.

For my purposes, I'm thinking, once again, that outweighs the automated data protection that RAID offers.. especially when I consider the fact that my problems were actually caused by a "failure" in the system itself, and not with the drive. Plus, should the NAS itself fail completely, unless I bought the exact same model, the drives, with my data, couldn't even be transferred over to a different QNAP device.

So, it seems to me, the most beneficial / safe solution, is to err on the side of available space, balanced with automated backups to an external drive.

Anyway, I hope you've enjoyed my stream of consciousness. I hope someone finds my experience useful. Also, thanks to all those that fought the battles before, and shared their findings as they did, so I could get back up and running again. Hopefully for good.