"Disk Failed", then "Disk Unplugged" errors

Questions about SNMP, Power, System, Logs, disk, & RAID.
Post Reply
Flotsch123
Know my way around
Posts: 117
Joined: Sun Mar 12, 2017 9:03 pm

Re: "Disk Failed", then "Disk Unplugged" errors

Post by Flotsch123 » Thu Dec 20, 2018 9:25 pm

P3R wrote:
Thu Dec 20, 2018 8:11 pm
Flotsch123 wrote:
Thu Dec 20, 2018 6:10 pm
I thought i would get an RMA ID so tha rti can track the status.
I on the other hand would be very surprised if you got that.

Your relation is to your reseller. The reseller have the relation to Qnap so when things happen, they're the ones that should be informed about it. That would be my expectation of how things work.
ok, sounds reasonable. i will wait until 4 weeks (first week of january 2019 and ask my reseller again. is there a kind of "time limit" (for example 6 weeks), and if this limit is reached, i can ask to give me my money back or for a new NAS? otherwise they could let me wait "forever" for my NAS to be returned.

P3R
Guru
Posts: 11745
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: "Disk Failed", then "Disk Unplugged" errors

Post by P3R » Fri Dec 21, 2018 12:51 am

Flotsch123 wrote:
Thu Dec 20, 2018 9:25 pm
ok, sounds reasonable. i will wait until 4 weeks (first week of january 2019 and ask my reseller again. is there a kind of "time limit" (for example 6 weeks), and if this limit is reached, i can ask to give me my money back or for a new NAS? otherwise they could let me wait "forever" for my NAS to be returned.
If you're an end consumer, you need to check the consumer rights on your specific market. If you represent a business, there are probably other laws surrounding the business-to-business deal with your reseller. That's at least how it work in my country.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

davaze
New here
Posts: 3
Joined: Sun Dec 17, 2017 7:13 pm

Re: "Disk Failed", then "Disk Unplugged" errors

Post by davaze » Wed Dec 26, 2018 9:20 pm

This week i suddenly have disk 3 failed on my TVS 473. It is using WD RED 4TB drives (WD40EFRX-68N32N0).
Running firmware 4.3.6.0776. I have taken out the drive from the TVS 473 and connected it to my desktop PC. Running WD Lifeguard diagnostic tool extended test and quick test all came out without any errors.
To me it looks there is no drive issue. The TVS473 is just 1,5 years old. Now my RAID 5 has been degraded and running 3 remaining disks (slot 1, 2 and 4) 3 is out.
Already send a ticket to QNAP to find out what is happening. Did anyone having similar issues already have an solution or in all cases it was hardware failure and need repair from QNAP.

Thanks and wish everyone a happy 2019.
Dave

P3R
Guru
Posts: 11745
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: "Disk Failed", then "Disk Unplugged" errors

Post by P3R » Wed Dec 26, 2018 10:41 pm

davaze wrote:
Wed Dec 26, 2018 9:20 pm
Did anyone having similar issues already have an solution or in all cases it was hardware failure and need repair from QNAP.
As far as I know there are three possible causes for this problem:
  • A now fixed QTS-issue on a few isolated models - your model was not affected by that
  • The disk have failed - normally the most likely reason but pretty unlikely in your case as the WD test came out clean
  • The Qnap backplane is defective - likely in your case as your model lines may be among the ones most affected by it
Qnap will help you diagnose.

Make sure your backups are up to date. If you have no external backups, now it's very high time to make them.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

davaze
New here
Posts: 3
Joined: Sun Dec 17, 2017 7:13 pm

Re: "Disk Failed", then "Disk Unplugged" errors

Post by davaze » Thu Dec 27, 2018 4:04 pm

P3R wrote:
Wed Dec 26, 2018 10:41 pm
davaze wrote:
Wed Dec 26, 2018 9:20 pm
Did anyone having similar issues already have an solution or in all cases it was hardware failure and need repair from QNAP.
As far as I know there are three possible causes for this problem:
  • A now fixed QTS-issue on a few isolated models - your model was not affected by that
  • The disk have failed - normally the most likely reason but pretty unlikely in your case as the WD test came out clean
  • The Qnap backplane is defective - likely in your case as your model lines may be among the ones most affected by it
Qnap will help you diagnose.

Make sure your backups are up to date. If you have no external backups, now it's very high time to make them.
Thanks for your clear feedback. I'll wait for QNAP support to diagnose the problem. Once clear i will post it here for others having similar issues. Though i'm really surprised having issues already within 2 years. Did not expect this from QNAP. Let's wait and see what comes out. Just happy its still within warranty.

P3R
Guru
Posts: 11745
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: "Disk Failed", then "Disk Unplugged" errors

Post by P3R » Thu Dec 27, 2018 6:42 pm

davaze wrote:
Thu Dec 27, 2018 4:04 pm
Though i'm really surprised having issues already within 2 years. Did not expect this from QNAP.
All manufacturers of technical equipment (including much more expensive brands than Qnap) will have a few that fail prematurely. That's the reason warranties exist.

We all of course expect/hope not to be affected personally because the risk is very small but unfortunately it happen to some of us.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

blub
Starting out
Posts: 13
Joined: Mon May 16, 2011 2:38 am

Re: "Disk Failed", then "Disk Unplugged" errors

Post by blub » Thu Jan 03, 2019 4:19 am

davaze wrote:
Wed Dec 26, 2018 9:20 pm
This week i suddenly have disk 3 failed on my TVS 473. It is using WD RED 4TB drives (WD40EFRX-68N32N0).
Running firmware 4.3.6.0776. I have taken out the drive from the TVS 473 and connected it to my desktop PC. Running WD Lifeguard diagnostic tool extended test and quick test all came out without any errors.
To me it looks there is no drive issue. The TVS473 is just 1,5 years old. Now my RAID 5 has been degraded and running 3 remaining disks (slot 1, 2 and 4) 3 is out.
Already send a ticket to QNAP to find out what is happening. Did anyone having similar issues already have an solution or in all cases it was hardware failure and need repair from QNAP.

Thanks and wish everyone a happy 2019.
Dave
Same here with Disk 4 and a TS-473- FW is from end of November.

My Problem is that the NAS is in a physical location I can't reach (10000 KM away) and I have to decide what to do next or what to bring on my next visit.

First I thought well its bad drive but after reading this thread and looking at my logs I am pretty sure it is not the drive. Additionally I had similar issues with a 45x NAS last year and all drives were fine when I tested them - and now the replacement NAS is also broken with a similar issue - oh man I am so ** right now.

blub
Starting out
Posts: 13
Joined: Mon May 16, 2011 2:38 am

Re: "Disk Failed", then "Disk Unplugged" errors

Post by blub » Thu Jan 03, 2019 9:51 pm

Is there any NAS I can buy from Qnap that is not affected?

Thisisnotmyname
Easy as a breeze
Posts: 353
Joined: Mon Nov 19, 2018 1:21 am

Re: "Disk Failed", then "Disk Unplugged" errors

Post by Thisisnotmyname » Thu Jan 03, 2019 10:24 pm

blub wrote:
Thu Jan 03, 2019 9:51 pm
Is there any NAS I can buy from Qnap that is not affected?
All technology can fail.

You haven't said much about your situation but maybe you'd be better with some sort of managed or cloud service instead. Make hardware failures someone else's issue. Then being 10k km away doesn't matter. Or if this is a critical application that MUST be on premise I'd go for a heck of a lot more redundancy. And then you need to think about redundancy of every link in the chain (do you have redundant networking [power supplies in your gear, cabling, switches, etc...], power [not just UPS but generator], connectivity into the site, etc...) Depending on criticality I'd also be thinking about disaster recovery and getting those data replicated off site real (as near as possible) time, you never know if that location 10k km away will burn to the ground and all your data with it.

Some of that may be overkill for you but maintaining 99.999% uptime in a total no touch lights out environment requires a corresponding investment in tech (and you'll still not be guaranteed 100% up).

Some things to consider.

blub
Starting out
Posts: 13
Joined: Mon May 16, 2011 2:38 am

Re: "Disk Failed", then "Disk Unplugged" errors

Post by blub » Thu Jan 03, 2019 11:55 pm

Thisisnotmyname wrote:
Thu Jan 03, 2019 10:24 pm
blub wrote:
Thu Jan 03, 2019 9:51 pm
Is there any NAS I can buy from Qnap that is not affected?
All technology can fail.

You haven't said much about your situation but maybe you'd be better with some sort of managed or cloud service instead. Make hardware failures someone else's issue. Then being 10k km away doesn't matter. Or if this is a critical application that MUST be on premise I'd go for a heck of a lot more redundancy. And then you need to think about redundancy of every link in the chain (do you have redundant networking [power supplies in your gear, cabling, switches, etc...], power [not just UPS but generator], connectivity into the site, etc...) Depending on criticality I'd also be thinking about disaster recovery and getting those data replicated off site real (as near as possible) time, you never know if that location 10k km away will burn to the ground and all your data with it.

Some of that may be overkill for you but maintaining 99.999% uptime in a total no touch lights out environment requires a corresponding investment in tech (and you'll still not be guaranteed 100% up).

Some things to consider.
It is just a NAS for movies and music, so nothing critical and I have more than one backup. The only problem with that setup is that it is really remote - not only from my main living area but also within the area where it is located - i.e. to get spare parts it is always a whole day trip, sometimes even two days.
Internet is slow and unreliable, so cloud service is not an option - and UPS we do have.

But nevertheless it is really annoying to be plagued by the same issue on 2 different brand new Qnap NAS machines within 15 months - there is no denying that.

From what I have read here is that there is some sort of an issue with backplanes from certain NAS series and I wanted to ask if that issues is resolved or if any NAS series is not affected - if that is the case I might just by a new one/3rd one and move the disks. If a buy another brand I need new disks to copy the data over which increases the cost even more

Thisisnotmyname
Easy as a breeze
Posts: 353
Joined: Mon Nov 19, 2018 1:21 am

Re: "Disk Failed", then "Disk Unplugged" errors

Post by Thisisnotmyname » Fri Jan 04, 2019 12:32 am

blub wrote:
Thu Jan 03, 2019 11:55 pm

It is just a NAS for movies and music, so nothing critical and I have more than one backup. The only problem with that setup is that it is really remote - not only from my main living area but also within the area where it is located - i.e. to get spare parts it is always a whole day trip, sometimes even two days.
Internet is slow and unreliable, so cloud service is not an option - and UPS we do have.

But nevertheless it is really annoying to be plagued by the same issue on 2 different brand new Qnap NAS machines within 15 months - there is no denying that.

From what I have read here is that there is some sort of an issue with backplanes from certain NAS series and I wanted to ask if that issues is resolved or if any NAS series is not affected - if that is the case I might just by a new one/3rd one and move the disks. If a buy another brand I need new disks to copy the data over which increases the cost even more


I've seen a couple references to x73 back plane failures on the forums but anecdotal data is anecdotal. QNAP would be the only ones with real comprehensive data and aside from issuing a recall they'd probably never publicize that sort of issue. That said, my totally unscientific survey of remembering threads here says that it's more likely with the x73 line but I've seen others with a dead backplane too.

P3R
Guru
Posts: 11745
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: "Disk Failed", then "Disk Unplugged" errors

Post by P3R » Fri Jan 04, 2019 1:02 am

blub wrote:
Thu Jan 03, 2019 11:55 pm
But nevertheless it is really annoying to be plagued by the same issue on 2 different brand new Qnap NAS machines within 15 months - there is no denying that.
I would of course feel the same. You definitely had your fair share of bad luck! :cry:
From what I have read here is that there is some sort of an issue with backplanes from certain NAS series and I wanted to ask if that issues is resolved or if any NAS series is not affected...
The model lines being affected in any significant numbers are as far as I've noticed TS-X63 and TS-X73 (I'd say probably around 50/50). With significant I mean maybe 200 units alltogether having been mentioned here in the forum, probably not significant in comparison with actual sales. Apart from those two model lines I'd say that no other lines are really affected. There are a few TS-X53 as well reported but those are high-volume models, so considering my guess about sales numbers I'd say those cases are insignificant.

The above are of course only the speculations and guesses from me (not employee, only long-time user and forum participant) that have followed the matter here with some interest.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

blub
Starting out
Posts: 13
Joined: Mon May 16, 2011 2:38 am

Re: "Disk Failed", then "Disk Unplugged" errors

Post by blub » Fri Jan 04, 2019 1:31 am

Thisisnotmyname wrote:
Fri Jan 04, 2019 12:32 am
blub wrote:
Thu Jan 03, 2019 11:55 pm

It is just a NAS for movies and music, so nothing critical and I have more than one backup. The only problem with that setup is that it is really remote - not only from my main living area but also within the area where it is located - i.e. to get spare parts it is always a whole day trip, sometimes even two days.
Internet is slow and unreliable, so cloud service is not an option - and UPS we do have.

But nevertheless it is really annoying to be plagued by the same issue on 2 different brand new Qnap NAS machines within 15 months - there is no denying that.

From what I have read here is that there is some sort of an issue with backplanes from certain NAS series and I wanted to ask if that issues is resolved or if any NAS series is not affected - if that is the case I might just by a new one/3rd one and move the disks. If a buy another brand I need new disks to copy the data over which increases the cost even more


I've seen a couple references to x73 back plane failures on the forums but anecdotal data is anecdotal. QNAP would be the only ones with real comprehensive data and aside from issuing a recall they'd probably never publicize that sort of issue. That said, my totally unscientific survey of remembering threads here says that it's more likely with the x73 line but I've seen others with a dead backplane too.

OK, thx. However Qnap hasent answered to my support ticket yet

kommisar
New here
Posts: 9
Joined: Thu Nov 01, 2018 4:35 pm

Re: "Disk Failed", then "Disk Unplugged" errors

Post by kommisar » Tue Jan 08, 2019 5:05 am

After almost a year in service my TVS-473 NAS suddenly dropped disk 3 and the slot refused to work since when. Sounds familiar? :)

Before continuing I would like to emphasize, following IS NOT approved, confirmed or authorized by QNAP! Use it with caution!

First of all don't panic your data as well as the disk itself are fine and you only need a way to backup it. To do that:
1) Power Off your NAS
2) Take the "bad" disk out
3) Connect it to the NAS backplane with male-to-female SATA data extension cable
4) Use any external power source with SATA power connector to provide power to the "bad" disk. It can be standalone PC PSU but you need to figure out how to turn it on.
5) !!!!!!!!!!!!! IT IS IMPORTANT TO MAKE ALL THE CONNECTIONS WITHOUT POWER TO NAS OR EXTERNAL SATA POWER SOURCE !!!!!!!!!!!!!!!!!!!!
6) Power on the external SATA PSU first and NAS after that
7) Wait for NAS to start, RAID to synchronize and backup all your data to external storage.

Now, when the data is safe, it is the time to contact QNAP support, and RMA your faulty NAS. It will take time and in some case money but you will get fixed NAS and keep it under the warranty.
But it is also possible to try and fix the issue by yourself. The cost - penny and 15 minutes with soldering station. !!!!!!!!!! BUT YOU WILL VOID WARRANTY AND ANY UNAUTHORIZED/UNPROFESSIONAL MODIFICATION CAN DAMAGE YOUR NAS. IT ALSO REQUIRES SOLDERING SKILLS AND KNOWLEDGE HOW TO PROPERLY HANDLE DELICATE ELECTRONIC COMPONENTS !!!!!!!!!!!!!!!!

In my opinion the only option to go is to contact QNAP. QNAP support team is helpful and professional. The issue is known to QNAP and RMA is issued on spot. And you need really good reasons to try and fix the NAS by yourself.

If you brave enough to take the second option anyway, lets start from the issue background. It looks like the devices affected are TVS-x73 manufactured before late fall of 2017. On those devices disks 3-8 SATA power is controlled by a chain of MOSFETs. High level on the Gate of control N-channel MOSFET opens high-power P-channel MOSFET and the corresponded SATA slot gets power. And it looks like the control part of the power circuit is responsible for the issue in subject. For some reason control circuit disables high-power MOSFET for a moment and it starts chain of events leading to the disk to be unplugged. Sudden loss of the power also makes HDD to produce loud "click" mentioned earlier in this thread. Another evidence is the following error in kernel log:

ata6: SError: { RecovComm PHYRdyChg 10B8B Dispar } -- Count:x
ata6: SError: { RecovComm PHYRdyChg CommWake 10B8B Dispar DevExch } -- Count:x

It is possible to modify the control circuit to lock the high-power MOSFET in open state permanently and fix the issue for good. I will not provide exact instructions what to do to avoid unnecessary casualties. But if you have right set of skills, info above is more than enough to figure out offending part and right fix procedure.

To summarize, the issue is HW related. At this stage there is no SW/FW fix. If your "Disk Unplugged", your NAS is on warranty and you see symptoms described above - contact QNAP. If it is not possible just follow the provided clues.
Last edited by kommisar on Tue Jan 08, 2019 5:15 am, edited 3 times in total.

User avatar
OneCD
Ask me anything
Posts: 6983
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: "Disk Failed", then "Disk Unplugged" errors

Post by OneCD » Tue Jan 08, 2019 5:10 am

Hi and welcome to the forum. :)
kommisar wrote:
Tue Jan 08, 2019 5:05 am
For some reason control circuit disables high-power MOSFET for a moment and it starts chain of event leading to the disk to be unplugged. Sudden loss of the power also makes HDD to produces loud "click" mentioned earlier in this thread.
Good find!
kommisar wrote:
Tue Jan 08, 2019 5:05 am
To summarize, the issue is HW related. At this stage there is no SW/FW fix.
Will this require QNAP to replace the affected board with a newer version?

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage

Post Reply

Return to “System & Disk Volume Management”