"Disk Failed", then "Disk Unplugged" errors

Questions about SNMP, Power, System, Logs, disk, & RAID.
Post Reply
foz111
Getting the hang of things
Posts: 63
Joined: Fri Oct 23, 2015 2:37 am

Re: "Disk Failed", then "Disk Unplugged" errors

Post by foz111 »

I have this dam problem on my ts-253pro
it started with SMART error (at this time i only had disc 1) see thread below
viewtopic.php?f=15&t=152002&p=749026#p749026
i purchased 2 new HDD's as i decided to raid it up.
Now i only have disc 2 in raid disc 1 is not seen in RAID the log shows backplane issue and out of warranty.
In storage - system enclosure the NAS OS see's both disk in green but storage/snapshots DataVol1 raid group 1 only see's disk 2
so my question, has anyone repaired Ts-253 pro and if so could you share the information on what needs doing.
Thanks in advance
TS-253D
QTS 5.0.1.2376 Firmware
DataMeister
Starting out
Posts: 16
Joined: Fri Jan 27, 2017 3:22 am

Re: "Disk Failed", then "Disk Unplugged" errors

Post by DataMeister »

WillHuu wrote: Tue Mar 24, 2020 5:10 am I'm using a TS-831X with 8x8Tb in RAID6 and had 2 bays failing. The disks tried to spinup and failed, over and over again producing the 'clicking' as described in this thread. Sometimes, the NAS did boot correct and started rebuilding, but always failed eventually.
After testing both disks multiple times on different systems without any issues, i desperately bought new 8tb hdd's to be able to have the NAS working again. But my worst fear came true, the same unreliable behavior occurred again. Looking for similar experiences this thread showed up. And i'm glad it did!

Reading about the dysfunctional backplanes, and QNAP not replacing them within acceptable price-range made me quite disappointed.

As at least one user mentioned, simply cooling the unit made the issues disappear. So, i first tried just manually setting the fan to max speed. It's a bit noisy but, huh, the NAS seems stable?! Rebuilding finished successfully and in an attempt to tune down the noise i turned on the 'Smart Fan' automatic fan adjust again. Set manually to the minimum possible thresholds, low speed at 35°C and high speed at 45°C. Within half an hour, both disks failed again!

Now, i simply placed heat-sinks on all MOSFET's, and the system is running fine for days now, with the automatic fan speed turned on.
Rebuilding without issues, and both the old HDD's and new (but ofcourse..) HDD's passing all tests without issues.
Assuming the MOSFET's will fail eventually, it feels great to know i can still bypass them completely when needed.

I really wanted to share this experience since it has been quite a hit on our workflow and my wallet.

I can't thank the community enough, especially the users who tracked down the actual issue with the failing MOSFET's which are responsible for the HDD's power management.

And for any QNAP employees / developers reading this, it would be nice to be able to adjust the automatic fan thresholds to lower values.
A minimum of 35°C for low fan speed and 45°C for high doesn't make sense, at least the ability to set it lower would be nice. And could probably expand the lifetime for a lot of your sold products.
What kind of heat sinks did you use? Are they the same foot print as the MOSFETs, are they self adhesive, or...?
foz111
Getting the hang of things
Posts: 63
Joined: Fri Oct 23, 2015 2:37 am

Re: "Disk Failed", then "Disk Unplugged" errors

Post by foz111 »

Below is a quick shot of the backplane on my TS-253 pro
can anyone point out the mosfet that controls the power to RAID for disc 1?
Image
TS-253D
QTS 5.0.1.2376 Firmware
User avatar
dolbyman
Guru
Posts: 35024
Joined: Sat Feb 12, 2011 2:11 am
Location: Vancouver BC , Canada

Re: "Disk Failed", then "Disk Unplugged" errors

Post by dolbyman »

just by looking at the proximity and traces, I'd say bottom left/middle one is for disk0 (left) and bottom right one is for disk1

you can always measure continuity to the connector it with a voltmeter (there could be diodes in the path, so make sure you have the polarity of the probes right)
darfie21
Starting out
Posts: 29
Joined: Thu May 25, 2017 6:35 am

Re: "Disk Failed", then "Disk Unplugged" errors

Post by darfie21 »

Slot 2 went bad on TS-231 a few weeks ago. Any disk I put in refuses to rebuild. After about 5% the rebuild errors out with:

3>[71680.008043] ata2.00: status: { DRDY }
<6>[71680.011750] ata2: hard resetting link
<6>[71683.179357] ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
<4>[71683.198265] Check proc_name[ahci].
<6>[71683.205951] ata2.00: configured for UDMA/33
<6>[71683.210295] ata2: EH complete
<3>[71687.334872] ata2.00: exception Emask 0x10 SAct 0x7 SErr 0x10002 action 0xe frozen
<3>[71687.342475] ata2.00: irq_stat 0x00400000, PHY RDY changed
<3>[71687.347893] ata2: SError: { RecovComm PHYRdyChg }
<3>[71687.352645] ata2.00: failed command: READ FPDMA QUEUED
<3>[71687.357820] ata2.00: cmd 60/08:00:d0:ff:1c/00:00:00:00:00/40 tag 0 ncq 4096 in
<3>[71687.357823] res 40/00:10:e0:ff:1c/00:00:00:00:00/40 Emask 0x10 (ATA bus error)
<3>[71687.373105] ata2.00: status: { DRDY }
<3>[71687.376779] ata2.00: failed command: READ FPDMA QUEUED
<3>[71687.381951] ata2.00: cmd 60/08:08:d8:ff:1c/00:00:00:00:00/40 tag 1 ncq 4096 in
<3>[71687.381955] res 40/00:10:e0:ff:1c/00:00:00:00:00/40 Emask 0x10 (ATA bus error)
<3>[71687.397238] ata2.00: status: { DRDY }
<3>[71687.400921] ata2.00: failed command: READ FPDMA QUEUED
<3>[71687.406084] ata2.00: cmd 60/08:10:e0:ff:1c/00:00:00:00:00/40 tag 2 ncq 4096 in
<3>[71687.406088] res 40/00:10:e0:ff:1c/00:00:00:00:00/40 Emask 0x10 (ATA bus error)
<3>[71687.421434] ata2.00: status: { DRDY }
<6>[71687.425158] ata2: hard resetting link
<6>[71690.649363] ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 310)

I dont about you but I have had all kinds of problems with this unit from random reboots etc etc. I could go on! I like the software but the HW has not been stable. I would consider asustor or a Synology in your case.


patrikr wrote: Sun Jun 21, 2020 1:14 am Hi all,

Add me to the list of people affected by this... My TS-231 worked great for 3½ years, but a few days ago it suddenly reported that disk 2 failed. I replaced it with a new WD Red, and that one "failed" 3% into rebuilding the RAID 1. Then I found this thread through a Google search. A big thank you to all the knowledgeable people here who have discovered and explained the root cause!

My question now is: has QNAP learned from their mistakes and fixed the backplane in their newer models? I'm looking at maybe buying a TS-231K as a replacement, it was apparently released a few months ago so hopefully it's improved?

Or should I give up on QNAP and get a Synology instead?
microsolder
Starting out
Posts: 11
Joined: Fri May 15, 2020 12:49 am

Re: "Disk Failed", then "Disk Unplugged" errors

Post by microsolder »

foz111 wrote: Fri Jul 10, 2020 4:28 pm Below is a quick shot of the backplane on my TS-253 pro
can anyone point out the mosfet that controls the power to RAID for disc 1?
The component marked Q4 appears to be a two-channel MOSFET, so that likely is the high-side switch of the disk 2 (HDD 2 connector).
Q3 right from it is its driver. If needed, Q4 can be easily jumpered following the instructions presented earlier for
other devices.

U1 between the SATA connectors at bottom is not a switch MOSFET but something else. Don't jumper it. If there is a switch MOSFET
for the disk 1 (HDD 1 connector) it is probably under the metal shielding left of the connector and hence not visible in the photo.
Digging it out (probably marked Q1 for the driver and Q2 for the switch) requires a bit more dismantling of the device.
User avatar
dolbyman
Guru
Posts: 35024
Joined: Sat Feb 12, 2011 2:11 am
Location: Vancouver BC , Canada

Re: "Disk Failed", then "Disk Unplugged" errors

Post by dolbyman »

@microsolder..could you mark continuity testpoints in the pic ? (at least the ones visible)

so the person asking could make sure what components are what ? bridging was already shown earlier (either accorss the component or be removing the mosfet alltogether and bridging the traces)

even thought replacing it would be safest..right ?
microsolder
Starting out
Posts: 11
Joined: Fri May 15, 2020 12:49 am

Re: "Disk Failed", then "Disk Unplugged" errors

Post by microsolder »

dolbyman wrote: Sun Jul 12, 2020 5:44 am @microsolder..could you mark continuity testpoints in the pic ? (at least the ones visible)
so the person asking could make sure what components are what ?
TS-253-pro-bp.png
Continuity testing an unpowered board will not detect the problem. What one needs to do is to
monitor the +5 V and +12 V lines at the SATA power connector. The related pins are marked in the
picture. The critical line is the +5 V. While +12 V tolerates 10% (1.2V) drop and 10.8 V is OK, the
+5 V has only 5% tolerance. The disk begins to reset when the +5 V drops below 4.75 V. So the
right method for the analysis is to monitor the +5 V during normal operation. A DMM with min/max
sampling is helpful to catch the voltage drops.

One can also measure the gate pins of the switch MOSFET, the ones on the input side next to the
input voltage lines. They are connected together and driven by the same voltage. When on the
gate voltage should be below +1 V, which I suspect it isn't before modifying the driver circuit.
In TS-431 it was +3.2 V which results in only -1.8V gate drive -- not enough.

The method to locate the resistor to replace is to check which resistor is the one that is connected
directly to the gate pins. In the picture it may be R70. Then measure its resistance. If it is 100 kΩ
replace it with a 10 kΩ one and recheck the gate voltage. It should then be much lower.
This replacement without damaging anything around it requires some skill, though,
due to the size of the component and little space around it.
dolbyman wrote: Sun Jul 12, 2020 5:44 am bridging was already shown earlier (either accorss the component or be removing the mosfet alltogether and bridging the traces)
even thought replacing it would be safest..right ?
Replacing the switch MOSFET is useful only if the driver is also modified so that the gate voltage is low enough.
At least that is the case with TS-431. Jumpering the MOSFET always works even though the power switching
functionality is lost.
You do not have the required permissions to view the files attached to this post.
User avatar
dolbyman
Guru
Posts: 35024
Joined: Sat Feb 12, 2011 2:11 am
Location: Vancouver BC , Canada

Re: "Disk Failed", then "Disk Unplugged" errors

Post by dolbyman »

I just asked for continuity testing, to ID the correct component(s) ..not to measure the functionality of a switched off circuit ;)

For most its safest to assume their component is shot, starting to probe around in a running system to monitor for voltage drops ...doubt that is a good idea, or even practical(unless that person is versed enough to do it)

Thanks for the help
foz111
Getting the hang of things
Posts: 63
Joined: Fri Oct 23, 2015 2:37 am

Re: "Disk Failed", then "Disk Unplugged" errors

Post by foz111 »

Thanks for the info, before i went ahead and started stripping this unit down i wanted to confirm the repair would in fact work, so i connected sata cable and external power supply and boom raid rebuilt over night and seems to be working fine. so i will be attempting the repair with help in near future from a friend that is very good with these types of repairs tv's etc.
If anyone reading this can send me the guide from kommisar for bypassing this mosfet or anything that may help my mate, i would really appreciate it.
Image
Image
TS-253D
QTS 5.0.1.2376 Firmware
User avatar
dolbyman
Guru
Posts: 35024
Joined: Sat Feb 12, 2011 2:11 am
Location: Vancouver BC , Canada

Re: "Disk Failed", then "Disk Unplugged" errors

Post by dolbyman »

nothing needs to be sent..just read the last 10 pages or so, of this thread
foz111
Getting the hang of things
Posts: 63
Joined: Fri Oct 23, 2015 2:37 am

Re: "Disk Failed", then "Disk Unplugged" errors

Post by foz111 »

Pictures of the back plane removed:
Hopefully this is a better picture now its been removed to identify the failed mosfet on this TS-253 pro backplane.

Image
Image
TS-253D
QTS 5.0.1.2376 Firmware
microsolder
Starting out
Posts: 11
Joined: Fri May 15, 2020 12:49 am

Re: "Disk Failed", then "Disk Unplugged" errors

Post by microsolder »

The TS-235 pro back plane you show does not have a second MOSFET switch. Like other QNAPs the disk 1 is fed directly from the connector. Therefore either there is a failing switch on the main board (can't tell if there is any without seeing the main board) or the 5 V buck regulator, also on the main board, is failing. Next, photo of the main board, please. :D (This gets interesting.)
foz111
Getting the hang of things
Posts: 63
Joined: Fri Oct 23, 2015 2:37 am

Re: "Disk Failed", then "Disk Unplugged" errors

Post by foz111 »

The problem is definitely on that backplane board not the main board, i say that will 100% certainty because i took a gamble and purchased a complete but none booting broken ts-253 pro, it had 8gb ram and mine only had 4 and gave £50 for it from eBay so thought it was worth a punt, hoping i could salvage the ram and backplane, it arrived this week and on Thursday evening i removed the RAM and the backplane board and placed into my nas, now my nas is functioning correctly with both drives inserted and in raid 1, and now with 8gb ram :D . I am not normally this lucky :-0
I will at some stage have a go at seeing if i can get the broken one booting but seems to me to be a bios issue as i get no beeps or hdmi output when starting up but that's another story and thread for another day.
I was hoping to get this backplane working if possible just in case the other failed at some stage, currently i have a usb fan now blowing air in towards the hdd trays and the main fan set manual - medium speed now running between 29 and 32c where it always seemed to be 34-36 before.
Thanks again

P.S. I can and will upload some photos of the main board but it will be from the broken none booting nas if that''s ok? same board so hopefully you'll get to see what you want to from that.
Just to add, it's been working since Thursday evening so approx 40 hours up time and fingers crossed no issues or messages about failing SMART errors so far, both drives showing healthy and working as should but not wanting to tempt fate.
TS-253D
QTS 5.0.1.2376 Firmware
DataMeister
Starting out
Posts: 16
Joined: Fri Jan 27, 2017 3:22 am

Re: "Disk Failed", then "Disk Unplugged" errors

Post by DataMeister »

foz111 wrote: Sat Jul 25, 2020 9:13 pm The problem is definitely on that backplane board not the main board, i say that will 100% certainty because i took a gamble and purchased a complete but none booting broken ts-253 pro, it had 8gb ram and mine only had 4 and gave £50 for it from eBay so thought it was worth a punt, hoping i could salvage the ram and backplane, it arrived this week and on Thursday evening i removed the RAM and the backplane board and placed into my nas, now my nas is functioning correctly with both drives inserted and in raid 1, and now with 8gb ram :D . I am not normally this lucky :-0
I will at some stage have a go at seeing if i can get the broken one booting but seems to me to be a bios issue as i get no beeps or hdmi output when starting up but that's another story and thread for another day.
I was hoping to get this backplane working if possible just in case the other failed at some stage, currently i have a usb fan now blowing air in towards the hdd trays and the main fan set manual - medium speed now running between 29 and 32c where it always seemed to be 34-36 before.
Thanks again

P.S. I can and will upload some photos of the main board but it will be from the broken none booting nas if that''s ok? same board so hopefully you'll get to see what you want to from that.
Just to add, it's been working since Thursday evening so approx 40 hours up time and fingers crossed no issues or messages about failing SMART errors so far, both drives showing healthy and working as should but not wanting to tempt fate.
It's disappointing QNAP won't just sell a replacement back plane and ship it to the customer for onsite replacement.
Post Reply

Return to “System & Disk Volume Management”