"Disk Failed", then "Disk Unplugged" errors

[hscorpio15201]

To follow back up, the issue reared its ugly head again with a vengeance. Working with support now to see what the next steps are.

[Lord_Rocker]

Hi Guys,

Also have the same problem for the last couple of months after an update (Don't remember which one though!). I run a 453Pro with 4 x 5tb Toshibas for the last 2 years and lose drive 4 sometimes after a bootup. I've ran the QNAP diagnostic tool and found no problems listed in the SMART. I've set the stress test for 5 days with no problems. Definitely an odd problem, thinking about picking up another 5tb Toshiba for a swap over and see what happens!!

[carliant]

Hi Guys,
I have same problem on a Qnap TVS-473 on firmware 4.3.3.0238 using WD30EFRX 3.0TB hdd.
48 hrs after upgrading to current firmware i received a failed disk (in slot 3) message followed by a few minutes later with drive has been detected but is inaccessible then disk unplugged error message.
I then replaced hdd with a new disk exact same make and model it started rebuilding and at around 90% rebuild, exact same errors again.
I proceeded to replace the hdd with yet another brand new disk of exactly the same make and model.
I first tested it by connecting it to a Pc and made sure it was working, after making sure it was ok i inserted hdd in nas it then started rebuilding and at near completion of rebuild i again got same errors disk failed followed eventually by drive has been detected but is inaccessible then disk unplugged.
Original disk that failed (the 1st one) works perfectly on a Pc, the second one that failed is dead even on Pc, the 3rd drive that failed works ok on Pc.
So after this i decided to downgrade firmware to Version 4.3.3.0210 and restore to Factory Default.
Once at version 4.3.3.0210 i re- inserted 1st drive that had failed into nas to check it was detected and all loked ok for about 5min, after that i get the same errors again.
I then removed the drive again and tested it with Western Digital Data LifeGuard Diagnostics and all test OK, also tested that 3rd drive and it comes out ok also.
I created a support ticket, and i am waiting for a response, very strange problem 3 drives all of them have different manufacturing dates.
Don't really want to try another new drive in there.
Re-inserted the 3rd drive that had previously failed it was detected ok then dissapeared after a few minutes and the re-appeared after about an hour??
So i am currently re-creating my raid 10 array it has not re-failed yet, will post later when this thing finishes.

[MrVideo]

Report this issue to QNAP via their helpdesk support ticket.

[carliant]

Hi guys,
Quick update, I was able to re-create the array but it failed not long after, while it was syncing, below is an outtake of the log.
From that log one could think it's simply a bad drive but 3 of them one after the other ???
Still waiting on an update from that support ticket i created with Qnap helpdesk...


1107 Error 7:34:37 System 127.0.0.1 localhost Host: Disk 3 unplugged.
1106 Error 7:34:35 System 127.0.0.1 localhost A drive has been detected but is inaccessible. Please check it for faults.
1105 Information 5:19:09 System 127.0.0.1 localhost [Pool 1] Started rebuilding with RAID Group 1.
1104 Information 5:15:21 System 127.0.0.1 localhost Host: Disk 3 plugged in.
1103 Error 5:15:09 System 127.0.0.1 localhost Host: Disk 3 unplugged.
1102 Warning 5:15:08 System 127.0.0.1 localhost [Pool 1] RAID Group 1 is in degraded mode.
1101 Information 5:15:08 System 127.0.0.1 localhost Host: Disk 3 Device removed successfully.
1100 Error 5:15:02 System 127.0.0.1 localhost A drive has been detected but is inaccessible. Please check it for faults.
1099 Information 5:13:13 System 127.0.0.1 localhost Host: Disk 3 plugged in.
1098 Error 5:13:13 System 127.0.0.1 localhost Host: Disk 3 unplugged.
1097 Warning 5:13:12 System 127.0.0.1 localhost [Pool 1] RAID Group 1 is in degraded mode.
1096 Information 5:13:11 System 127.0.0.1 localhost Host: Disk 3 Device removed successfully.
1095 Error 5:13:00 System 127.0.0.1 localhost A drive has been detected but is inaccessible. Please check it for faults.
1094 Information 5:12:57 System 127.0.0.1 localhost Host: Disk 3 Disabled NCQ since timeout error.
1093 Warning 5:09:30 System 127.0.0.1 localhost Host: Disk 3 Read I/O error, ILLEGAL REQUEST sense_key=0x5, asc=0x21, ascq=0x4, CDB=88 00 00 00 00 00 17 17 51 08 00 00 0c 80 00 00 .
1092 Information 3:24:21 admin xxx.xxx.xxx.xxx --- [System Administration] Server name changed from [xxxxx] to [yyyyy]
1091 Information 3:17:19 System 127.0.0.1 localhost [Pool 1] Successfully set threshold to 50%.
1090 Information 3:12:22 System 127.0.0.1 localhost [Pool 1] Created storage pool successfully (Capacity: 5.44 TB, Threshold 0%) with RAID Group 1 ( Host Drive: 'Disk 1'-'Disk 2' 'Disk 3'-'Disk 4', RAID Level: RAID10 ).
1089 Information 3:12:21 System 127.0.0.1 localhost [Pool 1] Successfully disabled snapshot reserved space.
1088 Information 3:12:03 System 127.0.0.1 localhost [RAID Group 1] Started resyncing.
1087 Information 3:12:02 System 127.0.0.1 localhost Started to create storage pool.
1086 Information 2:58:34 System xxx.xxx.xxx.xxx localhost [Disk S.M.A.R.T.] Enabled All Hard Drives warning temperature to 51 ºC.
1085 Information 2:54:25 System 127.0.0.1 localhost [Disk S.M.A.R.T.] Host: Disk 3 Complete Test result: Completed without error.
1084 Information 2:14:47 System 127.0.0.1 localhost [Disk S.M.A.R.T.] Host: Disk 3 Complete Test started.
1083 Information 2:13:44 System 127.0.0.1 localhost [Disk S.M.A.R.T.] Host: Disk 3 Rapid Test result: Completed without error.
1082 Information 2:11:44 System 127.0.0.1 localhost [Disk S.M.A.R.T.] Host: Disk 3 Rapid Test started.
1081 Information 23:55:54 System 127.0.0.1 localhost Host: Disk 3 plugged in.

[ridney]

I am also having this problem. I solved it before by deleting system partitions and reformatting all drives. Now, I may have to do it all over again. Its a firmware problem, silent data corruption.

https://www.qnap.com/en/technical-advis ... -201707-01

[P3R]

Its a firmware problem, silent data corruption.

https://www.qnap.com/en/technical-advis ... -201707-01

Yes that advisory is about silent data corruption during a successful RAID 5/6 rebuild. I would consider that a totally different issue than "Disk Failed" and "Disk Unplugged" messages so off-topic here.

Frequent "Disk Failed" and "Disk Unplugged" errors are most of the times because of NAS hardware issues.

[hoserama]

I have submitted a ticket for this very same issue - I experienced it twice in the past 2-3 weeks.

I have TS-653 Pro with 6 8tb Western Digital Reds


Sent from my iPhone using Tapatalk

[ehaase]

That's interesting -- I have a TVS-463 (16GB upgrade, good brand memory) with four WD red pro 6TB. I run regular updates and upgrades. When drive 3 malfunctioned (less than a year old!) I replaced it. New drive went red, so I replaced it. New drive went red again, so I replaced it with a WD red non-pro. New drive went red again, so I replaced it. New drive went red, so I swapped the TVS-463. The drive was still marked red / abnormal but I could manually get it to rebuild. SMART tests have been run, both quick and full and error-free. SMART values are all 'good'. RAID5 group has finished rebuilding okay. Drive 3 is still in "ABNORMAL" state and I wonder how do I get it out of there.

Best,
--Enver

[MrVideo]

Firmware version and build date?

Open a support ticket.

[hoserama]

Qnap support wants me to pull my upgraded memory and see if the issue still exists. This pushes me closer to building my own box when my Qnap becomes end of life.


Sent from my iPhone using Tapatalk

[ServalCat]

Hi all.
I put in a help request (see my previous post for what was tried first). The support person required remote support access to patch my system and then I was able to do a manual update to 4.3.3.0262 2017/07/27.
The system has now been up and running without flaws for 6 days, no indication that disks 3 or 4 are acting abnormal, RAID data scrubbing went ok, touch wood. Thanks again for everyone's advice.
-----------------------------------------
Model: TS451A 8GB (single Qnap RAM board) Firmware version 4.3.3.0262 Build 20170727 (primarily as fileserver)
Disks: 2 + 2 + 3 + 3 TB as RAID 5 (will likely revert to a safer configuration at some point).
Disk models: (model firmware) 2x (WDC WD20EFRX-68EUZN0 82.00A82), 2x (WDC WD30EFRX-68EUZN0 82.00A82)
UPS: CPS Value800EIGP Network: 1Gb Ethernet from NAS port 1 to FRITZ!Box router.

[redboot]

This seems to be a common problem now. I also am having the same problem on a TVS-473. Using version 4.3.3.0262. Yes, I've submitted a ticket using the Helpdesk "app" and am waiting for a first response.
From what I've been reading above, this seems to often be "bay" oriented, mine is bay #3. It does not seem to be the disk. I have, in fact, swapped out a known good identical disk and have the same issue.
What I've also found interesting is that when I looked at the SMART data for my four original 6TB drives, only the one in the bad bay had some strange stats:
Power_Cycle_Count of 68 where others have 9
Power-Off_Retract_Count of 62 where others have 1
My hypothesis is that something in the QNAP (bad SATA board or bad QTS) is in fact shutting these drives down.

FWIW, here's the details:
Summary problem: Despite replacement of a disk with an identical disk known to be good, my QNAP continually powers off the disk in Bay #3, reports it as missing, then attempts to rebuild the array after it's re-inserted.

What was attempted to resolve problem:

1. The RAID was shown as degraded and no disk in bay #3. The log shows that the drive was considered disconnected and the icon on the screen appeared like no disk existed.
2. I pulled the disk caddy out of the QNAP a few inches. Waited several seconds and re-inserted the disk. It recognized the disk and began re-building the array.
3. At some point the disk was reported as having errors and it suggested a bad block scan. I ran the scan and it reported no bad blocks, came back on-line and re-built the array.
4. The same error returned.
5. I looked at the SMART statistics for the drive and found two peculiar stats:
Power_Cycle_Count of 68 where others have 9
Power-Off_Retract_Count of 62 where others have 1
This seems to indicate that the drive is somehow being powered off at arbitrary times.
6. I assumed it might be a bad disk drive.
7. I pulled an identical drive from a Windows array (known to be good as it had run as a mirror for 7 days without a problem) and inserted the drive.
8. I recognized the drive, and but within 5 minutes the QNAP had turned itself off completely.
9. I restarted the QNAP. The logs show that the same kinds of errors occurred as before. So the problems have NOT gone away but have continued.
10. I powered off the QNAP, removed bay 3 disk and caddy completely. Used canned air to blow out SATA connection for the disk drive. Made sure drive was firmly re-inserted back into the bay.
11. Restarted QNAP and found the same error cycle to continue.


It suspect that the QNAP power supply is not sufficiently powering the disk in bay#3. I also have no idea why the whole unit would have powered itself down (see step 8 above.)

The QNAP is currently running in a degraded RAID state.

I'll spare you the whole log, but it cycles through like this:

Code: Select all

A drive has been detected but is inaccessible. Please check it for faults.
A drive has been detected but is inaccessible. Please check it for faults.
A drive has been detected but is inaccessible. Please check it for faults.
[Pool 1] Started migrating with RAID Group 1.
[Pool 1] Migration skipped with RAID Group 1.
Host: Disk 3 unplugged.
Host: Disk 3 Device removed successfully.

[dpsmits]

Hi guys,

I'm helping out a friend of mine who has a Qnap TS-421 using version 4.3.3.0262. As far as I know memory has never been upgraded or replaced. There is 1 RAID5 volume on 4 1,5TB Seagate disks.

A few days ago the system reported Drive 1 and 2 where unplugged. Drive 3&4 where OK. We pulled each disk out and placed it back in, one after another. Then the system was fine again until a couple of days later when it was reported drive 2 was unplugged again. A few hours later drive 1 had the same issue. We did the same thing but now grive 2 was mentioned as a 3.4GB drive instead of 1,5TB. We replaced the faulty disk (drive 2) with a new WD red 2TB disk so the RAID volume could be restored. Both drive 1 and 2 were not recognized at first. Switching drive 1 and new drive 2 it seemed to work but soon after we lost drive 1 again (the one with the original disk) with the unplugged error again.

We again pulled out this disk and placed it back. For some reason now both drive 1 and 2 are not recognized anymore although we did not do anything to the drive containing the new WD disk. We put the new drive in a external disk casing to confirm the drive was working. It was and we put it back but still both drives are not recognised.

I do have the feeling the Qnap system is somehow not responding anymore. It is still reachable through the web management but I'm scared to restart the device.

We submitted a ticket with (Dutch) Qnap support early in the process but we got a reply telling us it is possible to duplicate a disk using software on a Windows computer. I don't see what this has to do with our problem. We'll be contacting them again. I do not wish to take out drives to often as we did it already more than I like...

Any of you have an idea if we can restart the Qnap device and maybe solve the problem of not recognising the disks? We need te reconstruct the raid volume so we can replace all troubling disks one at a time. Any help would be welcome!

Thanks!

[MrVideo]

The first time the NAS declared the first two drives as unplugged, the RAID should have been toast, as RAID5 can only handle one drive failure. It is beginning to sound like a hardware issue. I do hope you have backups of all your data.