Disks spontaneously disconnecting - requiring RAID rebuild

[Time4aNap]

Hello,

For the second time in a month, a disk has spontaneously disconnected from the QNAP. I have to reinsert it, then rebuild the RAID, which takes around 60 hours, which is a long time to be in a degraded state.

NAS is a TS-435XeU
HDDs: 3 x Toshiba N300 12TB in a RAID 5
QTS: 5.0.1.2248

The first time, it was Bay 1 where the disk disconnected, so I removed it and added a NEW disk in Bay 4, and rebuilt the RAID. I then added another disk in Bay 1 to see if the Bay was broken, and it worked fine.

Now it's Bay 2 where the disk has spontaneously disconnected.

I am pretty sure the first disk that disconnected from Bay 1 is not faulty, because I am currently using it in another NAS with no issues. So this 2nd disk is probably fine as well. They are only about 6 months old and although they hold a lot of data, they don't get a whole bunch of I/O.

I am aware of the mosfet issues and have temperature alarms on the disks. Nothing has gone off, and the 2 other disks that are running cool. 37C. I have no reason to believe anything is overheating.

I guess I'm looking for any tips anyone may have, and moral support as I rebuild this large 21TB RAID for the second time in a month.

[dolbyman]

Open a support ticket, if you are aware of the mosfet issues, then you know it's a hardware defect..it needs to be dealt with by swap (within warranty) or hardware fix (soldering)

[Time4aNap]

I am not sure if this is a mosfet issue, i just know this issue exists in general.
I'll open a ticket with QNAP. I did the first time it happened but they weren't super helpful.
Thank you for the response!

[Time4aNap]

Looking at the logs, I see "NCQ disabled due to timeout" a minute before the drive disconnected. Ugh.

[dolbyman]

If the bus or drive has issues the system disables features and resorts to slower fallback speeds to keep it running. (3Gbit vs 6Gbit SATA)

[Time4aNap]

QNAP support said to replace the disk. Off I go to replace the disk.

I already took the old one out, since it was already disconnected.

I'll put the new one in, same exact model and size (Toshiba N300 12TB, which btw is NOT on the compatibility list, but the 16TB and the 8TB are on the list, so I believe that just means that this combination hasn't been tested, but should be fine since other sizes from the same line are compatible. Maybe I am wrong about this. The reason I have disks that are not on the compatibility list is that I migrated them from another QNAP (TS-431k) with which these drives ARE compatible.)

I'll keep the QNAP running and put the drive into one of the 2 vacant bays. Haven't decided which bay, because I've now had a drive fail in both of them. Flip a coin. I'll put it in Bay 2, which is where the drive I am replacing was. Bay 1 is the other available bay, a drive failed in there a month ago.

I expect the the RAID to start rebuilding as soon as I insert the new drive.

It will take many hours (40, 50, maybe 60 hours), and I will check the progress constantly, each time holding my breath and hoping the rebuild has not failed, or another disk died. I'm driving through the desert with no water a no spare tire.

At 99.99% complete, I will probably be watching with anticipation for the rebuild to complete and my precious RAID array to turn green.

I do have nice backups by the way, but obviously, I'd rather not loose the RAID array, as that would interrupt the work of the busy bees while I restored everything.

I'll be back with news!

[Time4aNap]

Well, after 40 hours, the RAID was rebuilt. It lasted 12 hours before Disk 3 (a brand new disk I added last week) disconnected, and now I'm degraded again. Contacted qnap support, again.

[Time4aNap]

While waiting for qnap support to respond, I am rebuilding the array, again.
I suspect at some point qnap will tell me to send the unit in for repair, so I am ordering a new qnap, because the busy bees have to work. Doh!

[dolbyman]

if you are planning to migrate the disks, be aware of migration directions .. you can migrate ARM to x86 but not the other way

https://www.qnap.com/en/nas-migration

[Time4aNap]

if you are planning to migrate the disks, be aware of migration directions .. you can migrate ARM to x86 but not the other way

https://www.qnap.com/en/nas-migration

Thanks, good to know. I did check to make sure I could migrate the disks to the new nas and I should be fine. Moving to a TVS-h474 which is indeed an ARM to x86 move. Going for a slightly more robust system this time. Fingers crossed. This would be fun if it wasnt so stressful

[Time4aNap]

QNAP Support says: "I was informed this may be a issue with the drive trays and the size of the drives causing the drives to disconnect. I will go ahead and start a RMA to have mylar stickers sent to you. You would put these stickers on the drive tray such as in the video link provided."

YAY! Stickers! lol

Here's the video: https://www.dropbox.com/s/xaoidb0d5douq ... l.mov?dl=0

Not impossible I guess. The NAS is located downtown where big trucks driving by sometime make the building shake a bit, and since the NAS is in a rack mounted to the wall, I can see how it could vibrate the drives loose if they weren't in tightly enough. Odd nonetheless though

[Time4aNap]

Update: It has been around 6 weeks since my adventure with randomly failing disks (the disk in bay 1 failed, i replaced it and rebuilt the raid. Then a week later it was bay 2, and then bay 3). No problems since then. The disks that failed are happily running in another QNAP nas, so nothing wrong with the disks. QNAP sent me some stickers to put on the disks so that they would fit more tightly in the bays, so maybe that's what has fixed the issue.