SSD cache crash now all storage down ?

[Sylrob]

Good Morning,

Got an SSD drive failure, this drive was for the cache acceleration, so I replace the disk with a new one but RAID group 1 stay inactive, So my Storage pool 2 has an error and my Datavol1 say unmounted ?

How I can restart my Raid group 1 or deactivate completely to be able to start my storage pool 2 !
I already tried manage-> remove in the Cache acceleration and it don't work !
I also unplug the SDD and plug it back it don't work !

TVS871 QTS 5.0.0.2055

Thank you very much for your help !

Sylvain

[dolbyman]

No matter if read or write cache ..if the cache fails your cached volume will be bust(A RAID1+ is highly recommended..sadly you used an single disk cache)...seems counter intuitive for read cach ..but thats what users have reported

Open a ticket with QNAP (hope you have solid backup)

[Sylrob]

Hi Dolbyman,

I wans't aware of this . Yes I have solid backup no problem but have a lot of TB to restore and redo the share and security settings

Already open a ticket with QNAP hopy I'll be able to restart my Storage pool !. I'll not used the SSD cache anymor because I have no spare space for another SSD !

Anyhing that I can't try ?

Thank you for your precious time !
Sylvain

[dolbyman]

Best to wait for QNAP...rule of thumb..if your NAS has no dedicated cache slots (2.5inch or M2 onboard)...don't even bother..no need to cannibalize a free slot for a single cache drive

[Sylrob]

Hope they will be able to restart my Pool2 this one is on a RAID 6 wich is completly different from the cache RAID and all disk are working perfectly !!

I'll never use SSD caching again with a RAID 1 group, and I'll not cannibalize 2 slots for this !

TY for your precious time dolbyman

[dosborne]

I wans't not aware of this . Yes I have solid backup no problem but have a lot of TB to restore and redo the share and security settings

I would hope that backing up your NAS configuration would include some or all of the elements you mentioned. I've never had to recover, and only use minimal shares, users, etc, so I don't really know, but part of my routine maintenance is to keep a copy of every firmware version for my units and after any significant change, or once a month, whichever comes first, I backup the configuration and rename it to include the data and firmware version associated with it (example: "2022-06-28 NAS30_settings FW4.5.4.2012.bin"). These get stored as part of my cloud backup plan.

Just an idea for moving forward for "the next time"

[Sylrob]

dosborne,

I'm pretty sure that I'll have to restart from scratch (hope not) but I'll see, here the NAS is a file server share for all windows users (more than 50 users work on it every day) and I have a SMTP server Xeam running on it !. Saving config is a great idea will try next time, but now it's a nogo as you said !. Will take a LOOONNNGGG time to restart from scratch !. Will be easier if I can restart my RAID 6 storage pool 2 without the cache which is working ok,!!

Thank's for taking the time to answer !

Sylvain

[dolbyman]

I'll never use SSD caching again with a RAID 1 group, and I'll not cannibalize 2 slots for this !

According to the screenshot, "RAID"GROUP1 was a single disk (hence the total loss upon single drive failure)

[mapotter99]

I just had this same problem after some freezing issues that started as soon as I changed my SSD cache.

What is the status for you?

[Sylrob]

Good Morning,
For the record, I had to contact QNAP tech support to fix my problem !. I was lucky that my cache was a read-only cache !!

This is what tech support told me after the fix:
" There's a known issue with the SSD cache in the old QTS 5.0 version. If the SSD cache suddenly fails for whatever reasons, the storage pool will be inactive. You must contact us to troubleshoot the issue and recover the storage pool. "

For that reason I deactivate the SSD cache for all the installation that I have with QTS 5.0. Just before doing it at one site, I began to have strange behavior in 3 software hosted on the QNAP samba share, the SDD cache wasn't read only, when I stopped the cache and waiting for the cache to flush all pending update, I see one of my Windows EXE with a length of -1 !

I'll never use SSD cache on a QNAP with a QTS 5.0, I have a TVS-H1288X with Quts hero 5.0.0 build 20220614 and I didn't see that kind of behavior (for now) !.

In short if your SSD cache drive completely failed you'll have to call Qnap tech support to reactivate your storage pool if you are using QTS 5.0 !

HTH,

Sylvain

[gab696]

Hi,
I have a very similar problem. My ssd is marked Disk Access History I/O" ERROR
Volumes are hidden but visible from the command line

# md_checker

Code: Select all

Welcome to MD superblock checker (v2.0) - have a nice day~

Scanning system...


RAID metadata found!
UUID:		fac0db83:345be4fa:274beb08:6f8a43f9
Level:		raid5
Devices:	3
Name:		md1
Chunk Size:	512K
md Version:	1.0
Creation Time:	Oct 12 09:04:45 2016
Status:         ONLINE (md1) [UUU]
===============================================================================================
 Enclosure | Port | Block Dev Name | # | Status |   Last Update Time   | Events | Array State
===============================================================================================
 NAS_HOST       1        /dev/sda3   0   Active   Aug 14 11:41:18 2022  4095546   AAA                      
 NAS_HOST       2        /dev/sdb3   1   Active   Aug 14 11:41:18 2022  4095546   AAA                      
 NAS_HOST       3        /dev/sdc3   2   Active   Aug 14 11:41:18 2022  4095546   AAA                      
===============================================================================================


RAID metadata found!
UUID:		f47a9eae:f8da3c9f:4b5fdcce:559421c2
Level:		raid1
Devices:	1
Name:		md2
Chunk Size:	-
md Version:	1.0
Creation Time:	Oct 31 14:27:58 2021
Status:         ONLINE (md2) [U]
===============================================================================================
 Enclosure | Port | Block Dev Name | # | Status |   Last Update Time   | Events | Array State
===============================================================================================
 NAS_HOST       4        /dev/sdd3   0   Active   Aug 14 11:40:51 2022      167   A                        
===============================================================================================

# mdadm --detail /dev/md2

Code: Select all

/dev/md2:
        Version : 1.0
  Creation Time : Sun Oct 31 14:27:58 2021
     Raid Level : raid1
     Array Size : 502850368 (479.56 GiB 514.92 GB)
  Used Dev Size : 502850368 (479.56 GiB 514.92 GB)
   Raid Devices : 1
  Total Devices : 1
    Persistence : Superblock is persistent

    Update Time : Sun Aug 14 11:40:51 2022
          State : active, Rescue 
 Active Devices : 1
Working Devices : 1
 Failed Devices : 0
  Spare Devices : 0

           Name : 2
           UUID : f47a9eae:f8da3c9f:4b5fdcce:559421c2
         Events : 167

    Number   Major   Minor   RaidDevice State
       0       8       51        0      active sync   /dev/sdd3

# mdadm --detail /dev/md1

Code: Select all

/dev/md1:
        Version : 1.0
  Creation Time : Wed Oct 12 09:04:45 2016
     Raid Level : raid5
     Array Size : 15608142848 (14885.08 GiB 15982.74 GB)
  Used Dev Size : 7804071424 (7442.54 GiB 7991.37 GB)
   Raid Devices : 3
  Total Devices : 3
    Persistence : Superblock is persistent

    Update Time : Sun Aug 14 11:41:18 2022
          State : clean 
 Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 512K

           Name : 1
           UUID : fac0db83:345be4fa:274beb08:6f8a43f9
         Events : 4095546

    Number   Major   Minor   RaidDevice State
       4       8        3        0      active sync   /dev/sda3
       3       8       19        1      active sync   /dev/sdb3
       5       8       35        2      active sync   /dev/sdc3

Can I clone the SSD drive with a SATA Duplicator and pur the new drive on the NAS ?

Thanks !

[Sylrob]

Good Afternoon,

Sorry for the late answer, I'll strongly suggest to open a ticket at Qnap !

Best regards,

Sylvain

[ottl05]

Good Morning,

Got an SSD drive failure, this drive was for the cache acceleration, so I replace the disk with a new one but RAID group 1 stay inactive, So my Storage pool 2 has an error and my Datavol1 say unmounted ?

How I can restart my Raid group 1 or deactivate completely to be able to start my storage pool 2 !
I already tried manage-> remove in the Cache acceleration and it don't work !
I also unplug the SDD and plug it back it don't work !

TVS871 QTS 5.0.0.2055

Thank you very much for your help !

Sylvain

I have a similar issue.
Due to a power outage a SSD from the cache (raid1) failed.
Unplugging the SSD and plug back don´t work, so I added a new one (same model), but the cache raid status is failed and my datavols are unmounted.

How could I rebuild the Cache-Raid?

TS-473A QTS 5.0.0.2131
Thanks.

[Toxic17]

I have a similar issue.
Due to a power outage a SSD from the cache (raid1) failed.
Unplugging the SSD and plug back don´t work, so I added a new one (same model), but the cache raid status is failed and my datavols are unmounted.

How could I rebuild the Cache-Raid?

TS-473A QTS 5.0.0.2131
Thanks.

Sorry for the late answer, I'll strongly suggest to open a ticket at Qnap !

[ottl05]

I have a similar issue.
Due to a power outage a SSD from the cache (raid1) failed.
Unplugging the SSD and plug back don´t work, so I added a new one (same model), but the cache raid status is failed and my datavols are unmounted.

How could I rebuild the Cache-Raid?

TS-473A QTS 5.0.0.2131
Thanks.

Sorry for the late answer, I'll strongly suggest to open a ticket at Qnap !

with the help from support, my system is running again.
this time with ssd as read-cache