RAID6 not rebuilding/resyncing after drive replace

[mcmnky]

Hello,

I have a TVS-471 with firmware 5.0.0.2055 and 4 HDD in RAID6. I am starting the process to replace all 4 with larger disks and expand capacity. After replacing the first disks, the RAID status is Degraded. Rebuilding did not start automatically after putting in the new drive. I am not able to manually start rebuilding.

NAS03_2.png

NAS03_3.png

I am following the steps here: https://www.qnap.com/en-us/how-to/tutor ... ty-upgrade
I also referred to this: https://www.qnap.com/en/how-to/faq/arti ... hould-i-do

The new disks are WD121KFBX-68EF5N0 and are on the compatibility list.

NAS03_6.png

Going to Storage/Snapshots both the pool and the volume have status "Warning (Degraded)". Going in to manage the pool, all 4 disks show status good with a green check.
The available options to manage the pool are only resync priority and rebuild RAID group.

NAS03_4.png

If I try to manually rebuild, no disks are available to select.

NAS03_5.png

Any ideas? Is it possible the volume is rebuilding without the status saying so?

[dolbyman]

did you swap the disk hot ? (while the NAS is running)

can you do a

Code: Select all

cat /proc/mdstat

&

Code: Select all

md_checker

[mcmnky]

did you swap the disk hot ? (while the NAS is running)

Yes, hot swap.

can you do a

Code: Select all

cat /proc/mdstat

Code: Select all

[Admin@nas03 ~]$ cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] [multipath]
md1 : active raid6 sdd3[7](S) sda3[5] sdb3[4] sdc3[6]
      11701135488 blocks super 1.0 level 6, 64k chunk, algorithm 2 [4/3] [U_UU]
        resync=PENDING

md322 : active raid1 sdd5[3](S) sdc5[2](S) sdb5[1] sda5[0]
      7235136 blocks super 1.0 [2/2] [UU]
      bitmap: 0/1 pages [0KB], 65536KB chunk

md256 : active raid1 sdd2[3](S) sdc2[2](S) sdb2[1] sda2[0]
      530112 blocks super 1.0 [2/2] [UU]
      bitmap: 0/1 pages [0KB], 65536KB chunk

md13 : active raid1 sdd4[24] sda4[26] sdc4[27] sdb4[25]
      458880 blocks super 1.0 [24/4] [UUUU____________________]
      bitmap: 1/1 pages [4KB], 65536KB chunk

md9 : active raid1 sdd1[24] sda1[26] sdc1[27] sdb1[25]
      530048 blocks super 1.0 [24/4] [UUUU____________________]
      bitmap: 1/1 pages [4KB], 65536KB chunk

unused devices: <none>
[Admin@nas03 ~]$

Code: Select all

md_checker

Code: Select all

[Admin@nas03 ~]$ md_checker

Welcome to MD superblock checker (v2.0) - have a nice day~

/usr/bin/md_checker: line 25: /mnt/HDA_ROOT/md_backup_2022-08-10_13.02.26: Permission denied
Scanning system...

[Admin@nas03 ~]$

Looking into why I'm getting permission denied for md_checker.

Thanks!

[mcmnky]

Figured out the md_checker problem--it only works for ADMIN, not other users with admin privileges. (Normally I keep the default admin user disabled. I changed the user name in the output above.)

Code: Select all

[~] # md_checker

Welcome to MD superblock checker (v2.0) - have a nice day~

Scanning system...


RAID metadata found!
UUID:           e91315b0:9b11f370:327b6479:e1ec86ea
Level:          raid6
Devices:        4
Name:           md1
Chunk Size:     64K
md Version:     1.0
Creation Time:  Mar 24 15:32:15 2016
Status:         ONLINE (md1) [U_UU]
===============================================================================================
 Enclosure | Port | Block Dev Name | # | Status |   Last Update Time   | Events | Array State
===============================================================================================
 NAS_HOST       1        /dev/sda3   0   Active   Aug 10 13:19:34 2022  4355055   A.AA
 ----------------------------------  1  Missing   -------------------------------------------
 NAS_HOST       3        /dev/sdc3   2   Active   Aug 10 13:19:34 2022  4355055   A.AA
 NAS_HOST       2        /dev/sdb3   3   Active   Aug 10 13:19:34 2022  4355055   A.AA
===============================================================================================

[~] #

[dolbyman]

disabling the admin user is smoke and mirrors introduced by qnap (exploits don't care what users are en or disabled .. exposed NAS still get infected by malware .. no matter the users or stuff like 2FA)

try

Code: Select all

mdadm --readwrite /dev/md1

[mcmnky]

try

Code: Select all

mdadm --readwrite /dev/md1

Code: Select all

[~] # mdadm --readwrite /dev/md1
mdadm: failed to set writable for /dev/md1: Device or resource busy
[~] #

Also ran:

Code: Select all

[~] # mdadm --detail /dev/md1
/dev/md1:
        Version : 1.0
  Creation Time : Thu Mar 24 15:32:15 2016
     Raid Level : raid6
     Array Size : 11701135488 (11159.07 GiB 11981.96 GB)
  Used Dev Size : 5850567744 (5579.54 GiB 5990.98 GB)
   Raid Devices : 4
  Total Devices : 4
    Persistence : Superblock is persistent

    Update Time : Wed Aug 10 13:40:13 2022
          State : clean, degraded, resyncing (PENDING)
 Active Devices : 3
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 1

         Layout : left-symmetric
     Chunk Size : 64K

           Name : 1
           UUID : e91315b0:9b11f370:327b6479:e1ec86ea
         Events : 4355487

    Number   Major   Minor   RaidDevice State
       5       8        3        0      active sync   /dev/sda3
       2       0        0        2      removed
       6       8       35        2      active sync   /dev/sdc3
       4       8       19        3      active sync   /dev/sdb3

       7       8       51        -      spare   /dev/sdd3
[~] #

[dolbyman]

try a reboot of the NAS

if that does not help try

Code: Select all

echo repair > /sys/block/md1/md/sync_action 

[mcmnky]

Running a manual backup (in addition to the scheduled daily backup) before rebooting. I will reboot and check back in when that is done.

Thank you!

[mcmnky]

Reboot, back, and rebuilding.

Thank you, dolbyman!!