Volume unmounted, fsck is failing

[miodzicho]

Hi All

I have opened ticket with support, but right now they pointed me to HDD manufacturer tools. Now my hope is in community.
My volume is showing as unmounted, and fsck is failing, I tried :

Code: Select all

[~] # e2fsck_64 -fp -C 0 /dev/mapper/cachedev1
Data: recovering journal
Data: Superblock needs_recovery flag is clear, but journal has data.
Data: Run journal anyway.
e2fsck_64: Unknown code ____ 251 while recovering journal of Data
e2fsck_64: unable to set superblock flags on Data


Data: ********** WARNING: Filesystem still has errors **********

And :

Code: Select all

[~] # e2fsck_64 -fp -C 0 -b 32768 /dev/mapper/cachedev1
Data: recovering journal
Data: Superblock needs_recovery flag is clear, but journal has data.
Data: Recovery flag not set in backup superblock, so running journal anyway.
e2fsck_64: Unknown code ____ 251 while recovering journal of Data
e2fsck_64: unable to set superblock flags on Data


Data: ***** FILE SYSTEM WAS MODIFIED *****

Data: ********** WARNING: Filesystem still has errors **********

[storageman]

What does md_checker say?
I assume the RAID is offline.

[miodzicho]

Code: Select all

[~] # md_checker

Welcome to MD superblock checker (v2.0) - have a nice day~

Scanning system...


RAID metadata found!
UUID:           8ca92779:3ffcb2ee:93543c94:957454a2
Level:          raid5
Devices:        3
Name:           md1
Chunk Size:     512K
md Version:     1.0
Creation Time:  Jul 28 07:23:41 2016
Status:         ONLINE (md1) [UUU]
===============================================================================================
 Enclosure | Port | Block Dev Name | # | Status |   Last Update Time   | Events | Array State
===============================================================================================
 NAS_HOST       2        /dev/sda3   0   Active   Apr  2 13:23:19 2020    21356   AAA
 NAS_HOST       1        /dev/sdb3   1   Active   Apr  2 13:23:19 2020    21356   AAA
 NAS_HOST       3        /dev/sdc3   2   Active   Apr  2 13:23:19 2020    21356   AAA
===============================================================================================


RAID metadata found!
UUID:           80806708:364304a0:b438f0c4:77d0c7e2
Level:          raid1
Devices:        1
Name:           md2
Chunk Size:     -
md Version:     1.0
Creation Time:  Jul 28 07:25:32 2016
Status:         ONLINE (md2) [U]
===============================================================================================
 Enclosure | Port | Block Dev Name | # | Status |   Last Update Time   | Events | Array State
===============================================================================================
 NAS_HOST       4        /dev/sdd3   0   Active   Apr  2 13:23:35 2020      266   A
===============================================================================================

[~] #

[storageman]

This is weird, how many disks did you think were in the RAID 5?
And was it just one volume?
Can u run this and see if it corrects anything?
"/etc/init.d/init_lvm.sh"

[miodzicho]

I do have 3 disks in RAID5 and one single in second volume.

Code: Select all

[~] # /etc/init.d/init_lvm.sh
Changing old config name...
Reinitialing...
Detect disk(8, 80)...
ignore non-root enclosure disk(8, 80).
Detect disk(8, 48)...
dev_count ++ = 0Detect disk(8, 16)...
dev_count ++ = 1Detect disk(8, 64)...
ignore non-root enclosure disk(8, 64).
Detect disk(8, 32)...
dev_count ++ = 2Detect disk(8, 0)...
dev_count ++ = 3Detect disk(8, 80)...
ignore non-root enclosure disk(8, 80).
Detect disk(8, 48)...
Detect disk(8, 16)...
Detect disk(8, 64)...
ignore non-root enclosure disk(8, 64).
Detect disk(8, 32)...
Detect disk(8, 0)...

sys_startup_p2:got called count = -1

  --==  Thank you for participating in the global usage survey  ==--
The server's response is:

you are the 644th user to install this version
  WARNING: duplicate PV AZrqDew8gDqQ3hELMWaAdDen9FUWDgDP is being used from both devices /dev/drbd1 and /dev/md1
  Found duplicate PV AZrqDew8gDqQ3hELMWaAdDen9FUWDgDP: using /dev/drbd1 not /dev/md1
  Using duplicate PV /dev/drbd1 from subsystem DRBD, ignoring /dev/md1
  WARNING: duplicate PV otxD2DWrtBddJv3Rr3B2HUdGmz5Ow1zS is being used from both devices /dev/drbd2 and /dev/md2
  Found duplicate PV otxD2DWrtBddJv3Rr3B2HUdGmz5Ow1zS: using /dev/drbd2 not /dev/md2
  Using duplicate PV /dev/drbd2 from subsystem DRBD, ignoring /dev/md2
sh: /sys/block/dm-5/dm/pool/tier/relocation_rate: Permission denied
Done

[~] #

[storageman]

Unknown code ____ 251 while recovering journal of Data
Typically means bad block issues. Try SMART checking all the drives.

Does mounting read only work? At least you could backup then.
"mount -t ext4 -o ro /dev/mapper/cachedev1 /share/CACHEDEV1_DATA"

[miodzicho]

Unfortunately mounting doesn't work...
[~] # mount -t ext4 -o ro /dev/mapper/cachedev1 /share/CACHEDEV1_DATA
mount: /dev/mapper/cachedev1: can't read superblock

And SMART seems just fine for all 4 drivers...but I just noticed, there was low space warning on this volume, could it be related ?

[storageman]

Try file check with backup superblock, otherwise I'm out of ideas.
"e2fsck_64 -fp -C 0 -b 32768 /dev/mapper/cachedev1"

[miodzicho]

Code: Select all

[~] # e2fsck_64 -fp -C 0 -b 32768 /dev/mapper/cachedev1
Data: recovering journal
Data: Superblock needs_recovery flag is clear, but journal has data.
Data: Recovery flag not set in backup superblock, so running journal anyway.
e2fsck_64: Unknown code ____ 251 while recovering journal of Data
e2fsck_64: unable to set superblock flags on Data


Data: ***** FILE SYSTEM WAS MODIFIED *****

Data: ********** WARNING: Filesystem still has errors **********

[~] #

Seems I may be out of luck...

[CieminskiW]

Any solution? I have similar problem.

After upgrade, one of volume won't mount.


Neither manually


Repair failed


md_checker

[Tony Feliu]

Hi,
I have extactly the same problem.
On a TS-453B with a RAID5 (4 disks) the Datavol is unmounted.
When I try to do a Filesystem check through web interface stops at 1% with an error.

This is the md_checker results:

Code: Select all

[~] # md_checker

Welcome to MD superblock checker (v2.0) - have a nice day~

Scanning system...


RAID metadata found!
UUID:           fdb1284b:df7ea5c3:4c104f1d:8ac02f22
Level:          raid5
Devices:        4
Name:           md1
Chunk Size:     64K
md Version:     1.0
Creation Time:  May 27 08:36:16 2018
Status:         ONLINE (md1) [UUUU]
===============================================================================================
 Enclosure | Port | Block Dev Name | # | Status |   Last Update Time   | Events | Array State
===============================================================================================
 NAS_HOST       1        /dev/sdc3   0   Active   Jul 18 11:13:47 2021   193038   AAAA
 NAS_HOST       2        /dev/sdd3   1   Active   Jul 18 11:13:47 2021   193038   AAAA
 NAS_HOST       4        /dev/sdb3   2   Active   Jul 18 11:13:47 2021   193038   AAAA
 NAS_HOST       3        /dev/sda3   3   Active   Jul 18 11:13:47 2021   193038   AAAA
===============================================================================================

The result of e2fsck_64:

Code: Select all

[~] # e2fsck_64 -f -v -C 0 /dev/mapper/cachedev1
e2fsck 1.43.9 (8-Feb-2018)
DataVol1: recovering journal
Superblock needs_recovery flag is clear, but journal has data.
Run journal anywaye2fsck_64: Unknown code ____ 251 while recovering journal of DataVol1
e2fsck_64: unable to set superblock flags on DataVol1


DataVol1: ********** WARNING: Filesystem still has errors **********

Resulto of dumpe2fs:

Code: Select all

[~] # dumpe2fs /dev/mapper/cachedev1 | grep superblock
dumpe2fs 1.43.9 (8-Feb-2018)
  Primary superblock at 0, Group descriptors at 1-1602
  Backup superblock at 32768, Group descriptors at 32769-34370
  Backup superblock at 98304, Group descriptors at 98305-99906
  Backup superblock at 163840, Group descriptors at 163841-165442
  Backup superblock at 229376, Group descriptors at 229377-230978
  Backup superblock at 294912, Group descriptors at 294913-296514
  Backup superblock at 819200, Group descriptors at 819201-820802
  Backup superblock at 884736, Group descriptors at 884737-886338
  Backup superblock at 1605632, Group descriptors at 1605633-1607234
  Backup superblock at 2654208, Group descriptors at 2654209-2655810
  Backup superblock at 4096000, Group descriptors at 4096001-4097602
  Backup superblock at 7962624, Group descriptors at 7962625-7964226
  Backup superblock at 11239424, Group descriptors at 11239425-11241026
  Backup superblock at 20480000, Group descriptors at 20480001-20481602
  Backup superblock at 23887872, Group descriptors at 23887873-23889474
  Backup superblock at 71663616, Group descriptors at 71663617-71665218
  Backup superblock at 78675968, Group descriptors at 78675969-78677570
  Backup superblock at 102400000, Group descriptors at 102400001-102401602
  Backup superblock at 214990848, Group descriptors at 214990849-214992450
  Backup superblock at 512000000, Group descriptors at 512000001-512001602
  Backup superblock at 550731776, Group descriptors at 550731777-550733378
  Backup superblock at 644972544, Group descriptors at 644972545-644974146
  Backup superblock at 1934917632, Group descriptors at 1934917633-1934919234
  Backup superblock at 2560000000, Group descriptors at 2560000001-2560001602
  Backup superblock at 3855122432

Trying to do a Filesystem with a backup superblock:

Code: Select all

[~] # e2fsck_64 -fp -C 0 -b 32768 /dev/mapper/cachedev1
DataVol1: Superblock needs_recovery flag is clear, but journal has data.
DataVol1: Recovery flag not set in backup superblock, so running journal anyway.
DataVol1: recovering journal
DataVol1: Superblock needs_recovery flag is clear, but journal has data.
DataVol1: Recovery flag not set in backup superblock, so running journal anyway.
e2fsck_64: Unknown code ____ 251 while recovering journal of DataVol1
DataVol1: Superblock needs_recovery flag is clear, but journal has data.
DataVol1: Recovery flag not set in backup superblock, so running journal anyway.
e2fsck_64: unable to set superblock flags on DataVol1


DataVol1: ***** FILE SYSTEM WAS MODIFIED *****

DataVol1: ********** WARNING: Filesystem still has errors **********

Trying to mount Datavol on read-only to backup data:

Code: Select all

[~] # mount -t ext4 -o ro /dev/mapper/cachedev1 /share/CACHEDEV1_DATA
mount: /dev/mapper/cachedev1: can't read superblock

Did somebody find a solution?.
Of course I oppened a ticket on QNAP but no answer at the moment.

[AntonioManuel]

This is weird, how many disks did you think were in the RAID 5?
And was it just one volume?
Can u run this and see if it corrects anything?
"/etc/init.d/init_lvm.sh"

DataVol1 Read only Status
Thanks. This worked for me.