Hello!
Unfortunately I had to hard shutdown my TS-410 after one rsync job and some other services stopped working. The box wasn't accessable any more.
System is Raid5 Ext4 with 4x3GB Hitachi-Disks. Additionally disk 3 showed warning because of read failure at smart on 1 sector. 8TB MD-Volume was about 50% in use with data.
After booting up everything was fine execpt the system advised to perform a fsck. I knew this will not be completed for some time. Never the less I started the process because haven't done this for long time. And also because of the smart error in disk 3.
Now it is running about 38 hours and still shows 0% status. I know I've to be patient ...
But I'm not sure at the moment if this will ever end. Because I can see in the webui that the disks have about 28°C which is really little. Normally they have 46-48°C constant. So they nearly turn very slow I guess. Or even not?
And meanwhile disk 3 shows more errors: 46 current_pending_sectors & 4 reallocated_sector_ct - that is completely new status.
Hope this will end in a happy way.
I know I should have changed disk 3 earlier. But I thought this single error is not a big problem some weeks ago. Planned it this week because of holidays but it mostly comes different as you think or plan.
Anyway. Can any one calm me down only to be more patient? How can I additionally check if the fsck is this going on and just takes more time? Status change of disk 3 may be one way. But I like control more real time and with reliable method
BR,
pah96
FSCK on TS-410 still running after 264 hours
-
- Starting out
- Posts: 17
- Joined: Sun Jun 15, 2014 2:04 pm
FSCK on TS-410 still running after 264 hours
Last edited by pah96 on Tue May 18, 2021 4:07 am, edited 1 time in total.
TS-451+ with 4x4TB (RAID5) QTS 5.0.1
TS-410 with 4x3TB (RAID5) QTS 4.2.6
TS-412 with 4x2TB (Singles) QTS 4.2.6
TS-410 with 4x3TB (RAID5) QTS 4.2.6
TS-412 with 4x2TB (Singles) QTS 4.2.6
- dolbyman
- Guru
- Posts: 35253
- Joined: Sat Feb 12, 2011 2:11 am
- Location: Vancouver BC , Canada
Re: FSCK on TS-410 still running after 38 hours
are the disks nas disks ? If not, missing TLER would slow everything to a crawl
pop the busted disk out and see if it comes back to normal speeds
hope you have backups
pop the busted disk out and see if it comes back to normal speeds
hope you have backups
-
- Starting out
- Posts: 17
- Joined: Sun Jun 15, 2014 2:04 pm
Re: FSCK on TS-410 still running after 38 hours
Thanks for reply!
Is this really the only hint/possibility?
Ok, let me update you with following additional informations:
Yes, these are NAS disks - HDS723030ALA640
And yes, there is a backup (an a TS-412).
And/But - how can it be otherwise - not everything is rsynced as I decided meanwhile 7 years ago but now it would be a painful loss of data
Thanks for any further hint!
BR,
pah96
Is this really the only hint/possibility?
Ok, let me update you with following additional informations:
Code: Select all
[~] # cat /proc/mdstat
Personalities : [raid1] [linear] [raid0] [raid10] [raid6] [raid5] [raid4]
md0 : active raid5 sda3[0] sdd3[3] sdc3[2] sdb3[1]
8786092608 blocks super 1.0 level 5, 64k chunk, algorithm 2 [4/4] [UUUU]
[~] # cat /etc/mdadm.conf
ARRAY /dev/md0 devices=/dev/sda3,/dev/sdb3,/dev/sdc3,/dev/sdd3
[~] # mdadm -Q /dev/md0
/dev/md0: 8379.07GiB raid5 4 devices, 0 spares. Use mdadm --detail for more detail.
[~] # mdadm -D /dev/md0
/dev/md0:
Version : 01.00.03
Creation Time : Fri Jun 13 18:56:43 2014
Raid Level : raid5
Array Size : 8786092608 (8379.07 GiB 8996.96 GB)
Used Dev Size : 2928697536 (2793.02 GiB 2998.99 GB)
Raid Devices : 4
Total Devices : 4
Preferred Minor : 0
Persistence : Superblock is persistent
Update Time : Sat May 8 15:35:31 2021
State : clean
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 64K
Name : 0
UUID : 63ed0a23:5bc99009:aa4a639d:1215a52b
Events : 770265
Number Major Minor RaidDevice State
0 8 3 0 active sync /dev/sda3
1 8 19 1 active sync /dev/sdb3
2 8 35 2 active sync /dev/sdc3
3 8 51 3 active sync /dev/sdd3
And yes, there is a backup (an a TS-412).
And/But - how can it be otherwise - not everything is rsynced as I decided meanwhile 7 years ago but now it would be a painful loss of data
Thanks for any further hint!
BR,
pah96
TS-451+ with 4x4TB (RAID5) QTS 5.0.1
TS-410 with 4x3TB (RAID5) QTS 4.2.6
TS-412 with 4x2TB (Singles) QTS 4.2.6
TS-410 with 4x3TB (RAID5) QTS 4.2.6
TS-412 with 4x2TB (Singles) QTS 4.2.6
- dolbyman
- Guru
- Posts: 35253
- Joined: Sat Feb 12, 2011 2:11 am
- Location: Vancouver BC , Canada
Re: FSCK on TS-410 still running after 38 hours
Cant find any NAS specification for the disk model...says deskstar (not deskstar nas)
the mdstat only says that the raid has not failed yet..not if it is busy waiting
I cannot decipher the rest of your post
the mdstat only says that the raid has not failed yet..not if it is busy waiting
I cannot decipher the rest of your post
-
- Starting out
- Posts: 17
- Joined: Sun Jun 15, 2014 2:04 pm
Re: FSCK on TS-410 still running after 38 hours
Indeed it is some days ago but you can find it still here:
https://www.qnap.com/en/compatibility/? ... category=1
For sure, deskstar not "nasstar", that's right. Not even Enterprise. But recommended as well.
But the only question was:
How can I check the real status of the fsck process - the webui isn't reliable.
Thanks a lot!
pah96
BTW: in hope it has finished meanwhile: don't really know but still no new log entrys
https://www.qnap.com/en/compatibility/? ... category=1
For sure, deskstar not "nasstar", that's right. Not even Enterprise. But recommended as well.
But the only question was:
How can I check the real status of the fsck process - the webui isn't reliable.
Thanks a lot!
pah96
BTW: in hope it has finished meanwhile: don't really know but still no new log entrys
TS-451+ with 4x4TB (RAID5) QTS 5.0.1
TS-410 with 4x3TB (RAID5) QTS 4.2.6
TS-412 with 4x2TB (Singles) QTS 4.2.6
TS-410 with 4x3TB (RAID5) QTS 4.2.6
TS-412 with 4x2TB (Singles) QTS 4.2.6
- dolbyman
- Guru
- Posts: 35253
- Joined: Sat Feb 12, 2011 2:11 am
- Location: Vancouver BC , Canada
Re: FSCK on TS-410 still running after 38 hours
try e2fsck with -c flag on the md0 device
-
- Starting out
- Posts: 17
- Joined: Sun Jun 15, 2014 2:04 pm
Re: FSCK on TS-410 still running after 38 hours
Update after 60hrs with status 0.0%
doesn't change on anytime.
I guess there is nothing going on any more - am I right?
I'm really surprised how less number of tools on qnap (NAS!) are available specially for disk mgmt.
br,
pah96
Code: Select all
[~] # iostat -Dx
extended device statistics
device mgr/s mgw/s r/s w/s kr/s kw/s size queue wait svc_t %b
mtdblock4 0 0 0.0 0.0 0.0 0.0 16.0 0.0 40.0 40.0 0
mtdblock5 0 0 0.0 0.0 0.0 0.0 3.9 0.0 11.4 11.4 0
sda 1 1 1.2 0.5 17.0 6.1 13.2 0.2 112.7 7.0 1
sdb 1 1 1.1 0.5 13.7 8.5 13.9 0.2 113.4 7.4 1
sdc 0 0 0.6 0.3 11.3 2.7 15.3 0.0 29.8 10.2 1
sdd 0 0 0.6 0.3 11.7 2.7 15.9 0.0 20.7 9.5 1
sds 0 0 0.1 0.0 0.9 0.0 6.2 0.0 11.5 2.3 0
md9 0 0 0.1 0.0 0.8 0.2 8.9 0.0 0.0 0.0 0
md13 0 0 0.9 0.4 20.3 1.6 16.4 0.0 0.0 0.0 0
md4 0 0 1.5 0.8 6.2 3.4 4.0 0.0 0.0 0.0 0
md0 0 0 2.6 0.1 18.4 0.8 7.1 0.0 0.0 0.0 0
sdwa 0 0 0.0 0.0 0.1 0.0 13.7 0.0 5.2 0.9 0
I guess there is nothing going on any more - am I right?
should/can this be done without stopping any process or rebooting?e2fsck -c # Check for bad blocks and add them to the badblock list
I'm really surprised how less number of tools on qnap (NAS!) are available specially for disk mgmt.
br,
pah96
TS-451+ with 4x4TB (RAID5) QTS 5.0.1
TS-410 with 4x3TB (RAID5) QTS 4.2.6
TS-412 with 4x2TB (Singles) QTS 4.2.6
TS-410 with 4x3TB (RAID5) QTS 4.2.6
TS-412 with 4x2TB (Singles) QTS 4.2.6
-
- Starting out
- Posts: 17
- Joined: Sun Jun 15, 2014 2:04 pm
Re: FSCK on TS-410 still running after 38 hours
Update:
after 264 hours nothing changed, nothing new.
after 264 hours nothing changed, nothing new.
TS-451+ with 4x4TB (RAID5) QTS 5.0.1
TS-410 with 4x3TB (RAID5) QTS 4.2.6
TS-412 with 4x2TB (Singles) QTS 4.2.6
TS-410 with 4x3TB (RAID5) QTS 4.2.6
TS-412 with 4x2TB (Singles) QTS 4.2.6