RAID 5 constant activity / no silence / QuTSHero

Discussion about hard drive spin down (standby) feature of NAS.
Mylek
New here
Posts: 2
Joined: Sat Nov 28, 2020 6:41 pm

Re: RAID 5 constant activity / no silence / QuTSHero

Post by Mylek »

The patch notes and download link for the latest QuTSHero update still work but are unlisted for some reason. Maybe it was by mistake or maybe there was a problem with the release. If you can't wait until it's relisted here are the links. As a disclaimer you may want to update your backups first and be aware that if there are issues downgrading to h4.5.3 or before isn't supported.
https://www.qnap.com/en/release-notes/q ... 1/20210825
https://download.qnap.com/Storage/QuTSh ... 4.1771.zip

mikael123456 wrote: Wed Sep 01, 2021 11:04 am . . . I still see some writes to md9 and md13, the "span all drives raid1" devices. . . .
As I understand it the main issue with h4.5.3/h4.5.2 comes from ZFS subsystem activity unrelated to md9 and md13. If you don't have the ZFS issue and then fail your spinning disks from md9/md13 it should allow them to spin down when idle. Note that the next time you reboot your system md9 and md13 will be rebuilt across all the drives and include them again. I've been doing this on h4.5.1 with two nvme drives in raid1 as my system pool.

Code: Select all

 SSH in and confirm which disks are part of md9 and md13:
[~] # cat /proc/mdstat
...
md13 : active raid1 sdf4[64] sde4[65] sdd4[66] sdc4[67] sdb4[68] sda4[69] sdg4[70] sdh4[71] nvme0n1p4[0] nvme1n1p4[1]
      458880 blocks super 1.0 [64/10] [UUUUUUUUUU______________________________________________________]
      bitmap: 1/1 pages [4KB], 65536KB chunk

md9 : active raid1 sdf1[64] sde1[65] sdd1[66] sdc1[67] sdb1[68] sda1[69] sdg1[70] sdh1[71] nvme0n1p1[0] nvme1n1p1[1]
      530048 blocks super 1.0 [64/10] [UUUUUUUUUU______________________________________________________]
      bitmap: 1/1 pages [4KB], 65536KB chunk

Fail the spinning disks (in my case all sata disks) leaving at least two (in my case nvme ssds) behind for redundancy:
mdadm /dev/md9 --fail /dev/sda1
mdadm /dev/md9 --fail /dev/sdb1
mdadm /dev/md9 --fail /dev/sdc1
mdadm /dev/md9 --fail /dev/sdd1
mdadm /dev/md9 --fail /dev/sde1
mdadm /dev/md9 --fail /dev/sdf1
mdadm /dev/md9 --fail /dev/sdg1
mdadm /dev/md9 --fail /dev/sdh1
mdadm /dev/md13 --fail /dev/sda4
mdadm /dev/md13 --fail /dev/sdb4
mdadm /dev/md13 --fail /dev/sdc4
mdadm /dev/md13 --fail /dev/sdd4
mdadm /dev/md13 --fail /dev/sde4
mdadm /dev/md13 --fail /dev/sdf4
mdadm /dev/md13 --fail /dev/sdg4
mdadm /dev/md13 --fail /dev/sdh4

After a few moments they will be dropped leaving only your ssds:

cat /proc/mdstat
md13 : active raid1 nvme0n1p4[0] nvme1n1p4[1]
      458880 blocks super 1.0 [64/2] [UU______________________________________________________________]
      bitmap: 1/1 pages [4KB], 65536KB chunk

md9 : active raid1 nvme0n1p1[0] nvme1n1p1[1]
      530048 blocks super 1.0 [64/2] [UU______________________________________________________________]
      bitmap: 1/1 pages [4KB], 65536KB chunk
authorleon
Getting the hang of things
Posts: 86
Joined: Tue Feb 28, 2012 4:08 am

Re: RAID 5 constant activity / no silence / QuTSHero

Post by authorleon »

Hello all,

Has anyone tried this?

QuTS hero h4.5.4.1813 build 20211006

https://www.qnap.com/en/release-notes/q ... 3/20211006


Thanks
User avatar
ragnarlb@gmx.com
Starting out
Posts: 10
Joined: Wed Aug 04, 2021 10:12 pm

Re: RAID 5 constant activity / no silence / QuTSHero

Post by ragnarlb@gmx.com »

I have a TVS-871 and my disks just started doing this as well. I do run a PLEX server which I do serve out on the internet via port forwarding. But I have had this PLEX server for years and it never did this. I disabled everything I possibly could to keep what I need running, got rid of Multimedia Console, etc.

My disk sound like they are in an endless loop of writing every second. This is annoying because the NAS is in my office but I'm also concerned about the added wear and tear it must be putting on my disks. They even spin like this when no one is accessing the PLEX server or my network disk share.

I am running RAID 6 with 8 disks of 6TB each. It never did this before.

I'm running firmware 5.0.0.1828. My NAS has 4GB of RAM and I have more than half of it free (according to Resource Monitor).

Is there a better way to drill down and see what is causing the disks to write like this constantly/nonstop?
User avatar
dolbyman
Guru
Posts: 35005
Joined: Sat Feb 12, 2011 2:11 am
Location: Vancouver BC , Canada

Re: RAID 5 constant activity / no silence / QuTSHero

Post by dolbyman »

Your NAS does not support QuTS and you have posted a sepperate topic anyways...why post in here ?
User avatar
ragnarlb@gmx.com
Starting out
Posts: 10
Joined: Wed Aug 04, 2021 10:12 pm

Re: RAID 5 constant activity / no silence / QuTSHero

Post by ragnarlb@gmx.com »

dolbyman wrote: Thu Oct 28, 2021 10:11 pm Your NAS does not support QuTS and you have posted a sepperate topic anyways...why post in here ?
I was in the wrong window.
authorleon
Getting the hang of things
Posts: 86
Joined: Tue Feb 28, 2012 4:08 am

Re: RAID 5 constant activity / no silence / QuTSHero

Post by authorleon »

Please can you let me know:

Hello all,

Has anyone tried this?

QuTS hero h4.5.4.1813 build 20211006

https://www.qnap.com/en/release-notes/q ... 3/20211006
authorleon
Getting the hang of things
Posts: 86
Joined: Tue Feb 28, 2012 4:08 am

Re: RAID 5 constant activity / no silence / QuTSHero

Post by authorleon »

Hello,

Has anyone tried this please: QTS 5.0.0.1891 build 20211221

Does it removed the 5 sec constant writes.

Thanks
kcon_at
First post
Posts: 1
Joined: Sat Jan 01, 2022 6:16 am

Re: RAID 5 constant activity / no silence / QuTSHero

Post by kcon_at »

Mine does that too. Very annoying. Going to fix this tomorrow.
User avatar
PIRATA!
Know my way around
Posts: 107
Joined: Sun Feb 28, 2016 12:05 am

Re: RAID 5 constant activity / no silence / QuTSHero

Post by PIRATA! »

Hello there.
Has anyone resolved this issue?
Ok my Raid 1 I have continuous hdd activity all day long, like if they are working constantly, even with computer turned off and nothing communicating with the Nas.
Would really like to lower their noise a bit..
Thank you.

Sent from my SM-G970F using Tapatalk

QNAP TS-251+ with 2x Seagate IronWolf 4Tb (ST4000VN008)
googleg
Getting the hang of things
Posts: 90
Joined: Sun Mar 14, 2010 2:20 pm

Re: RAID 5 constant activity / no silence / QuTSHero

Post by googleg »

Mylek wrote: Sun Sep 05, 2021 8:26 am

Code: Select all

cat /proc/mdstat
md13 : active raid1 nvme0n1p4[0] nvme1n1p4[1]
      458880 blocks super 1.0 [64/2] [UU______________________________________________________________]
      bitmap: 1/1 pages [4KB], 65536KB chunk

md9 : active raid1 nvme0n1p1[0] nvme1n1p1[1]
      530048 blocks super 1.0 [64/2] [UU______________________________________________________________]
      bitmap: 1/1 pages [4KB], 65536KB chunk
Hello,
Maybe I'm saying something stupid but wouldn't be better to run the mdadm command so that md9 and md13 are limited to 2 disks:

Code: Select all

sudo mdadm --grow /dev/md13 --raid-devices=2
sudo mdadm --grow /dev/md9 --raid-devices=2 
I did it on my NAS because I thought it would solve the constant disk access problem (it did not) and this configuration change is kept after reboot.
termiman
New here
Posts: 6
Joined: Sat Jul 03, 2021 4:25 pm

Re: RAID 5 constant activity / no silence / QuTSHero

Post by termiman »

I am having this problem running latest QutS 5 firmware, I have used "iotop" and "zpool iostat zpool1 1" to try and troubleshoot.
iotop shows basically no writes to the filesystem.
"zpool iostat zpool1 1" shows about 10MB writes every 5-10 second to the disk.
I noticed this and started looking into it because my SSDs are wearing alot faster then I expected. I calculated it's writing 86GB a day (or 1TB every 12 days) to a set of SSD that are storing 50GB of data.
Definitely something up with the firmware implementation, seems to be related to the ZFS side of things.
I have opened a ticket in relation to this hopefully we can get this solved.
jord42
First post
Posts: 1
Joined: Thu Jul 14, 2022 4:27 pm

Re: RAID 5 constant activity / no silence / QuTSHero

Post by jord42 »

such an informative thread, Thanks everyone
termiman
New here
Posts: 6
Joined: Sat Jul 03, 2021 4:25 pm

Re: RAID 5 constant activity / no silence / QuTSHero

Post by termiman »

After a back and forth with QNAP support and them remotely accessing my NAS to diagnose, they assured me this is normal behaviour (it is not).

The IO activity was significant, and continued to wear my SSDs at unusually high rate, another 1% of life was removed within 1 week (500GB Evo Plus NVME with only 50GB allocated the rest is over provisioned).
The problem seems to be caused by any program using many small writes (databases), this includes Qmail and any docker containers using databases.
I have now solved this problem by adjusting the zfs txg timeout from 5 seconds to 120 seconds by adding the following line to my startup script.

echo 120 > /sys/module/zfs/parameters/_vfs_zfs_txg_timeout

There is a slight increase in risk to data making this change, and you should read up and understand this before doing it yourself, but the daily writes to my system volume have been reduced from ~86GB down to ~5GB a day.

This is really only a stop-gap solution, as there is some sort of underlying problem with write amplification occurring with small transaction, a less than 1KB write is turning into 100KB-500KB write.
My understanding of ZFS is that the block size (ZFS recordsize) is only the maximum size of a logical block and it automatically sizes records as needed, and I tested this by reducing the record size to 4KB which made no difference to the constant write rate. So the only amplification would be related to the SSDs physical block size probably something like 8KB.
pandaMinor
New here
Posts: 5
Joined: Tue Aug 02, 2022 9:56 am

Re: RAID 5 constant activity / no silence / QuTSHero

Post by pandaMinor »

I assume QNAP has abandoned fixing this?

My SSDs also have worn down 1% in the 1 week I have owned this device -- which is not filling me with love.

I will try your suggestion (I am on latest - 5.0.0.2069) if this is not going to be repaired soon.
termiman
New here
Posts: 6
Joined: Sat Jul 03, 2021 4:25 pm

Re: RAID 5 constant activity / no silence / QuTSHero

Post by termiman »

pandaMinor wrote: Wed Aug 03, 2022 12:15 pm I assume QNAP has abandoned fixing this?

My SSDs also have worn down 1% in the 1 week I have owned this device -- which is not filling me with love.

I will try your suggestion (I am on latest - 5.0.0.2069) if this is not going to be repaired soon.
Open a support ticket with them. The more they have to deal with it through support, the more likely they are to address the problem. I provided my fix to them as part of the support ticket.
Locked

Return to “HDD Spin Down (HDD Standby)”