Slow internal write/read performance with thick volume in RAID 5 storage pool

[somy1982]

Hi,
Recently I noticed read/write performance decrease, initially I thought it was SMB or network problem, but now it looks like some file system issues. I have bad performance on all my thick volumes (unencrypted) created in a RAID 5 storage pool with 4X4TB HDDs (I know I should avoid RAID 5).
I tried the following to test the speed, all uncessary services/apps were disabled when I test:
1) qcli_storage -T for disk speed, which looks fine:

Enclosure Port Sys_Name Throughput RAID RAID_Type RAID_Throughput Pool

NAS_HOST 1 /dev/sdd 139.10 MB/s /dev/md1 RAID 5 226.29 MB/s 1

NAS_HOST 2 /dev/sdc 139.51 MB/s /dev/md1 RAID 5 226.29 MB/s 1

NAS_HOST 3 /dev/sdb 121.89 MB/s /dev/md1 RAID 5 226.29 MB/s 1

NAS_HOST 4 /dev/sda 112.86 MB/s /dev/md1 RAID 5 226.29 MB/s 1


2) qcli_storage -t for file system speed:
VolID VolName Pool Mapping_Name Throughput Mount_Path FS_Throughput

1 SYSTEM 1 /dev/mapper/cachedev1 65.11 MB/s /share/CACHEDEV1_DATA 57.04 MB/s

3 BACKUP 1 /dev/mapper/cachedev3 50.26 MB/s /share/CACHEDEV3_DATA 57.74 MB/s

I have tried the following without help:
1) Scanned HDDs, no problem
2) Run through a RAID scrubbing
3) Checked file systems of the volumes


What could go wrong? Any suggestion is appreciated!

[dolbyman]

what nas model?
what firmware and build?
why would you want to avoid raid5?
what is the cpu usage during benchmarks?

[somy1982]

what nas model?
what firmware and build?
why would you want to avoid raid5?
what is the cpu usage during benchmarks?

Sorry didn't provide all info.
The NAS is TS-453mini based on X86 J1900 CPU, and running with 16G memory.
The firmware is 4.3.6 the latest official one.
I read the problem with RAID 5/URE, so if I could do it again I would avoid RAID 5 (nothing related to the problem).
CPU usage is between 10% -30% during test.

[dolbyman]

please state the build as there was two recent recalled builds

so you would go with raid6 then ? (curiosity)

[somy1982]

please state the build as there was two recent recalled builds

so you would go with raid6 then ? (curiosity)

It is 4.3.6.1050
I would simply go for RAID 1 + single SSD (not cache)

[dolbyman]

that one was recalled .. best to downgrade

viewtopic.php?f=142&t=150355

not sure where all the fear for raid levels 5 and 6 come from

I run RAID6 for dual parity with comprehensive backups...all good

[Toxic17]

what Model of Hard disk are you running?

Code: Select all

qcli_storage -d

[somy1982]

what Model of Hard disk are you running?

Code: Select all

qcli_storage -d

I have a mix of HDDs from different manufactures:

Enclosure Port Sys_Name Type Size Alias Signature Partitions Model
NAS_HOST 1 /dev/sdd HDD:data 3.64 TB -- QNAP FLEX 5 WDC WD4001FFSX-68JNUN0
NAS_HOST 2 /dev/sdc HDD:data 3.64 TB -- QNAP FLEX 5 Seagate ST4000VN008-2DR166
NAS_HOST 3 /dev/sdb HDD:data 3.64 TB -- QNAP FLEX 5 HGST HDN724040ALE640
NAS_HOST 4 /dev/sda HDD:data 3.64 TB -- QNAP FLEX 5 HGST HDN724040ALE640

Recall the performance was Ok initially.

[somy1982]

that one was recalled .. best to downgrade

viewtopic.php?f=142&t=150355

not sure where all the fear for raid levels 5 and 6 come from

I run RAID6 for dual parity with comprehensive backups...all good

Looks like it's pulled due to SMB issues and the performance problem I have now was there for a few months.
I will downgrade anyways to be sure.

[somy1982]

Downgraded to 0933 same problem

[Toxic17]

the 3 drive manufactured models have different spindle speeds.

WD 5400 RPM
Seagate: 5900 RPM
HGST: 7200 RPM.

this is not an ideal setup. whether this has a knock on affect in disk access I dont know, but I would guess its not helping matters when disk data is being accessed across multiple speed drives in one RAID5 array.

As to you claims of the performance being "Ok" initially, was this months or years ago? and what speed if any was "ok"? was their much data on the drives then?

you might get better performance putting the two HGST drives in RAID1 since they are the same spec drive. the other two are less of a difference in speed and you might get away with another RAID1, but putting them all in the same Raid Array will in my eyes cause throughput issues.

[somy1982]

the 3 drive manufactured models have different spindle speeds.

WD 5400 RPM
Seagate: 5900 RPM
HGST: 7200 RPM.

this is not an ideal setup. whether this has a knock on affect in disk access I dont know, but I would guess its not helping matters when disk data is being accessed across multiple speed drives in one RAID5 array.

As to you claims of the performance being "Ok" initially, was this months or years ago? and what speed if any was "ok"? was their much data on the drives then?

you might get better performance putting the two HGST drives in RAID1 since they are the same spec drive. the other two are less of a difference in speed and you might get away with another RAID1, but putting them all in the same Raid Array will in my eyes cause throughput issues.

True, two of them are 5400RPM and the other two are 7200RPM. Maybe it's time to upgrade......