Slow internal write/read performance with thick volume in RAID 5 storage pool
-
- Easy as a breeze
- Posts: 372
- Joined: Fri Apr 08, 2016 6:42 pm
Slow internal write/read performance with thick volume in RAID 5 storage pool
Hi,
Recently I noticed read/write performance decrease, initially I thought it was SMB or network problem, but now it looks like some file system issues. I have bad performance on all my thick volumes (unencrypted) created in a RAID 5 storage pool with 4X4TB HDDs (I know I should avoid RAID 5).
I tried the following to test the speed, all uncessary services/apps were disabled when I test:
1) qcli_storage -T for disk speed, which looks fine:
Enclosure Port Sys_Name Throughput RAID RAID_Type RAID_Throughput Pool
NAS_HOST 1 /dev/sdd 139.10 MB/s /dev/md1 RAID 5 226.29 MB/s 1
NAS_HOST 2 /dev/sdc 139.51 MB/s /dev/md1 RAID 5 226.29 MB/s 1
NAS_HOST 3 /dev/sdb 121.89 MB/s /dev/md1 RAID 5 226.29 MB/s 1
NAS_HOST 4 /dev/sda 112.86 MB/s /dev/md1 RAID 5 226.29 MB/s 1
2) qcli_storage -t for file system speed:
VolID VolName Pool Mapping_Name Throughput Mount_Path FS_Throughput
1 SYSTEM 1 /dev/mapper/cachedev1 65.11 MB/s /share/CACHEDEV1_DATA 57.04 MB/s
3 BACKUP 1 /dev/mapper/cachedev3 50.26 MB/s /share/CACHEDEV3_DATA 57.74 MB/s
I have tried the following without help:
1) Scanned HDDs, no problem
2) Run through a RAID scrubbing
3) Checked file systems of the volumes
What could go wrong? Any suggestion is appreciated!
Recently I noticed read/write performance decrease, initially I thought it was SMB or network problem, but now it looks like some file system issues. I have bad performance on all my thick volumes (unencrypted) created in a RAID 5 storage pool with 4X4TB HDDs (I know I should avoid RAID 5).
I tried the following to test the speed, all uncessary services/apps were disabled when I test:
1) qcli_storage -T for disk speed, which looks fine:
Enclosure Port Sys_Name Throughput RAID RAID_Type RAID_Throughput Pool
NAS_HOST 1 /dev/sdd 139.10 MB/s /dev/md1 RAID 5 226.29 MB/s 1
NAS_HOST 2 /dev/sdc 139.51 MB/s /dev/md1 RAID 5 226.29 MB/s 1
NAS_HOST 3 /dev/sdb 121.89 MB/s /dev/md1 RAID 5 226.29 MB/s 1
NAS_HOST 4 /dev/sda 112.86 MB/s /dev/md1 RAID 5 226.29 MB/s 1
2) qcli_storage -t for file system speed:
VolID VolName Pool Mapping_Name Throughput Mount_Path FS_Throughput
1 SYSTEM 1 /dev/mapper/cachedev1 65.11 MB/s /share/CACHEDEV1_DATA 57.04 MB/s
3 BACKUP 1 /dev/mapper/cachedev3 50.26 MB/s /share/CACHEDEV3_DATA 57.74 MB/s
I have tried the following without help:
1) Scanned HDDs, no problem
2) Run through a RAID scrubbing
3) Checked file systems of the volumes
What could go wrong? Any suggestion is appreciated!
- dolbyman
- Guru
- Posts: 35248
- Joined: Sat Feb 12, 2011 2:11 am
- Location: Vancouver BC , Canada
Re: Slow internal write/read performance with thick volume in RAID 5 storage pool
what nas model?
what firmware and build?
why would you want to avoid raid5?
what is the cpu usage during benchmarks?
what firmware and build?
why would you want to avoid raid5?
what is the cpu usage during benchmarks?
-
- Easy as a breeze
- Posts: 372
- Joined: Fri Apr 08, 2016 6:42 pm
Re: Slow internal write/read performance with thick volume in RAID 5 storage pool
Sorry didn't provide all info.
The NAS is TS-453mini based on X86 J1900 CPU, and running with 16G memory.
The firmware is 4.3.6 the latest official one.
I read the problem with RAID 5/URE, so if I could do it again I would avoid RAID 5 (nothing related to the problem).
CPU usage is between 10% -30% during test.
- dolbyman
- Guru
- Posts: 35248
- Joined: Sat Feb 12, 2011 2:11 am
- Location: Vancouver BC , Canada
Re: Slow internal write/read performance with thick volume in RAID 5 storage pool
please state the build as there was two recent recalled builds
so you would go with raid6 then ? (curiosity)
so you would go with raid6 then ? (curiosity)
-
- Easy as a breeze
- Posts: 372
- Joined: Fri Apr 08, 2016 6:42 pm
- dolbyman
- Guru
- Posts: 35248
- Joined: Sat Feb 12, 2011 2:11 am
- Location: Vancouver BC , Canada
Re: Slow internal write/read performance with thick volume in RAID 5 storage pool
that one was recalled .. best to downgrade
viewtopic.php?f=142&t=150355
not sure where all the fear for raid levels 5 and 6 come from
I run RAID6 for dual parity with comprehensive backups...all good
viewtopic.php?f=142&t=150355
not sure where all the fear for raid levels 5 and 6 come from
I run RAID6 for dual parity with comprehensive backups...all good
- Toxic17
- Ask me anything
- Posts: 6477
- Joined: Tue Jan 25, 2011 11:41 pm
- Location: Planet Earth
- Contact:
Re: Slow internal write/read performance with thick volume in RAID 5 storage pool
what Model of Hard disk are you running?
Code: Select all
qcli_storage -d
Regards Simon
Qnap Downloads
MyQNap.Org Repository
Submit a ticket • QNAP Helpdesk
QNAP Tutorials, User Manuals, FAQs, Downloads, Wiki
When you ask a question, please include the following
NAS: TS-673A QuTS hero h5.1.2.2534 • TS-121 4.3.3.2420 • APC Back-UPS ES 700G
Network: VM Hub3: 500/50 • UniFi UDM Pro: 3.2.9 • UniFi Network Controller: 8.0.28
USW-Aggregation: 6.6.61 • US-16-150W: 6.6.61 • 2x USW Mini Flex 2.0.0 • UniFi AC Pro 6.6.62 • UniFi U6-LR 6.6.62
UniFi Protect: 2.11.21/8TB Skyhawk AI • 3x G3 Instants: 4.69.55 • UniFi G3 Flex: 4.69.55 • UniFi G5 Flex: 4.69.55
Qnap Downloads
MyQNap.Org Repository
Submit a ticket • QNAP Helpdesk
QNAP Tutorials, User Manuals, FAQs, Downloads, Wiki
When you ask a question, please include the following
NAS: TS-673A QuTS hero h5.1.2.2534 • TS-121 4.3.3.2420 • APC Back-UPS ES 700G
Network: VM Hub3: 500/50 • UniFi UDM Pro: 3.2.9 • UniFi Network Controller: 8.0.28
USW-Aggregation: 6.6.61 • US-16-150W: 6.6.61 • 2x USW Mini Flex 2.0.0 • UniFi AC Pro 6.6.62 • UniFi U6-LR 6.6.62
UniFi Protect: 2.11.21/8TB Skyhawk AI • 3x G3 Instants: 4.69.55 • UniFi G3 Flex: 4.69.55 • UniFi G5 Flex: 4.69.55
-
- Easy as a breeze
- Posts: 372
- Joined: Fri Apr 08, 2016 6:42 pm
Re: Slow internal write/read performance with thick volume in RAID 5 storage pool
I have a mix of HDDs from different manufactures:Toxic17 wrote: ↑Mon Sep 16, 2019 11:15 pm what Model of Hard disk are you running?
Code: Select all
qcli_storage -d
Enclosure Port Sys_Name Type Size Alias Signature Partitions Model
NAS_HOST 1 /dev/sdd HDD:data 3.64 TB -- QNAP FLEX 5 WDC WD4001FFSX-68JNUN0
NAS_HOST 2 /dev/sdc HDD:data 3.64 TB -- QNAP FLEX 5 Seagate ST4000VN008-2DR166
NAS_HOST 3 /dev/sdb HDD:data 3.64 TB -- QNAP FLEX 5 HGST HDN724040ALE640
NAS_HOST 4 /dev/sda HDD:data 3.64 TB -- QNAP FLEX 5 HGST HDN724040ALE640
Recall the performance was Ok initially.
-
- Easy as a breeze
- Posts: 372
- Joined: Fri Apr 08, 2016 6:42 pm
Re: Slow internal write/read performance with thick volume in RAID 5 storage pool
Looks like it's pulled due to SMB issues and the performance problem I have now was there for a few months.dolbyman wrote: ↑Mon Sep 16, 2019 10:39 pm that one was recalled .. best to downgrade
viewtopic.php?f=142&t=150355
not sure where all the fear for raid levels 5 and 6 come from
I run RAID6 for dual parity with comprehensive backups...all good
I will downgrade anyways to be sure.
-
- Easy as a breeze
- Posts: 372
- Joined: Fri Apr 08, 2016 6:42 pm
Re: Slow internal write/read performance with thick volume in RAID 5 storage pool
Downgraded to 0933 same problem
- Toxic17
- Ask me anything
- Posts: 6477
- Joined: Tue Jan 25, 2011 11:41 pm
- Location: Planet Earth
- Contact:
Re: Slow internal write/read performance with thick volume in RAID 5 storage pool
the 3 drive manufactured models have different spindle speeds.
WD 5400 RPM
Seagate: 5900 RPM
HGST: 7200 RPM.
this is not an ideal setup. whether this has a knock on affect in disk access I dont know, but I would guess its not helping matters when disk data is being accessed across multiple speed drives in one RAID5 array.
As to you claims of the performance being "Ok" initially, was this months or years ago? and what speed if any was "ok"? was their much data on the drives then?
you might get better performance putting the two HGST drives in RAID1 since they are the same spec drive. the other two are less of a difference in speed and you might get away with another RAID1, but putting them all in the same Raid Array will in my eyes cause throughput issues.
WD 5400 RPM
Seagate: 5900 RPM
HGST: 7200 RPM.
this is not an ideal setup. whether this has a knock on affect in disk access I dont know, but I would guess its not helping matters when disk data is being accessed across multiple speed drives in one RAID5 array.
As to you claims of the performance being "Ok" initially, was this months or years ago? and what speed if any was "ok"? was their much data on the drives then?
you might get better performance putting the two HGST drives in RAID1 since they are the same spec drive. the other two are less of a difference in speed and you might get away with another RAID1, but putting them all in the same Raid Array will in my eyes cause throughput issues.
Regards Simon
Qnap Downloads
MyQNap.Org Repository
Submit a ticket • QNAP Helpdesk
QNAP Tutorials, User Manuals, FAQs, Downloads, Wiki
When you ask a question, please include the following
NAS: TS-673A QuTS hero h5.1.2.2534 • TS-121 4.3.3.2420 • APC Back-UPS ES 700G
Network: VM Hub3: 500/50 • UniFi UDM Pro: 3.2.9 • UniFi Network Controller: 8.0.28
USW-Aggregation: 6.6.61 • US-16-150W: 6.6.61 • 2x USW Mini Flex 2.0.0 • UniFi AC Pro 6.6.62 • UniFi U6-LR 6.6.62
UniFi Protect: 2.11.21/8TB Skyhawk AI • 3x G3 Instants: 4.69.55 • UniFi G3 Flex: 4.69.55 • UniFi G5 Flex: 4.69.55
Qnap Downloads
MyQNap.Org Repository
Submit a ticket • QNAP Helpdesk
QNAP Tutorials, User Manuals, FAQs, Downloads, Wiki
When you ask a question, please include the following
NAS: TS-673A QuTS hero h5.1.2.2534 • TS-121 4.3.3.2420 • APC Back-UPS ES 700G
Network: VM Hub3: 500/50 • UniFi UDM Pro: 3.2.9 • UniFi Network Controller: 8.0.28
USW-Aggregation: 6.6.61 • US-16-150W: 6.6.61 • 2x USW Mini Flex 2.0.0 • UniFi AC Pro 6.6.62 • UniFi U6-LR 6.6.62
UniFi Protect: 2.11.21/8TB Skyhawk AI • 3x G3 Instants: 4.69.55 • UniFi G3 Flex: 4.69.55 • UniFi G5 Flex: 4.69.55
-
- Easy as a breeze
- Posts: 372
- Joined: Fri Apr 08, 2016 6:42 pm
Re: Slow internal write/read performance with thick volume in RAID 5 storage pool
True, two of them are 5400RPM and the other two are 7200RPM. Maybe it's time to upgrade......Toxic17 wrote: ↑Tue Sep 17, 2019 1:41 am the 3 drive manufactured models have different spindle speeds.
WD 5400 RPM
Seagate: 5900 RPM
HGST: 7200 RPM.
this is not an ideal setup. whether this has a knock on affect in disk access I dont know, but I would guess its not helping matters when disk data is being accessed across multiple speed drives in one RAID5 array.
As to you claims of the performance being "Ok" initially, was this months or years ago? and what speed if any was "ok"? was their much data on the drives then?
you might get better performance putting the two HGST drives in RAID1 since they are the same spec drive. the other two are less of a difference in speed and you might get away with another RAID1, but putting them all in the same Raid Array will in my eyes cause throughput issues.