Performance issues using 6-drive RAID on a TS-873

Questions about SNMP, Power, System, Logs, disk, & RAID.
robske_110
New here
Posts: 8
Joined: Fri Sep 06, 2019 12:53 am

Re: Performance issues using 6-drive RAID on a TS-873

Post by robske_110 » Fri Sep 06, 2019 5:01 pm

Bob Zelin wrote:
Fri Sep 06, 2019 6:54 am
it sounds like you have a Win 10 PC with a thunderbolt 3 port, and you are using the OWC T3 to 10G adaptor. I have no idea about your PC. If I am wrong, and you are using a Mac (you just said about the macOS SMB client) -
make sure to have SMB signing disabled in terminal, and reboot your machine. And enable Jumbo Frames in both System Preferences for your 10G port, and on the QNAP 10G port.
Why are you only using 6 drives in a TVS-873 - put eight drives in there. If you follow my instructions, you will get 800 MB/sec (and probably not more than 500 MB/sec write).
And stop staring at the specs on the QNAP website (2300 MB/sec) - I have no idea of what they are referring to. With a 10G connection, even theoretical bandwidth can never exceed 1250 MB/sec, and in real life, I have never
seen over 1100 MB/sec on ANY 10G client - no matter what. 2300 MB/sec probably means total aggregate bandwidth - and there is no way that a TVS-873 is going to perform like this. There is real world, and there is fantasy.

Bob Zelin
I have a MacBook Pro with macOS on it. I am going to get my windows boot drive to work again today, and try it with that.
I am of course not expecting the QNAP website specs - they used a RAID5 with 8 ssds and a dual 10GbE link (link aggregation) [And with that, their speeds are actually possible]. But since, the RAID controller seems to be able to do that, and since internal speed tests reports speeds of around 10GB/s on my setup currently, I was simply expecting to get ~900MB/s read on my client machine, especially since the 10G connection appears to work correctly (9.88Gbits/s). Jumbo frames are enabled and SMB signing is off.
I actually am kinda fine with 550 and 400, but today noticed speeds dropping significantly when the qnap cpu is under any load. Did I simply buy an underpowered NAS? (When running a Virtual Machine on another Volume (M.2 SATA SSD) speeds drop to 220 read and 200 write, which is quite disappointing). I think I was just misled by the specs on the qnap website, thinking this NAS is ok for me since they showed it able to do way over 10Gbit/s.
If what I want is impossible with this NAS, could you recommend me a NAS model that CAN actually reach 10Gbit/s with a 6-drive RAID5? And also, I see no reason to use a RAID6 for stuff that is backed up somewhere else anyway. RAID6 is still not a backup.

User avatar
MrVideo
Experience counts
Posts: 4654
Joined: Fri May 03, 2013 2:26 pm

Re: Performance issues using 6-drive RAID on a TS-873

Post by MrVideo » Fri Sep 06, 2019 5:55 pm

robske_110 wrote:
Fri Sep 06, 2019 5:01 pm
I see no reason to use a RAID6 for stuff that is backed up somewhere else anyway. RAID6 is still not a backup.
Correct, RAID6 is not a backup. Never said that it was. The point is that RAID6 will help shield you from total loss of the NAS configuration. If two drives were to fail under RAID5, you've lost everything and will have to rebuild the RAID from scratch and then restore everything. On the other hand, with RAID6 you have to lose three drives before this happens.

If, during a rebuild with RAID5, you lose another drive while the rebuild is occurring, you are toast. But, with RAID6, you can lose another drive and get the rebuild to complete.

While this isn't perfect, you stand a better chance of not having to spend lots of time rebuilding your NAS and restoring data.

That is the point of RAID6.
QTS 4.1.n/4.2.n/4.3.n MANUAL
Submit QNAP Support Ticket - QNAP Tutorials, FAQs, Downloads, Wiki - Product Support Status - Moogle's QNAP FAQ help V2
When you ask a question, please include the following
(Thanks to Toxic17 for the links)
QNAP md_checker nasreport (release 20180525)
===============================
Model: TS-869L -- RAM: 3G -- FW: QTS 4.1.4 Build 20150522 (used for data storage)
WD60EFRX-68L0BN1(x1)/68MYMN1(x7) Red HDDs -- RAID6: 8x6TB -- Cold spare: 1x6TB
Entware
===============================
Model: TS-451A -- RAM: 2G -- FW: QTS 4.2.3 Build 20170213 (used as a video server)
WL3000GSA6472(x3) White label NAS HDDs -- RAID5: 3x3TB
Entware -- MyKodi 17.3 (default is Kodi 16)
===============================
My 2017 Total Solar Eclipse Photos

User avatar
spikemixture
Been there, done that
Posts: 727
Joined: Wed Mar 07, 2018 11:04 pm
Location: 3rd World

Re: Performance issues using 6-drive RAID on a TS-873

Post by spikemixture » Fri Sep 06, 2019 6:13 pm

MrVideo wrote:
Fri Sep 06, 2019 10:59 am
Trexx wrote:
Fri Sep 06, 2019 5:02 am
He had stated previously he had tried raid-5/6 and was using raid-0 to test parity overhead impact on his low perf #'s.
Not that I saw. Only saw a RAID5 comment in the first post.
Really you 2 ? Who cares? :roll:
Get on with the discussion..
Qnap TS-1277 1700 6x8TB WD White, 2x8TB WD Red - Raid5,, 2x M.2 Crucial 1TB (Raid 1 VM),
2x SSD 860 EVO 500gb (Raid1 QTS), 2x SSD 860 EVO 250GB (Cache), 2x M.2 PCIe 970 500gb NVME (Raid1 Plex and Emby server)
Qnap TVS-1282 i7 - 8x10TB WD White - Raid 6, 2x M.2 Crucial 500gb (Raid1 VM),
2x SSD EVO 500gb (Raid1 QTS), 2x SSD EVO 250gb (Raid1 Cache), 2x M.2 PCIe Intel 512GB NVME (Raid1-Servers)
Synology -1817+ - 2x10,2x8,2x6 (Ironwolf) 2x6 (WD Red) - SHR
Drobo 5n - 3x2TB WD Green, 2x3TB WD Red - Drobo Raid
Buffalo LinkStation - 2x2TB WD
ProBox 8 Bay USB3 - 8x4TB Seagate - JBOD
All software is updated as soon as it's available.
I give my opinion from my experience i.e. I have (or had) that piece of equipment/software and used it! :roll:

robske_110
New here
Posts: 8
Joined: Fri Sep 06, 2019 12:53 am

Re: Performance issues using 6-drive RAID on a TS-873

Post by robske_110 » Fri Sep 06, 2019 8:05 pm

I now wanted to try this with Windows, but on windows I can't seem to get the 10GbE card to perform the way it should be.
I already tried setting the Jumbo packet size to 9000, is there anything else I need to do?
iperf is only reporting 2.31Gbit/s.
Capture.PNG
You do not have the required permissions to view the files attached to this post.

Bob Zelin
Been there, done that
Posts: 621
Joined: Mon Nov 21, 2016 12:55 am
Location: Orlando, FL.
Contact:

Re: Performance issues using 6-drive RAID on a TS-873

Post by Bob Zelin » Fri Sep 06, 2019 8:13 pm

Mr Robske writes -
If what I want is impossible with this NAS, could you recommend me a NAS model that CAN actually reach 10Gbit/s with a 6-drive RAID5?

This is what drives me crazy about QNAP. They have TOO MANY MODELS, and it's too confusing to figure out the specs on what will - and what will not - perform.
I repeat this exact line over and over on this forum. People ask me all the time "how do you know all this stuff". I DO NOT know all this stuff. I just install a lot of QNAP
products, and some work great, and some work terribly. The TS-873 is ok, but it's not great. Would any model 6 bay that I recommend work to your spec ? I don't know.
I can tell you that if you have the TVS-872XT, that is a GREAT QNAP. You put eight 7200 RPM drives in there, and you will get your 1000 MB/sec on your PC, and about 800 - 900 MB/secREAD on your Mac (write will be slower on the Mac - I don't know why).

SO - will the TVS-672XT do the same job for you ? I don't know. Never tried it. I have tried plenty of 4 and 6 bay QNAP's (like the TS-453BT3) - and it's slow - like 450 MB/sec slow.

Bob Zelin
Bob Zelin / Rescue 1, Inc.
http://www.bobzelin.com

robske_110
New here
Posts: 8
Joined: Fri Sep 06, 2019 12:53 am

Re: Performance issues using 6-drive RAID on a TS-873

Post by robske_110 » Fri Sep 06, 2019 8:36 pm

Bob Zelin wrote:
Fri Sep 06, 2019 8:13 pm
Mr Robske writes -
If what I want is impossible with this NAS, could you recommend me a NAS model that CAN actually reach 10Gbit/s with a 6-drive RAID5?

This is what drives me crazy about QNAP. They have TOO MANY MODELS, and it's too confusing to figure out the specs on what will - and what will not - perform.
I repeat this exact line over and over on this forum. People ask me all the time "how do you know all this stuff". I DO NOT know all this stuff. I just install a lot of QNAP
products, and some work great, and some work terribly. The TS-873 is ok, but it's not great. Would any model 6 bay that I recommend work to your spec ? I don't know.
I can tell you that if you have the TVS-872XT, that is a GREAT QNAP. You put eight 7200 RPM drives in there, and you will get your 1000 MB/sec on your PC, and about 800 - 900 MB/secREAD on your Mac (write will be slower on the Mac - I don't know why).

SO - will the TVS-672XT do the same job for you ? I don't know. Never tried it. I have tried plenty of 4 and 6 bay QNAP's (like the TS-453BT3) - and it's slow - like 450 MB/sec slow.

Bob Zelin
Than you for your insights!
The TVS-872XT is already so expensive that I am cheaper off with buying an mac mini (even with built-in 10GbE port) and a thunderbay6... So that won't work for me.

I was now able to get read speeds of around 1000MB/s on windows, and write speeds around 450MB/s. Still disappointed at the write speeds, but I am happy that I can read really well now.

User avatar
Trexx
Experience counts
Posts: 4938
Joined: Sat Oct 01, 2011 7:50 am
Location: Minnesota
Contact:

Re: Performance issues using 6-drive RAID on a TS-873

Post by Trexx » Fri Sep 06, 2019 11:36 pm

I have seen similar perf deltas in the past between Windows using SMB vs. Mac SMB to my x73 box.
Paul

Model: TS-877-1600 FW: 4.4.1.x
QTier (HDD): [RAID-5] 6 x 3TB HGST DeskStar NAS QTier (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
(SSD): [RAID-1] 2 x 500GB Evo 860
RAM: Kingston HyperX Fury 64GB DDR4-2666
GPU: EVGA GTX 1060, ACX 2.0(1 Fan), 6GB
UPS: CyberPower AVR1350 Ext. Backup: USB 3.0 Seagate 5TB
Media Boxes: Nvidia ShieldTV Pro, AppleTV 4k, Various Roku's

Model: TVS-673 32GB FW: 4.4.1.x Test/Backup Box
-----------------------------------------------------------------------------------------------------------------------------------------
NAS RAID Rebuild Times | Live QTS Videos | | QNAP NAS Guide | Information needed when you ask for HELP | QNAP Links, Tutorials, etc.
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq

Bob Zelin
Been there, done that
Posts: 621
Joined: Mon Nov 21, 2016 12:55 am
Location: Orlando, FL.
Contact:

Re: Performance issues using 6-drive RAID on a TS-873

Post by Bob Zelin » Sat Sep 07, 2019 5:06 am

since you are getting good speeds on your TS-873, let me tell you my experience, which is similar to what Trexx is seeing.
I install a lot of TVS-872XT systems. With a Mac, using the QNAP QNA-T3-10G1T, I get between 800 - 900 MB/sec READ and about 500 - 550 MB/sec WRITE.
If I put a QNAP QXG-10G1T into a x4 lane slot in a good Win 10 PC, I get 1000 - 1100 MB/sec READ and about 550 MB/sec WRITE.

If I take these same computers, and put them on a QNAP TS-1677X or TS-1685 (into the native 10G port, with the same 10G switch) - my WRITE speed goes up to about 800 - 900 MB/sec.
Why ? I have no idea, and I have no interest in torturing QNAP on the theoretical limits. Do you know why ?
Because I can take a 2012 Mac Pro, running macOS 10.13.6 with a QNAP QXG-10G1T, and plug it into a TS-1685, and I will get 900 MB/sec READ AND WRITE. I plug a brand new iMac Pro into the same TS-1685
using the native 10G chip in the very expensive iMac Pro (which uses the Aquantia AQC-107 - same chip as QNAP uses for many of these products, and all their new 10G adaptors) - and I get 900 - 1000 MB/sec READ, but
only about 300 - 350 MB/sec write. I have seen this over and over again. Which is why I always try to convince my customers with iMac Pro's to STILL purchase a QNAP QNA-T310G1T or Sonnet Solo 10G T3, instead of
the native port.
WHY ? I have no idea. And I certainly can't call up Apple and start to complain to them - because they will say "bring it to the genius bar" - and the "genius bar" will find nothing wrong (because they are just
plugging the 10G port of the iMac Pro into the internet, and the kid says "looks like it works fine to me".
This is what REAL LIFE testing is about. Some products work better than others. I think they call this "clinical trials" in the medical field. Some drug comes out with a drug that cures, XYZ, they convince doctors
that it's safe - the doctor gives it to you and all his patients, and then YOU get sick, but the others are cured. WHY ? I don't know - and neither does the doctor (and neither does the pharmaceutical company, until
they get sued, and start to research it).

AND if you are a Mac user - well, just wait until macOS 10.15 Catalina comes out later this year - then everything will stop working.

Bob Zelin
Bob Zelin / Rescue 1, Inc.
http://www.bobzelin.com

auvovee
First post
Posts: 1
Joined: Wed Jul 24, 2019 2:09 pm

Re: Performance issues using 6-drive RAID on a TS-873

Post by auvovee » Mon Sep 09, 2019 1:36 am

Hi folks!

I got my new TS-873 a week ago and I've been testing it.
There is one static volume: Raid 6 - 5x WD Red Pro 6TB. No SSDs. Here are the test results.

clip====

[~] # qcli_storage -d
Enclosure Port Sys_Name Type Size Alias Signature Partitions Model
NAS_HOST 3 /dev/sdc HDD:data 5.46 TB Disk 1 QNAP FLEX 5 WDC WD6003FFBX-68MU3N0
NAS_HOST 4 /dev/sdb HDD:data 5.46 TB Disk 2 QNAP FLEX 5 WDC WD6003FFBX-68MU3N0
NAS_HOST 5 /dev/sde HDD:data 5.46 TB Disk 3 QNAP FLEX 5 WDC WD6003FFBX-68MU3N0
NAS_HOST 6 /dev/sdd HDD:data 5.46 TB Disk 4 QNAP FLEX 5 WDC WD6003FFBX-68MU3N0
NAS_HOST 7 /dev/sda HDD:data 5.46 TB Disk 5 QNAP FLEX 5 WDC WD6003FFBX-68MU3N0
[~] # qcli_storage -T force=1
fio test command for physical disk: /sbin/fio --filename=test_device --direct=1 --rw=read --bs=1M --runtime=15 --name=test-read --ioengine=libaio --iodepth=32 &>/tmp/qcli_storage.log
fio test command for RAID: /sbin/fio --filename=test_device --direct=0 --rw=read --bs=1M --runtime=15 --name=test-read --ioengine=libaio --iodepth=32 &>/tmp/qcli_storage.log
Start testing!
Performance test is finished 100.000%...
Enclosure Port Sys_Name Throughput RAID RAID_Type RAID_Throughput Pool
NAS_HOST 3 /dev/sdc 240.83 MB/s /dev/md1 RAID 6 669.53 MB/s 288
NAS_HOST 4 /dev/sdb 239.21 MB/s /dev/md1 RAID 6 669.53 MB/s 288
NAS_HOST 5 /dev/sde 240.56 MB/s /dev/md1 RAID 6 669.53 MB/s 288
NAS_HOST 6 /dev/sdd 254.51 MB/s /dev/md1 RAID 6 669.53 MB/s 288
NAS_HOST 7 /dev/sda 252.91 MB/s /dev/md1 RAID 6 669.53 MB/s 288
[~] # qcli_storage -t force=1
fio test command for LV layer: /sbin/fio --filename=test_device --direct=0 --rw=read --bs=1M --runtime=15 --name=test-read --ioengine=libaio --iodepth=32 &>/tmp/qcli_storage.log
fio test command for File system: /sbin/fio --filename=test_device/qcli_storage --direct=0 --rw=read --bs=1M --runtime=15 --name=test-read --ioengine=libaio --iodepth=32 --size=128m &>/tmp/qcli_storage.log
Start testing!
Performance test is finished 100.000%...
VolID VolName Pool Mapping_Name Throughput Mount_Path FS_Throughput
1 DataVol1 288 /dev/mapper/cachedev1 689.29 MB/s /share/CACHEDEV1_DATA 563.88 MB/s

clip====


I have an old Asus i7/16GB/SSD-drive/1Gb lan desktop computer with Windows 10 and I can get maximum read ~980Mbits/s and write ~950Mbits/s when copying big Blueray files. When copying photo files I can get maximum read ~600Mbit/s and write ~500Mbit/s.

I have already ordered 10Gb-cards to my PC and QNAP and planning to buy three more HDDs. Let's see how this madness is going to end. ;-)
BackUp 2. : TS-439 Pro II - 2GB - 4.2.6 - Raid6 / 4x WDRed 3TB
BackUp 1. : TS-420 - 500MB- 4.3.3 - Raid6 / 4x WDRed 3TB
WorkDir : TS-873 - 12GB - 4.3.6 - Raid6 / 5x WDRed Pro 6TB

Post Reply

Return to “System & Disk Volume Management”