Speed comparison: comparing disk #, raid, CPU, encryption

Introduce yourself to us and other members here, or share your own product reviews, suggestions, and tips and tricks of using QNAP products.
qpio
Getting the hang of things
Posts: 61
Joined: Wed May 11, 2011 6:51 am

Re: Speed comparison: comparing disk #, raid, CPU, encryption

Post by qpio » Thu Oct 18, 2018 8:44 pm

a13antichrist wrote:
qpio wrote:(4) when considering RAID-10 In the numbers you can see that RAID-10 only gives a slight advantage when running on the Atom CPU. Also, note that only write speeds where compared. The striping advantage of RAID-10 is only effective when reading.


This is misleading. Well actually, not misleading, it's just incorrect. The striping advantage comes into play for writing. It's the mirroring and the reading from both mirrors simultaneously that makes the [primary] difference for read results. XOR 'penalty' is of course also relevant, but less so with more powerful processors, as we've seen.


Incorrect? Oh wait...

Thanks for correcting my incorrect use of the word 'striping' there. If you are writing 100 GB to a 10 disk RAID-10 setup, you will be writing 20 GB to each disk. When reading you only have to read back 10 GB from each disk. So the advantage when reading is the mirroring indeed, which means you only have to read back chunks of data from each disk in a cough cough striped fashion. ;-)

With RAID-5 and RAID-6 these numbers are different (those are left as an exercise to the reader..) but there will be more heavy work for the CPU (XOR work) as well. You seem to imply that XOR also plays a role with RAID-10 but I fail to see how. RAID-10 can be efficiently implemented with a set of DMA IO commands. The RAID-5 and RAID-6 setups create more load on the CPU, but have effectively less IO to the disks while retaining the same level of data security.

So in the era of faster CPU's and while considering a 8-bay QNAP, I see no practical use of RAID-10 except for in a few really edge cases. Your last remark ".. but less so with more powerful processors" is exactly the point I was trying to make.

a13antichrist wrote:So it should be very clear to see that the "RAID-10 performance myth" is anything -but- a myth. You just have to have your context right.


Did you mean to say "nothing" but a myth? That would be a bit strong, but close to where I am currently on this topic.

I have seen RAID-10 setups with 40-50 disks in a single array. After splitting them up into 3 separate RAID-6 setups (with hot spares) these systems have produced better everything: more reliable, more throughput, more total storage, faster response times, less impact of one application's IO on another application's IO.

Obviously there are still cases where RAID-10 shines. For instance, in very large scale data storage, where data is distributed redundantly over multiple nodes with software like glusterfs (FS-level) or ceph (LUN-level). On the node level, RAID-10 might just be the best fit.

I was focussing on typical QNAP home scenario's where RAID-10 shines, and I can imagine none. By all means, let anyone share their experiences so we can all learn. I think the most valuable contribution would be to show some experimental results, and then trying to understand the underlying mechanisms.
TS-872, TS-870 (needs repair - two slots no SATA power), TS-459 Pro II, TS-112

qpio
Getting the hang of things
Posts: 61
Joined: Wed May 11, 2011 6:51 am

Re: Speed comparison: comparing disk #, raid, CPU, encryption

Post by qpio » Thu Oct 18, 2018 8:48 pm

a13antichrist wrote:
dolbyman wrote:
Why would you want to retest the OP's results ? If you really want to .. invest some $$$ and test away


Because he only tested write performance whereas my claim is that read performance would show an advantage for R10. Besides, the whole point is that this thread is positioned to "refute the claim that R10 is faster than R5", which is just as misleading as the original "myth" that "R10 is faster". My only aim here was to point out that it is not as simple as "which is faster, which is slower", but that the context, usage, and characteristics of the arrays all play a role and there is no universal "faster" at all. Write Performance is not everything, and neither is Read Performance, and so it is disingenuous to test only one and then claim "the speed advantage is a myth".

That's all.


Agreed. I may be able to facilitate that because my TS-870 will have to be fully rebuilt anyway soon. If I find the time, I will do the testing and report back here.
TS-872, TS-870 (needs repair - two slots no SATA power), TS-459 Pro II, TS-112

User avatar
storageman
Ask me anything
Posts: 5511
Joined: Thu Sep 22, 2011 10:57 pm

Re: Speed comparison: comparing disk #, raid, CPU, encryption

Post by storageman » Fri Oct 19, 2018 4:58 pm

Not that old chestnut again!

Here's some figures:

4 Bay Synology Sequential writes
RAID 5
34359738368 bytes (34 GB) copied, 446.183 s, 77.0 MB/s
RAID 10
34359738368 bytes (34 GB) copied, 630.38 s, 54.5 MB/s

Same will follow with Qnap.

These are not SANs, all the figures you've seen on dual controller SANs and SAS disks do not apply.

RAID 10 on these boxes will give you better random IOPs but worse sequential performance. You choose...

P3R
Guru
Posts: 12689
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Speed comparison: comparing disk #, raid, CPU, encryption

Post by P3R » Thu Feb 14, 2019 3:37 pm

I wanted to contribute to this interesting thread so I went ahead and compared all possible RAID configurations while keeping everything else unchanged on a TS-1277-1600. It was a simple sequential write test done within the system only to avoid that other bottlenecking factors like network and client system performance influence the outcome. Therefore it was a pure measurement of the performance of the storage in the system. I did five runs on each configuration and noted the fastest and the slowest result. The disks are dated and very slow compared to modern disks so the absolute values aren't useful but at the same time, the slow disks decreases the risk that any any other NAS internal bottleneck affect the results. What is interesting is to compare the results between the different RAID levels. Please note that RAID 0 despite it's name isn't really RAID as the R in RAID stands for Redundancy, of which RAID 0 have none. Also RAID 50 and RAID 60 only begin to be useful from somewhere around 12 disks and was also tested only for completeness.

The tests was the same sequential write test the OP describe in the first post of this thread. The difference here was that below I write the best and worse result observed over 5 runs on an unencrypted array so I have no encrypted test results.

This is what I measured:

3 disks in raid-0 372.30 MB/s 368.69 MB/s
4 disks in raid-0 493.32 MB/s 473.42 MB/s
5 disks in raid-0 597.43 MB/s 565.24 MB/s
6 disks in raid-0 681.87 MB/s 646.31 MB/s
7 disks in raid-0 801.57 MB/s 703.12 MB/s
8 disks in raid-0 869.82 MB/s 825.39 MB/s

2 disks in raid-1 125.34 MB/s 122.46 MB/s

3 disks in raid-5 235.95 MB/s 232.82 MB/s
4 disks in raid-5 336.34 MB/s 330.79 MB/s
5 disks in raid-5 454.23 MB/s 428.74 MB/s
6 disks in raid-5 525.63 MB/s 506.12 MB/s
7 disks in raid-5 576.50 MB/s 512.61 MB/s
8 disks in raid-5 687.88 MB/s 603.68 MB/s

4 disks in raid-6 237.71 MB/s 229.19 MB/s
5 disks in raid-6 339.16 MB/s 324.55 MB/s
6 disks in raid-6 437.33 MB/s 422.79 MB/s
7 disks in raid-6 525.60 MB/s 495.07 MB/s
8 disks in raid-6 606.99 MB/s 589.82 MB/s

4 disks in raid-10 247.34 MB/s 245.06 MB/s
6 disks in raid-10 362.25 MB/s 356.38 MB/s
8 disks in raid-10 478.31 MB/s 472.27 MB/s

6 disks in raid-50 443.63 MB/s 427.94 MB/s
8 disks in raid-50 556.56 MB/s 503.41 MB/s

8 disks in raid-60 441.95 MB/s 418.45 MB/s
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

User avatar
storageman
Ask me anything
Posts: 5511
Joined: Thu Sep 22, 2011 10:57 pm

Re: Speed comparison: comparing disk #, raid, CPU, encryption

Post by storageman » Thu Feb 14, 2019 4:57 pm

Cool tests
How did you do this with dd or what?
I'd be interested to know what sequential reads were, after all NAS are more read than write focussed (you play that mp3 many times)
Random performance would be up with RAID 10.

Now where's Thisisnotmyname?!?!?

P3R
Guru
Posts: 12689
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Speed comparison: comparing disk #, raid, CPU, encryption

Post by P3R » Thu Feb 14, 2019 5:59 pm

storageman wrote:
Thu Feb 14, 2019 4:57 pm
How did you do this with dd or what?
Yes. Details are in the first post of the thread.
I'd be interested to know what sequential reads were, after all NAS are more read than write focussed (you play that mp3 many times)
The main argument for RAID 10 and against RAID 5/6 have for many years been that RAID 5 and RAID 6 have their "write penalty". That's what most people think is the big disadvantage of RAID 5 and RAID 6 and that's the reason most have for using RAID 10.

One example is in this thread, where the OP expected worse write speed when going from a 4-disk RAID 10 to a 6-disk RAID 6... :wink:

Also contrary to what most home users think, the RAID array is very rarely the bottleneck when streaming. Given a very fast (10 GbE end-to-end) network the RAID array may however be a bottleneck when writing media to the NAS or doing client backups so a write test is in my opinion more useful and interesting. So no I won't redo this to test read as well. It took me 2 weeks or so before I had the system up and running normally again after all this testing.
Random performance would be up with RAID 10.
Absolutely, but the focus of my testing was for the benefit of the +95 % home and SMB users in this forum that mainly have a sequential load.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

User avatar
storageman
Ask me anything
Posts: 5511
Joined: Thu Sep 22, 2011 10:57 pm

Re: Speed comparison: comparing disk #, raid, CPU, encryption

Post by storageman » Thu Feb 14, 2019 6:08 pm

Of course now you'll get some die hards saying you should have used twice as many drives to compare RAID 10 to RAID 5/6 :!:

User avatar
OneCD
Ask me anything
Posts: 9165
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: Speed comparison: comparing disk #, raid, CPU, encryption

Post by OneCD » Tue Jun 22, 2021 6:52 am

* topic locked to prevent necroposting *

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage

Locked

Return to “Users' Corner”