Slow transfer speed due to SSD cache acceleration

Questions about SNMP, Power, System, Logs, disk, & RAID.
Post Reply
vikino
Know my way around
Posts: 127
Joined: Sun Jul 10, 2016 3:06 am

Re: Slow transfer speed due to SSD cache acceleration

Post by vikino » Fri Apr 13, 2018 3:33 am

How to say that in short...with last firmware cache seems to be slightly better...In RAID 10 config im getting 800MB/s read and 450MB/s if cache is flushed and clean and 230MB/s write if cache is full...
TVS-1282T3-i7-64G + 8x WD Red Pro 10TB, 4x Samsung 850 512GB SSD + 2x Samsung 850 M.2 512GB cache, 2x 1TB Samsung 960 Pro M.2 NVMe SSD in QM2 card, Nvidia GTX1070Ti, 450W Corsair PSU
TS-453B-8G + 4x WD Red 6TB + 2x 128GB NVMe SSD in QM2 card for cache

BryceWilkins
Know my way around
Posts: 126
Joined: Mon Jun 21, 2010 5:39 am

Re: Slow transfer speed due to SSD cache acceleration

Post by BryceWilkins » Mon Apr 16, 2018 5:21 am

Trexx wrote:See the link for QTS Presentations in my signature. Has links to upcoming video/slides as well as previous presentations.

Excellent Trexx! That is perfect to have the slides to watch alongside the presentation and also as a reference for later -- tons of good information there!

dolbyman wrote:do QTIER instead

Thanks for your comment dolbyman! I will try it in the future, I think. QTier sounds great, and I read many good scenarios about it. I would to have a NAS as "test NAS" to experiment with these features but I don't have one for that that so I am a bit more cautious about the complexity of my setup. My concerns might be unwarranted, but for now I try to make the performance outcome more predictable. (If I put stuff on SSDs, it should be fast no matter what because that's all it can be...versus not being able to tell how well QTier is performing.)

I created everything with Storage Pools to give me flexibility to add 2.5" SSDs or NVMe drives into the drive pools in the future. I hope it will allow me to migrate to QTier that way if I choose to.

vikino wrote:How to say that in short...with last firmware cache seems to be slightly better...In RAID 10 config im getting 800MB/s read and 450MB/s if cache is flushed and clean and 230MB/s write if cache is full...


Thanks vikino for update! This is great news! It's seems really significantly better than before...and now it is not dropping to way below the HDD write speed which was the case before. I'm happy to hear -- it gives me some hope to trying it in the future.

User avatar
storageman
Ask me anything
Posts: 5511
Joined: Thu Sep 22, 2011 10:57 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by storageman » Mon Apr 16, 2018 10:16 pm

RAID 10 is not the right choice if you want the best sequential performance.

vikino
Know my way around
Posts: 127
Joined: Sun Jul 10, 2016 3:06 am

Re: Slow transfer speed due to SSD cache acceleration

Post by vikino » Mon Apr 16, 2018 10:41 pm

storageman wrote:RAID 10 is not the right choice if you want the best sequential performance.


Which one then? It is adviced by QNAP...
TVS-1282T3-i7-64G + 8x WD Red Pro 10TB, 4x Samsung 850 512GB SSD + 2x Samsung 850 M.2 512GB cache, 2x 1TB Samsung 960 Pro M.2 NVMe SSD in QM2 card, Nvidia GTX1070Ti, 450W Corsair PSU
TS-453B-8G + 4x WD Red 6TB + 2x 128GB NVMe SSD in QM2 card for cache

User avatar
storageman
Ask me anything
Posts: 5511
Joined: Thu Sep 22, 2011 10:57 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by storageman » Mon Apr 16, 2018 10:54 pm

Don't care what Qnap advise. Where have they advise it anyway and for what?
RAID 5 or 6.
RAID 10 is for random performance/higher IOPs

vikino
Know my way around
Posts: 127
Joined: Sun Jul 10, 2016 3:06 am

Re: Slow transfer speed due to SSD cache acceleration

Post by vikino » Mon Apr 16, 2018 11:04 pm

Overall is RAID10 better, for random as well as sequential with redundancy, this is meaning od RAID10, i did some tests on RAID5 cache and it did not performed better than RAID10...
TVS-1282T3-i7-64G + 8x WD Red Pro 10TB, 4x Samsung 850 512GB SSD + 2x Samsung 850 M.2 512GB cache, 2x 1TB Samsung 960 Pro M.2 NVMe SSD in QM2 card, Nvidia GTX1070Ti, 450W Corsair PSU
TS-453B-8G + 4x WD Red 6TB + 2x 128GB NVMe SSD in QM2 card for cache

User avatar
storageman
Ask me anything
Posts: 5511
Joined: Thu Sep 22, 2011 10:57 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by storageman » Tue Apr 17, 2018 12:13 am

I shouldn't have got involved!
Are you talking about for SSD cache or for the data volume?

vikino
Know my way around
Posts: 127
Joined: Sun Jul 10, 2016 3:06 am

Re: Slow transfer speed due to SSD cache acceleration

Post by vikino » Tue Apr 17, 2018 5:03 am

We are talking about SSD cache of 4 2.5" ssds for HDD datastore...
Btw., qnap does not support raid 5 for cache, only 0,1 and 10...
Ive tryied RAID5 cache on Adaptec controller and RAID10 performed better in random as well as sequence...
TVS-1282T3-i7-64G + 8x WD Red Pro 10TB, 4x Samsung 850 512GB SSD + 2x Samsung 850 M.2 512GB cache, 2x 1TB Samsung 960 Pro M.2 NVMe SSD in QM2 card, Nvidia GTX1070Ti, 450W Corsair PSU
TS-453B-8G + 4x WD Red 6TB + 2x 128GB NVMe SSD in QM2 card for cache

User avatar
storageman
Ask me anything
Posts: 5511
Joined: Thu Sep 22, 2011 10:57 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by storageman » Tue Apr 17, 2018 4:11 pm

vikino wrote:We are talking about SSD cache of 4 2.5" ssds for HDD datastore...
Btw., qnap does not support raid 5 for cache, only 0,1 and 10...
Ive tryied RAID5 cache on Adaptec controller and RAID10 performed better in random as well as sequence...


Yes,my mistake, I thought you were talking about volume RAID not the cache RAID.

But for HDDs my comment still holds.

User avatar
Trexx
Ask me anything
Posts: 5251
Joined: Sat Oct 01, 2011 7:50 am
Location: Minnesota

Re: Slow transfer speed due to SSD cache acceleration

Post by Trexx » Tue Apr 17, 2018 8:23 pm

vikino wrote:We are talking about SSD cache of 4 2.5" ssds for HDD datastore...
Qnap does not support raid 5 for cache, only 0,1 and 10...


I am not sure if that is a true statement to be honest or not. It could be that QNAP doesn't support RAID-5 for SSD Cache when the drives are spread across multiple "devices" i.e. QNAP + QM2 card due to risk if card is failed, then RAID-5 redundancy is gone.

They MAY support it if you had say 4x2.5" SSD drives internally, or one of the new QM2 cards that support 4x m.2 slots.
Paul

Model: TS-877-1600 FW: 4.4.3.x
QTS (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
Data (HDD): [RAID-5] 6 x 3TB HGST DeskStar
VMs (SSD): [RAID-1] 2 x 500GB Evo 860
Ext. (HDD): TR-004 [Raid-5] 4 x 4TB HGST Ultastor
RAM: Kingston HyperX Fury 64GB DDR4-2666
GPU: EVGA GTX 1060 6GB
UPS: CP AVR1350

Model:TVS-673 32GB FW: 4.4.3.x Test/Backup Box
Model:TS-228a FW: 4.4.3.x Test/Backup Box
-----------------------------------------------------------------------------------------------------------------------------------------
NAS RAID Rebuild Times | Live QTS Videos | | QNAP NAS Guide | Information needed when you ask for HELP | QNAP Links, Tutorials, etc.
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq

vikino
Know my way around
Posts: 127
Joined: Sun Jul 10, 2016 3:06 am

Re: Slow transfer speed due to SSD cache acceleration

Post by vikino » Thu Apr 19, 2018 5:17 am

Trexx wrote:
vikino wrote:We are talking about SSD cache of 4 2.5" ssds for HDD datastore...
Qnap does not support raid 5 for cache, only 0,1 and 10...


I am not sure if that is a true statement to be honest or not. It could be that QNAP doesn't support RAID-5 for SSD Cache when the drives are spread across multiple "devices" i.e. QNAP + QM2 card due to risk if card is failed, then RAID-5 redundancy is gone.

They MAY support it if you had say 4x2.5" SSD drives internally, or one of the new QM2 cards that support 4x m.2 slots.



We are talking about 4x 2,5" SATA SSD drives, they dont support R5 for SSD cache at all, no matter which configuration. And there is no QM2 card with 4 slots :-)
TVS-1282T3-i7-64G + 8x WD Red Pro 10TB, 4x Samsung 850 512GB SSD + 2x Samsung 850 M.2 512GB cache, 2x 1TB Samsung 960 Pro M.2 NVMe SSD in QM2 card, Nvidia GTX1070Ti, 450W Corsair PSU
TS-453B-8G + 4x WD Red 6TB + 2x 128GB NVMe SSD in QM2 card for cache

noodles_gb
Starting out
Posts: 42
Joined: Sun Apr 10, 2016 6:00 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by noodles_gb » Fri Nov 23, 2018 4:53 pm

Found this thread via Google. I'm copying a load of movies to an external USB3 drive and I was getting ~15MB/s transfer rate. As soon as I turned off read/write cache this jumped up to over 150MB/s + (most likely the max rate the external disk can support writing).

My NAS is a TVS 671 with a quad core i7 in it and 16GB of RAM. Slot 1 and 2 are the SSDs, Crucial 500GB ones I believe, and the other 4 slots are WD Red 4TB's.

I've always had random disk latency issues (such as when nzbget is doing a par check, unrar'ing and moving data internally), never found the problem, looks like I have now. Since the NAS is mainly used as a Plex server, I think I'll remove the SSDs and do without caching.

User avatar
storageman
Ask me anything
Posts: 5511
Joined: Thu Sep 22, 2011 10:57 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by storageman » Fri Nov 23, 2018 7:03 pm

noodles_gb wrote:
Fri Nov 23, 2018 4:53 pm
Found this thread via Google. I'm copying a load of movies to an external USB3 drive and I was getting ~15MB/s transfer rate. As soon as I turned off read/write cache this jumped up to over 150MB/s + (most likely the max rate the external disk can support writing).

My NAS is a TVS 671 with a quad core i7 in it and 16GB of RAM. Slot 1 and 2 are the SSDs, Crucial 500GB ones I believe, and the other 4 slots are WD Red 4TB's.

I've always had random disk latency issues (such as when nzbget is doing a par check, unrar'ing and moving data internally), never found the problem, looks like I have now. Since the NAS is mainly used as a Plex server, I think I'll remove the SSDs and do without caching.
Interesting.
If you leave on cache as read only what happens? Same speed, worse, better?
Did you set the write cache to ignore big blocks?

noodles_gb
Starting out
Posts: 42
Joined: Sun Apr 10, 2016 6:00 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by noodles_gb » Fri Nov 23, 2018 7:50 pm

storageman wrote:
Fri Nov 23, 2018 7:03 pm
noodles_gb wrote:
Fri Nov 23, 2018 4:53 pm
Found this thread via Google. I'm copying a load of movies to an external USB3 drive and I was getting ~15MB/s transfer rate. As soon as I turned off read/write cache this jumped up to over 150MB/s + (most likely the max rate the external disk can support writing).

My NAS is a TVS 671 with a quad core i7 in it and 16GB of RAM. Slot 1 and 2 are the SSDs, Crucial 500GB ones I believe, and the other 4 slots are WD Red 4TB's.

I've always had random disk latency issues (such as when nzbget is doing a par check, unrar'ing and moving data internally), never found the problem, looks like I have now. Since the NAS is mainly used as a Plex server, I think I'll remove the SSDs and do without caching.
Interesting.
If you leave on cache as read only what happens? Same speed, worse, better?
Did you set the write cache to ignore big blocks?
I haven't tried read only, I will do that after work.

I never set up anything for block size, I don't ever remember seeing the setting. The NAS was built some years ago, so I don't have the storage snapshot options and what not. The intent was to rebuild it hence copying the data off before a system backup, rebuild and restore.

noodles_gb
Starting out
Posts: 42
Joined: Sun Apr 10, 2016 6:00 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by noodles_gb » Sat Nov 24, 2018 1:31 am

storageman - I can't choose block size for sequential read/write, only for random IO which I don't use.

Post Reply

Return to “System & Disk Volume Management”