Slow transfer speed due to SSD cache acceleration

Questions about SNMP, Power, System, Logs, disk, & RAID.
Post Reply
vikino
Know my way around
Posts: 127
Joined: Sun Jul 10, 2016 3:06 am

Re: Slow transfer speed due to SSD cache acceleration

Post by vikino » Sun Nov 05, 2017 4:25 pm

robinqu wrote:I'm a 1282T3 owner, and SSD Cache acceleration **.

* 1282T3, QTS 4.3.3
* Seagate 10GB Enterprise Capcacity v6 * 8, static volume on RAID6;
* Samsung 850 EVO M.2 SATA 500GB * 2 as RAID0 read/write cache, LRU, non-blocksize-limit.


* about 250MB/s seq write speed over Thunderbolt
* about 110Mb/s for server-side benchmarks: dd if=/dev/zero of=/share/Storage/test1.img bs=4G count=1 conv=fsync


Seems also pretty slow...
TVS-1282T3-i7-64G + 8x WD Red Pro 10TB, 4x Samsung 850 512GB SSD + 2x Samsung 850 M.2 512GB cache, 2x 1TB Samsung 960 Pro M.2 NVMe SSD in QM2 card, Nvidia GTX1070Ti, 450W Corsair PSU
TS-453B-8G + 4x WD Red 6TB + 2x 128GB NVMe SSD in QM2 card for cache

P3R
Guru
Posts: 12293
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Slow transfer speed due to SSD cache acceleration

Post by P3R » Sun Nov 05, 2017 5:45 pm

robinqu wrote:* Samsung 850 EVO M.2 SATA 500GB * 2 as RAID0 read/write cache...
A redundant (RAID 1, RAID 10, RAID 6, RAID 5) SSD-configuration is strongly recommended if using a write cache. Using RAID 0 is even more dangerous than a single SSD. When (not if) either SSD fail, you will very likely have data loss and a messed up file system.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

cryptochrome
Been there, done that
Posts: 546
Joined: Sat Mar 09, 2013 7:26 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by cryptochrome » Sun Nov 05, 2017 6:22 pm

robinqu wrote:
* Samsung 850 EVO M.2 SATA 500GB * 2 as RAID0 read/write cache, LRU, non-blocksize-limit.



Are you sure you want to use RAID-0 on your read/write cache? That's pretty dangerous. If one of those two SSDs dies, you lose up to 1 TB of cached data that's not yet been written to disk. I would set that to RAID-1 if I were you (and it's what QNAP strongly advise to do).

User avatar
Trexx
Ask me anything
Posts: 5284
Joined: Sat Oct 01, 2011 7:50 am
Location: Minnesota

Re: Slow transfer speed due to SSD cache acceleration

Post by Trexx » Mon Nov 06, 2017 2:10 am

okenny wrote:But Trexx it shouldn't be slower with cache enabled, should it? Would you expect this from other platforms?


It depends on many factors...

To/From targets (inside qnap vs. outside)
NW connection Speed
Transfer type (sequential vs. read)
Block size
SSD model/config / Temperature of SSD controller/chips (is thermal throttling kicked in)
HDD config
etc.

For sequential access patterns involving a 1GbE Connection (to or from) SSD caching won't really help. Your 1GbE speed (~110MB/sec) is your bottleneck as even a single modern NAS rated drive can match that throughput much less say a 4-8 drive raid-set.

At most you MIGHT see some benefit from cache if you are frequently accessing the same data over and over again in a short period of time.

Here is a good analysis of flashcache in general:
http://iopscience.iop.org/article/10.10 ... 062023/pdf
Paul

Model: TS-877-1600 FW: 4.4.3.x
QTS (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
Data (HDD): [RAID-5] 6 x 3TB HGST DeskStar
VMs (SSD): [RAID-1] 2 x 500GB Evo 860
Ext. (HDD): TR-004 [Raid-5] 4 x 4TB HGST Ultastor
RAM: Kingston HyperX Fury 64GB DDR4-2666
GPU: EVGA GTX 1060 6GB
UPS: CP AVR1350

Model:TVS-673 32GB FW: 4.4.3.x Test/Backup Box
Model:TS-228a FW: 4.4.3.x Test/Backup Box
-----------------------------------------------------------------------------------------------------------------------------------------
NAS RAID Rebuild Times | Live QTS Videos | | QNAP NAS Guide | Information needed when you ask for HELP | QNAP Links, Tutorials, etc.
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq

vikino
Know my way around
Posts: 127
Joined: Sun Jul 10, 2016 3:06 am

Re: Slow transfer speed due to SSD cache acceleration

Post by vikino » Mon Nov 06, 2017 2:12 am

Trexx wrote:
okenny wrote:But Trexx it shouldn't be slower with cache enabled, should it? Would you expect this from other platforms?


It depends on many factors...

To/From targets (inside qnap vs. outside)
NW connection Speed
Transfer type (sequential vs. read)
Block size
SSD model/config / Temperature of SSD controller/chips (is thermal throttling kicked in)
HDD config
etc.

For sequential access patterns involving a 1GbE Connection (to or from) SSD caching won't really help. Your 1GbE speed (~110MB/sec) is your bottleneck as even a single modern NAS rated drive can match that throughput much less say a 4-8 drive raid-set.

At most you MIGHT see some benefit from cache if you are frequently accessing the same data over and over again in a short period of time.

Here is a good analysis of flashcache in general:
http://iopscience.iop.org/article/10.10 ... 062023/pdf


Have a look on my measuring,im copying data internally on qnap, temps are ok, ssds working fine as you can see on my tests...
TVS-1282T3-i7-64G + 8x WD Red Pro 10TB, 4x Samsung 850 512GB SSD + 2x Samsung 850 M.2 512GB cache, 2x 1TB Samsung 960 Pro M.2 NVMe SSD in QM2 card, Nvidia GTX1070Ti, 450W Corsair PSU
TS-453B-8G + 4x WD Red 6TB + 2x 128GB NVMe SSD in QM2 card for cache

vikino
Know my way around
Posts: 127
Joined: Sun Jul 10, 2016 3:06 am

Re: Slow transfer speed due to SSD cache acceleration

Post by vikino » Mon Nov 06, 2017 4:50 am

cryptochrome wrote:I can confirm the results Vikino has seen in 4.3.4. What is interesting though is that the resource monitor no longer shows extremely high IOPS latency when the speed crashes. I was always able to see between 20.000 and 50.000 ms latency in the resource monitor while copying large amounts of files "through" the cache. Whenever it hit those ridiculous amounts of latency, the speed crashed down. Now the speed is still crashing, but IOPS latency looks fairly normal (between 20 and 100 ms).

Needless to say (and this is directed at QNAP): Ignorance is bliss.


Try to install 4.3.4, completely delete cache, not just off, delete, and re-create new one...seems it little bit helped to me...
RG1=HDDs from where i copy
RG2=SSD Raid10 cache
RG3=destination SSDs RAID0
You do not have the required permissions to view the files attached to this post.
TVS-1282T3-i7-64G + 8x WD Red Pro 10TB, 4x Samsung 850 512GB SSD + 2x Samsung 850 M.2 512GB cache, 2x 1TB Samsung 960 Pro M.2 NVMe SSD in QM2 card, Nvidia GTX1070Ti, 450W Corsair PSU
TS-453B-8G + 4x WD Red 6TB + 2x 128GB NVMe SSD in QM2 card for cache

User avatar
Don
Guru
Posts: 12004
Joined: Thu Jan 03, 2008 4:56 am
Location: Long Island, New York

Re: Slow transfer speed due to SSD cache acceleration

Post by Don » Fri Nov 17, 2017 11:06 pm

Firmware 4.3.4.0387 (Beta2) has the following enhancement.

SSD Improvement

The SSD cache flush algorithms have been updated for better caching performance when under high load.
A disk trim and performance test is now automatically performed when creating a RAID group.


Maybe this will fix the issue?
Read the Online Manuals and use the forum search feature before posting.

It is a recommended to use RAID and have external backups. RAID will protect you from disk failure, keep your system running, and data accessible while the disk is replaced and the RAID rebuilt. Backups will allow you to recover data that is lost or corrupted, or from system failure. One does not replace the other.

Submit bugs and feature requests to QNAP via their Helpdesk app.

NAS: TVS-882BR | F/W: 4.3.6.0895 | 40GB | 2 x M.2 SATA RAID 1 (System/VMs) | 4 x M.2 NMVe QM2-4P-384A RAID 5 (Cache) | 5 x 4TB HDD RAID 6 (Data) | 1 x Blu-ray
NAS: TVS-663 | F/W: 4.4.2.1320 | 16GB | 2 x M.2 NMVe QM2-2P RAID 1 (Cache) | 4 x 4TB RAID 5
Apps: Boinc, Squid, DNSMasq, PLEX, iDrive, QVPN, QLMS, MP3fs, HBS, Entware, DLstation, +others

cryptochrome
Been there, done that
Posts: 546
Joined: Sat Mar 09, 2013 7:26 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by cryptochrome » Fri Nov 17, 2017 11:52 pm

That was already in the release notes for beta-1 and unfortunately, it didn't really fix the issue.

vikino
Know my way around
Posts: 127
Joined: Sun Jul 10, 2016 3:06 am

Re: Slow transfer speed due to SSD cache acceleration

Post by vikino » Sat Nov 18, 2017 12:35 am

Installed Beta2 today and tonight ill do some tests...
TVS-1282T3-i7-64G + 8x WD Red Pro 10TB, 4x Samsung 850 512GB SSD + 2x Samsung 850 M.2 512GB cache, 2x 1TB Samsung 960 Pro M.2 NVMe SSD in QM2 card, Nvidia GTX1070Ti, 450W Corsair PSU
TS-453B-8G + 4x WD Red 6TB + 2x 128GB NVMe SSD in QM2 card for cache

User avatar
storageman
Ask me anything
Posts: 5511
Joined: Thu Sep 22, 2011 10:57 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by storageman » Sat Nov 18, 2017 1:26 am

Personally I have real doubts about m.2 SSDs (or the way Qnap cool them). Think they get too hot and then start playing up.
New QM2 cards are the way to get, think they cool them better.

User avatar
Trexx
Ask me anything
Posts: 5284
Joined: Sat Oct 01, 2011 7:50 am
Location: Minnesota

Re: Slow transfer speed due to SSD cache acceleration

Post by Trexx » Sat Nov 18, 2017 1:40 am

With the newer m.2 heatsink design, I think the dissipation is better. From what I have seen in the video's for the TS-x77, it also looks like QNAP has learned some lessons from earlier models regarding the airflow/cooling design as well.
Paul

Model: TS-877-1600 FW: 4.4.3.x
QTS (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
Data (HDD): [RAID-5] 6 x 3TB HGST DeskStar
VMs (SSD): [RAID-1] 2 x 500GB Evo 860
Ext. (HDD): TR-004 [Raid-5] 4 x 4TB HGST Ultastor
RAM: Kingston HyperX Fury 64GB DDR4-2666
GPU: EVGA GTX 1060 6GB
UPS: CP AVR1350

Model:TVS-673 32GB FW: 4.4.3.x Test/Backup Box
Model:TS-228a FW: 4.4.3.x Test/Backup Box
-----------------------------------------------------------------------------------------------------------------------------------------
NAS RAID Rebuild Times | Live QTS Videos | | QNAP NAS Guide | Information needed when you ask for HELP | QNAP Links, Tutorials, etc.
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq

vikino
Know my way around
Posts: 127
Joined: Sun Jul 10, 2016 3:06 am

Re: Slow transfer speed due to SSD cache acceleration

Post by vikino » Sat Nov 18, 2017 4:17 am

Guys...read this thread first please...it is not about temps, we are talking about copying from HDDs cached by 2.5 disks, ALL TEMPS BELOW 40 degrees, copying to Samsung 960 Pro 1TB disks RAID0 in QM2 card, well cooled system.
TVS-1282T3-i7-64G + 8x WD Red Pro 10TB, 4x Samsung 850 512GB SSD + 2x Samsung 850 M.2 512GB cache, 2x 1TB Samsung 960 Pro M.2 NVMe SSD in QM2 card, Nvidia GTX1070Ti, 450W Corsair PSU
TS-453B-8G + 4x WD Red 6TB + 2x 128GB NVMe SSD in QM2 card for cache

vikino
Know my way around
Posts: 127
Joined: Sun Jul 10, 2016 3:06 am

Re: Slow transfer speed due to SSD cache acceleration

Post by vikino » Sat Nov 18, 2017 4:21 am

QTS 4.3.4 Beta2 results:
Copying with cache enabled gets continuous 160MB/s thru cache and then write to SSDs...
Flushing cache to disks - WOW... 140-150MB/s constant... (peaks at 260MB/s), slowly dropping speed, ending at approx 20MB/s
Copying without cache - 700MB/s....

so still the same as on 4.3.3
What a HUGE improvement... 5 times cheaper Synology works better then this expensive "disk box"...
TVS-1282T3-i7-64G + 8x WD Red Pro 10TB, 4x Samsung 850 512GB SSD + 2x Samsung 850 M.2 512GB cache, 2x 1TB Samsung 960 Pro M.2 NVMe SSD in QM2 card, Nvidia GTX1070Ti, 450W Corsair PSU
TS-453B-8G + 4x WD Red 6TB + 2x 128GB NVMe SSD in QM2 card for cache

DJ-Sunnie
First post
Posts: 1
Joined: Tue Mar 14, 2017 9:11 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by DJ-Sunnie » Sun Dec 03, 2017 12:00 am

Sorry that my 1st post here is a 'bad' one:
I second that slow down behaviour with the QTS 4.3.4 Beta2 also...

The Hardware-Information shows only (PCIe) for the QM2 board. :(
The I/O leds on the QM2 board aren't ligthen up on NW transfers :(
This board was just newly installed with two Intel 600P (I know - they are the slower ones)

System: TS-531X-4G + 5x WD 3TB + 2x 128GB NVMe SSD in QM2 card for cache

up2ucnx
New here
Posts: 7
Joined: Thu Dec 07, 2017 1:04 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by up2ucnx » Fri Dec 08, 2017 7:48 am

Sooo, I just got informed of here after I ordered a QM2 as I have two 250GB M2 to go in it & I see it is nearly a year later so has anyone got a solution?
I was thinking that 2 x 250GB M2 in Raid 1 would make my 471 blinding fast.
Is there any config that I can use?
Cheers, Ed

Post Reply

Return to “System & Disk Volume Management”