Slow transfer speed due to SSD cache acceleration

Questions about SNMP, Power, System, Logs, disk, & RAID.
Post Reply
User avatar
Trexx
Ask me anything
Posts: 5223
Joined: Sat Oct 01, 2011 7:50 am
Location: Minnesota

Re: Slow transfer speed due to SSD cache acceleration

Post by Trexx » Fri Dec 23, 2016 5:17 am

cryptochrome wrote:Where do you guys set random vs. sequential cache mode? Is that a 4.3 thing? I don't have such an option on my 4.2 box.


That may be a 4.3 option only. It was set as part of the SSD CREATE wizard in Storage Manager.

In 4.2, it looks like they have something similar if you click the "CACHE SETTING" option on the volume.
http://docs.qnap.com/nas/4.2/SMB/en/cac ... ration.htm
Paul

Model: TS-877-1600 FW: 4.4.2.x
QTS (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
Data (HDD): [RAID-5] 6 x 3TB HGST DeskStar
VMs (SSD): [RAID-1] 2 x 500GB Evo 860
Ext. (HDD): TR-004 [Raid-5] 4 x 4TB HGST Ultastor
RAM: Kingston HyperX Fury 64GB DDR4-2666
GPU: EVGA GTX 1060 6GB
UPS: CP AVR1350

Model:TVS-673 32GB FW: 4.4.2.x Test/Backup Box
Model:TS-228a FW: 4.4.2.x Test/Backup Box
-----------------------------------------------------------------------------------------------------------------------------------------
NAS RAID Rebuild Times | Live QTS Videos | | QNAP NAS Guide | Information needed when you ask for HELP | QNAP Links, Tutorials, etc.
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq

-hitman-
Starting out
Posts: 13
Joined: Fri Dec 30, 2016 6:14 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by -hitman- » Wed Jan 04, 2017 4:29 am

Also have the same issues, I have opened a new thread viewtopic.php?f=189&t=128829

that a member replied to, pointing me here.

I've also initiated a ticket to Qnap support, guess i'll have to disable cache until this get resolved :(

-hitman-
Starting out
Posts: 13
Joined: Fri Dec 30, 2016 6:14 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by -hitman- » Wed Jan 04, 2017 6:38 am

Disabled SSD cache and boom, constant full speed transfers with no issues, I'll relay this to Qnap support when they get in contact.

cryptochrome
Been there, done that
Posts: 546
Joined: Sat Mar 09, 2013 7:26 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by cryptochrome » Wed Jan 04, 2017 8:12 am

I can't stress enough how important it is to a) open support tickets AND b) point them to this thread here. They won't listen and run you through first level support questionaires (have you tried rebooting?) for weeks otherwise. Point them here.

pokrakam
Starting out
Posts: 43
Joined: Wed Sep 17, 2008 5:27 am

Re: Slow transfer speed due to SSD cache acceleration

Post by pokrakam » Wed Jan 04, 2017 5:31 pm

Unfortunately my support ticket was terminated because Crucial MX300 is not on the supported list. Apparently you guys who are experiencing the same issue with compatible drives could have a different root cause.
They suggested I purchase a 'compatible' SSD, which to me is a somewhat expensive experiment since the root cause is unknown and given the similar symptoms there is a good chance it will do the same.

For what it's worth, I have definitely narrowed it down to the caching. Setting the two SSDs up as individual volumes, I could not reproduce any slowdown with multiple operations between them via the console. But setting one SSD as a read cache (!?) and copying from HDD to the other SSD, I saw the slowdown after 100-odd GB.

I could live with it if it would drop to HDD-like speeds. But it goes well below that, 15-20MB/s is just not fun.

cryptochrome
Been there, done that
Posts: 546
Joined: Sat Mar 09, 2013 7:26 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by cryptochrome » Wed Jan 04, 2017 6:46 pm

Wow... what a shabby excuse from them... the drive is not compatible. That's one way of saying to f§%$ off. QNAP support keeps failing, as usual. Did you point them to this thread here?

-hitman-
Starting out
Posts: 13
Joined: Fri Dec 30, 2016 6:14 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by -hitman- » Wed Jan 04, 2017 6:52 pm

I've received my support email this morning from tech support, basically saying windows 10 does have transfer issues and also asking for log files from my Qnap to be sent over.
I'm at work at the moment but will reply when I get in tonight and reiterate that the issue is with SSD raid cache and is resolved only when disabling the SSD cache acceleration, nothing to do with windows 10 as the speed is at max /without SSD caching, along with sending the logs they asked for AND pointing them to this thread.

I will update when I have a response.

Thanks.

pokrakam
Starting out
Posts: 43
Joined: Wed Sep 17, 2008 5:27 am

Re: Slow transfer speed due to SSD cache acceleration

Post by pokrakam » Wed Jan 04, 2017 7:55 pm

cryptochrome wrote:Wow... what a shabby excuse from them... the drive is not compatible. That's one way of saying to f§%$ off. QNAP support keeps failing, as usual. Did you point them to this thread here?

Yup, pointed this thread out in my initial ticket. Apparently people with supported drives who are experiencing the same issue "could have a different cause", therefore my not-on-the-list SSD has been labelled as "incompatible" without understanding what the cause is.

I opted for the MX300 because, as it was a new model, I didn't know how long it would take to be tested, if at all. Not an unusual dilemma with new models.

User avatar
Trexx
Ask me anything
Posts: 5223
Joined: Sat Oct 01, 2011 7:50 am
Location: Minnesota

Re: Slow transfer speed due to SSD cache acceleration

Post by Trexx » Wed Jan 04, 2017 8:20 pm

pokrakam wrote:Unfortunately my support ticket was terminated because Crucial MX300 is not on the supported list. Apparently you guys who are experiencing the same issue with compatible drives could have a different root cause.
They suggested I purchase a 'compatible' SSD, which to me is a somewhat expensive experiment since the root cause is unknown and given the similar symptoms there is a good chance it will do the same.

For what it's worth, I have definitely narrowed it down to the caching. Setting the two SSDs up as individual volumes, I could not reproduce any slowdown with multiple operations between them via the console. But setting one SSD as a read cache (!?) and copying from HDD to the other SSD, I saw the slowdown after 100-odd GB.

I could live with it if it would drop to HDD-like speeds. But it goes well below that, 15-20MB/s is just not fun.



That is an incorrect statement by support. Mx300 are on the supported list for tvs-x73 family.


Sent from my iPad using Tapatalk
Paul

Model: TS-877-1600 FW: 4.4.2.x
QTS (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
Data (HDD): [RAID-5] 6 x 3TB HGST DeskStar
VMs (SSD): [RAID-1] 2 x 500GB Evo 860
Ext. (HDD): TR-004 [Raid-5] 4 x 4TB HGST Ultastor
RAM: Kingston HyperX Fury 64GB DDR4-2666
GPU: EVGA GTX 1060 6GB
UPS: CP AVR1350

Model:TVS-673 32GB FW: 4.4.2.x Test/Backup Box
Model:TS-228a FW: 4.4.2.x Test/Backup Box
-----------------------------------------------------------------------------------------------------------------------------------------
NAS RAID Rebuild Times | Live QTS Videos | | QNAP NAS Guide | Information needed when you ask for HELP | QNAP Links, Tutorials, etc.
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq

pokrakam
Starting out
Posts: 43
Joined: Wed Sep 17, 2008 5:27 am

Re: Slow transfer speed due to SSD cache acceleration

Post by pokrakam » Wed Jan 04, 2017 8:36 pm

The compatibility list only includes the M.2. As it's just a different form factor, I figured a similar device would have a good chance of success. I'm still not convinced there's any compatibility problem.

cryptochrome
Been there, done that
Posts: 546
Joined: Sat Mar 09, 2013 7:26 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by cryptochrome » Wed Jan 04, 2017 10:10 pm

Of course there is no compatibility problem. They are just using that statement as an excuse to not have to deal with you and a problem they have no clue on solving.

User avatar
Trexx
Ask me anything
Posts: 5223
Joined: Sat Oct 01, 2011 7:50 am
Location: Minnesota

Re: Slow transfer speed due to SSD cache acceleration

Post by Trexx » Wed Jan 04, 2017 10:29 pm

pokrakam wrote:The compatibility list only includes the M.2. As it's just a different form factor, I figured a similar device would have a good chance of success. I'm still not convinced there's any compatibility problem.


They are both SATA interfaces. They both use the same FW files from Crucial (which would imply same/similar Marvell SSD controller). I would challenge them to show you the difference.

Tell the first level tech that you want the issue escalated to RD (Taiwan).
Paul

Model: TS-877-1600 FW: 4.4.2.x
QTS (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
Data (HDD): [RAID-5] 6 x 3TB HGST DeskStar
VMs (SSD): [RAID-1] 2 x 500GB Evo 860
Ext. (HDD): TR-004 [Raid-5] 4 x 4TB HGST Ultastor
RAM: Kingston HyperX Fury 64GB DDR4-2666
GPU: EVGA GTX 1060 6GB
UPS: CP AVR1350

Model:TVS-673 32GB FW: 4.4.2.x Test/Backup Box
Model:TS-228a FW: 4.4.2.x Test/Backup Box
-----------------------------------------------------------------------------------------------------------------------------------------
NAS RAID Rebuild Times | Live QTS Videos | | QNAP NAS Guide | Information needed when you ask for HELP | QNAP Links, Tutorials, etc.
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq

pokrakam
Starting out
Posts: 43
Joined: Wed Sep 17, 2008 5:27 am

Re: Slow transfer speed due to SSD cache acceleration

Post by pokrakam » Wed Jan 04, 2017 11:08 pm

Trexx wrote:Tell the first level tech that you want the issue escalated to RD (Taiwan).

RD are the ones who have been on the case, the exact response I got:
Support wrote:Its hard for us to narrow down when it comes to debugging. Even if we want to reproduce the issue and analyze it which is not possible to do it if you the incompatible SSD drives.

Issues may sound common but root-cause could be different.


I will just wait until QTS 4.3 with a new kernel and/or for someone with this issue on a compatible SDD to get their ticket resolved, which should make my incompatible SSD compatible again :roll:

User avatar
Trexx
Ask me anything
Posts: 5223
Joined: Sat Oct 01, 2011 7:50 am
Location: Minnesota

Re: Slow transfer speed due to SSD cache acceleration

Post by Trexx » Wed Jan 04, 2017 11:23 pm

In the latest QNAP Diagnostics tool, there is an SSD dump option (at least under 4.3) that generates a file giving some information from Flashcache.

Have they had you do that at all?

You are supposed to run the SSD Dump while the SSD is getting heavily hit. Might try doing your transfer test where you see the speed drop. If you know roughly when the speed drop will happen, you could start the SSD Dump just prior the speed drop so it captures before/after.

Might be worth a shot.
Paul

Model: TS-877-1600 FW: 4.4.2.x
QTS (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
Data (HDD): [RAID-5] 6 x 3TB HGST DeskStar
VMs (SSD): [RAID-1] 2 x 500GB Evo 860
Ext. (HDD): TR-004 [Raid-5] 4 x 4TB HGST Ultastor
RAM: Kingston HyperX Fury 64GB DDR4-2666
GPU: EVGA GTX 1060 6GB
UPS: CP AVR1350

Model:TVS-673 32GB FW: 4.4.2.x Test/Backup Box
Model:TS-228a FW: 4.4.2.x Test/Backup Box
-----------------------------------------------------------------------------------------------------------------------------------------
NAS RAID Rebuild Times | Live QTS Videos | | QNAP NAS Guide | Information needed when you ask for HELP | QNAP Links, Tutorials, etc.
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq

pokrakam
Starting out
Posts: 43
Joined: Wed Sep 17, 2008 5:27 am

Re: Slow transfer speed due to SSD cache acceleration

Post by pokrakam » Wed Jan 04, 2017 11:49 pm

Hmm, they did use that and took the complete dump log, but not while it was struggling. I only just noticed the SSD option at the bottom now that you pointed it out, thanks!

Will give it a go but will take some time (4-6 hours to activate and synch the cache...).

Post Reply

Return to “System & Disk Volume Management”