Slow transfer speed due to SSD cache acceleration

Questions about SNMP, Power, System, Logs, disk, & RAID.
Post Reply
User avatar
SukkerFri
Starting out
Posts: 39
Joined: Sat Feb 25, 2012 6:07 am

Re: Slow transfer speed due to SSD cache acceleration

Post by SukkerFri »

I've always put off the poor cache performance due to too cheap SSD's (Kingston A400). They were just laying around so I've been playing abit with Cache. But that just grinded my performance to a near hold.

I have a 2012 R2 connected via 10GBit to my NAS, so thats not the bottleneck. The RAID5 with 8TB HDDs (IronWolf) or the RAID5 with the 3TB HDDs (WD Red) are the clear bottlenecks. But adding SSD cache (Even read only with Samsung Pro, the performance gets worse than just the HDD's alone). Luckily when using iSCSI in 2012 R2, the RAM is used as cache and tasks can be quickly done, as long as there is free RAM :) Then, maybe for minuts, I can in Task Manager see RAM getting free'd up and the Disk(iSCSI) and ethernet adapter is going at it, but slowly. Like 10MByte/sek or something.

ATM I see QNAP SSD Cache as completely broken.
If you can't explain it simply, you dont understand it well enough
TS-1635 - 4GB RAM:
3x8TB IronWolf - Raid5
8x3TB WD Red - Raid5
sohouser1
New here
Posts: 6
Joined: Wed Jan 16, 2019 1:12 am

Re: Slow transfer speed due to SSD cache acceleration

Post by sohouser1 »

Wow great work so far!

Just to confirm, I ran quite a few VM's using both qtier and SSD caching and very little sequential large file transfers. Even with random access VM's hitting the cache, it would still slow down when full. It was slow enough it corrupted 2 different VM's that I needed to restore from backup. (the OS got tired of waiting for I/O and timed out file access, not good with system files!)
joshuacant
New here
Posts: 3
Joined: Sat Aug 03, 2019 6:25 am

Re: Slow transfer speed due to SSD cache acceleration

Post by joshuacant »

Just wanted to report back that I let the SSD cache get up to 99% use, deleted it and re-added it, and the speeds never dropped below what I expected. Annoying, but not exactly a hardship to do once every few weeks. It'd be great if there were some command lines that could be run and a task scheduled to do it all every Sunday night at 3am, though.
MikeLagit
Easy as a breeze
Posts: 332
Joined: Fri Mar 22, 2013 11:40 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by MikeLagit »

Super depressing thread to say the least... I was just about to try using SSD cache hoping it would help in my workflows. I'm really glad I found this before investing, so thanks for that. Shouldn't QNAP just turn it off? :roll:
Model: TVS-872XT 16GB
Model: TS-877-1700 16GB
P3R
Guru
Posts: 13183
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Slow transfer speed due to SSD cache acceleration

Post by P3R »

klaasdj wrote: Fri Jul 17, 2020 9:30 pm First I tried Qtiering. Bad results, just like you all mentioned.
Not everyone report bad results with Qtier. There are senior members of this community that use it and are satisfied with it, as far as I have seen.

Maybe you didn’t give it enough time? The algorithm need to learn what data you use the most and have had the time to move that data to the SSD tier. You won't see any benefits from Qtier at all until all that have happened. Qtier isn’t for the impatient that wan’t immediate results, but it could probably be very good for those that use their storage in similar ways most of the time.
Beware that when you set the cache you have to select READ-WRITE!! And also select it for "Sequential I/O".
Yes if mostly used for sequential workloads (which is what almost all home and SMB users have) and if you want improved write performance for backups and other dumping of larger data volumes onto the storage, that’s a very good recommendation.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
MikeLagit
Easy as a breeze
Posts: 332
Joined: Fri Mar 22, 2013 11:40 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by MikeLagit »

I haven't heard of anyone that has been happy with it and left it in production. Seems like folks try it, get disappointed once it loads up, come here, then disable it. Done...
Model: TVS-872XT 16GB
Model: TS-877-1700 16GB
P3R
Guru
Posts: 13183
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Slow transfer speed due to SSD cache acceleration

Post by P3R »

klaasdj wrote: Sat Jul 18, 2020 12:54 pm What do you mean by senior members?
People that have extensive Qnap experience and that have spent much time in this forum trying to help less experienced users.
If Qtier is an algorithm for storing data on different tiers based on usage it should immediately place data that is for the first time written to the NAS on the SSD-tier.
15 minutes of reading to learn how Qtier actually works before selecting it would have saved you the day you now wasted on testing it having incorrect expectations of it's effect.
That is my experience with more professional solutions like Hitachi Vantara's DataOpps, Spectrum Protect (former IBM Tivoli Storage Manager), Symantec Enterprise Vault etc. do the job.
Unfortunately your experience with those professional solutions seem to have confused you more than help you this time as Qtier is different.
Most of the time these solutions even have a combination of caching and tiering in one and the same solution. or can be switched on or of on the same time.
Qtier can be combined with SSD caching, that can be switched on and off on the fly.

I think that Qtier can still only be switched on but not off. I'm not a Qtier user myself though so I'm not sure about the details.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
User avatar
dolbyman
Guru
Posts: 34903
Joined: Sat Feb 12, 2011 2:11 am
Location: Vancouver BC , Canada

Re: Slow transfer speed due to SSD cache acceleration

Post by dolbyman »

I use Qtier and it seems to work well

Via the pool write statistics you can sort of deduct how it works

Writes hit the fast tier first (then depending on usage, get offloaded to the lower tier right away or after a while) reads are coming from the low tier, unless they have been tiered up (scheduled task again) so folders with (e.g.) virtual machine vdisks with constant access get loaded into the tier for fast read

its sadly a very black box ..wish there was more transparency to it

oh also... I am 12 years old ..with 50 years of experience in "painting by numbers" and "selling fine leather jackets", I have 4 kids from 6 wifes a house with monkey and a horse
besides I invented the internet single handed, so there is that
MikeLagit
Easy as a breeze
Posts: 332
Joined: Fri Mar 22, 2013 11:40 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by MikeLagit »

Now that's funny! Lol. And here I always thought you were the great descendent of the king Earl Milton Dolby of Sound, who came off slave ships in the Africa West Indies....

Anything special with your setup to get it working for you Dolbyman? Anything I've tried the transfer speeds taper off madly once it's in production and I've always reverted back. Though I haven't tried it in 8 or 9 months, I doubt there's been any updates to it since then.
Model: TVS-872XT 16GB
Model: TS-877-1700 16GB
User avatar
dolbyman
Guru
Posts: 34903
Joined: Sat Feb 12, 2011 2:11 am
Location: Vancouver BC , Canada

Re: Slow transfer speed due to SSD cache acceleration

Post by dolbyman »

It might have to do with an optimal tier ratio

if you have a 20TB Storage pool and put a 256GB tier on it ..not sure it will do much good

for my 951x I have a 512GB Tier (-25% OP) for a 8TB thick volume ( 5x 4TB spinning RAID6 with some empty space)

for my 853Bu I have a 1TB Tier (-25% OP) for a 16TB thick volume ( 6x 6TB spinning RAID6 with some empty space)

On the 853Bu I hit the SSD write limit of around 500MB\s (10GbE network)
On the 951X I am limited externally by my crappy 5GbE adapter (hitting 370MB\s write)

both times sequential and random
P3R
Guru
Posts: 13183
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Slow transfer speed due to SSD cache acceleration

Post by P3R »

Some resources.

How to use Qtier.

Q: After upgrading a storage pool to use Qtier, why isn’t new data written to the SSD tier?
A: After upgrading a pool to a Qtier storage pool, Qtier does not start moving existing thick volume data to the SSD tier until the first scheduled auto-tiering task starts. This means that new data written to existing thick volumes before the first auto-tiering task is always stored in the HDD capacity tier.

Thin volumes and thick volumes created after upgrading to Qtier do not suffer from this limitation. To utilize the SSD tier immediately, create a new thick volume or convert your existing thick volumes to thin volumes.

Source: https://www.qnap.com/en/how-to/faq/arti ... e-ssd-tier

A review of Qnap Qtier and SSD caching by Storagereview.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
hardmack
New here
Posts: 5
Joined: Sun Jul 19, 2020 6:44 am

Re: Slow transfer speed due to SSD cache acceleration

Post by hardmack »

Brand new to qnap and the ecosystem... using my TS-332X for file storage, cloud backups and plex/minecraft. Very simple featureset use in other words... Got 3x Ironwolfs and 3x 128gb ssds

I didnt spend enough time researching either I guess.... as a very general end user I thought SSD cache would assist across the board for file transfer and system tasks. Reading through about 50% of this thread and AFTER getting all my network transfer speed back AFTER disabling caching, I was wondering if there is a consensus on if its even worth it for my use case. It was odd seeing 30MB/s filetransfers, while still using 1gbe I was hoping to see a pegged 100+MB/s transfer especially with 400ish MB/s speed tests of the m.2's in QTS
JRaymond
Starting out
Posts: 25
Joined: Mon Oct 27, 2014 4:11 am

Re: Slow transfer speed due to SSD cache acceleration

Post by JRaymond »

I started a thread on my issues before finding this one...located here viewtopic.php?f=25&t=156214

I'm seeing 35 MB/s for 10 NAS configured the same as my master. I need 5x that to make this project work in a timely fashion. I'm seeing 14 hours to transfer 2.5TB of data out from my master to the clients.

We just started this project...so I figured there would be SOME tweaking but FFS, I've got a 10GB pipe out of the NAS and can only get 35MB/s for 10 systems?
hardmack
New here
Posts: 5
Joined: Sun Jul 19, 2020 6:44 am

Re: Slow transfer speed due to SSD cache acceleration

Post by hardmack »

JRaymond wrote: Sun Aug 02, 2020 6:16 am I started a thread on my issues before finding this one...located here viewtopic.php?f=25&t=156214

I'm seeing 35 MB/s for 10 NAS configured the same as my master. I need 5x that to make this project work in a timely fashion. I'm seeing 14 hours to transfer 2.5TB of data out from my master to the clients.

We just started this project...so I figured there would be SOME tweaking but FFS, I've got a 10GB pipe out of the NAS and can only get 35MB/s for 10 systems?
I saw the same transfer speeds, I had to disable ssd caching all together and instantly once disabled full network speed
JRaymond
Starting out
Posts: 25
Joined: Mon Oct 27, 2014 4:11 am

Re: Slow transfer speed due to SSD cache acceleration

Post by JRaymond »

hardmack wrote: Sun Aug 02, 2020 11:05 am
JRaymond wrote: Sun Aug 02, 2020 6:16 am I started a thread on my issues before finding this one...located here viewtopic.php?f=25&t=156214

I'm seeing 35 MB/s for 10 NAS configured the same as my master. I need 5x that to make this project work in a timely fashion. I'm seeing 14 hours to transfer 2.5TB of data out from my master to the clients.

We just started this project...so I figured there would be SOME tweaking but FFS, I've got a 10GB pipe out of the NAS and can only get 35MB/s for 10 systems?
I saw the same transfer speeds, I had to disable ssd caching all together and instantly once disabled full network speed
I "have" no SSD caching...at all. There are no SSD's installed in the system.
Post Reply

Return to “System & Disk Volume Management”