Slow transfer speed due to SSD cache acceleration

Questions about SNMP, Power, System, Logs, disk, & RAID.
joshuacant
New here
Posts: 3
Joined: Sat Aug 03, 2019 6:25 am

Re: Slow transfer speed due to SSD cache acceleration

Post by joshuacant » Thu Sep 26, 2019 9:27 am

Thank you very much, AfroDieter. Great research done.

User avatar
SukkerFri
Starting out
Posts: 39
Joined: Sat Feb 25, 2012 6:07 am

Re: Slow transfer speed due to SSD cache acceleration

Post by SukkerFri » Thu Sep 26, 2019 4:03 pm

I've always put off the poor cache performance due to too cheap SSD's (Kingston A400). They were just laying around so I've been playing abit with Cache. But that just grinded my performance to a near hold.

I have a 2012 R2 connected via 10GBit to my NAS, so thats not the bottleneck. The RAID5 with 8TB HDDs (IronWolf) or the RAID5 with the 3TB HDDs (WD Red) are the clear bottlenecks. But adding SSD cache (Even read only with Samsung Pro, the performance gets worse than just the HDD's alone). Luckily when using iSCSI in 2012 R2, the RAM is used as cache and tasks can be quickly done, as long as there is free RAM :) Then, maybe for minuts, I can in Task Manager see RAM getting free'd up and the Disk(iSCSI) and ethernet adapter is going at it, but slowly. Like 10MByte/sek or something.

ATM I see QNAP SSD Cache as completely broken.
If you can't explain it simply, you dont understand it well enough
TS-1635 - 4GB RAM:
3x8TB IronWolf - Raid5
8x3TB WD Red - Raid5

sohouser1
New here
Posts: 5
Joined: Wed Jan 16, 2019 1:12 am

Re: Slow transfer speed due to SSD cache acceleration

Post by sohouser1 » Tue Oct 01, 2019 11:49 pm

Wow great work so far!

Just to confirm, I ran quite a few VM's using both qtier and SSD caching and very little sequential large file transfers. Even with random access VM's hitting the cache, it would still slow down when full. It was slow enough it corrupted 2 different VM's that I needed to restore from backup. (the OS got tired of waiting for I/O and timed out file access, not good with system files!)

joshuacant
New here
Posts: 3
Joined: Sat Aug 03, 2019 6:25 am

Re: Slow transfer speed due to SSD cache acceleration

Post by joshuacant » Sat Oct 05, 2019 5:37 am

Just wanted to report back that I let the SSD cache get up to 99% use, deleted it and re-added it, and the speeds never dropped below what I expected. Annoying, but not exactly a hardship to do once every few weeks. It'd be great if there were some command lines that could be run and a task scheduled to do it all every Sunday night at 3am, though.

MikeLagit
Easy as a breeze
Posts: 299
Joined: Fri Mar 22, 2013 11:40 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by MikeLagit » Thu Jun 11, 2020 9:30 am

Super depressing thread to say the least... I was just about to try using SSD cache hoping it would help in my workflows. I'm really glad I found this before investing, so thanks for that. Shouldn't QNAP just turn it off? :roll:
Model: TS-877-1700 16GB
(HDD): [RAID-5] 4 x 8TB Seagate Ironwolf

Model:TS-569L 3GB
(HDD): [RAID-5] 4 x 3TB WD Red

User avatar
klaasdj
New here
Posts: 6
Joined: Sat Apr 18, 2009 1:03 pm
Location: Maassluis

Re: Slow transfer speed due to SSD cache acceleration

Post by klaasdj » Fri Jul 17, 2020 9:30 pm

Good day all of you QNAP forum members,
Recently I bought a TS-453D with the QNAP QXG-10G1T 10GBe card. Filled it with 2 Harddrives (WD 6TB RED) and 2 Samsung 1TB SSD's. First I tried Qtiering. Bad results, just like you all mentioned. I even thought it had to do with my newly acquired 10GB switch. Throughput maximum at 150 - 200Mbit/s. Not very satisfactory. So, i dismantled Qtiering and went for an SSD storage pool only. Wow, now I got up to 500Mb/s !! That is what I would have expected with Qtier also. Unfortunately. after trying various configurations of Qtier I gave up.

After a good nights sleep I decided to go for SSD caching. This caused problems to me earlier as I accidentally erased a volume by removing the SSD from it/ (and no backup. And yes I know it is stupid!) But now I was more confident. I used the wizard to attach the 2 single SSD's to become the cache and almost again made the same mistake. Did not look further than my nose is long (is that known to you all).
Beware that when you set the cache you have to select READ-WRITE!!. And also select it for "Sequential I/O". It is not a very obvious setting as you need to scroll down in that pop-up screen!!!! (little triangel shows you can go down further)/ Once selected you cannot select the block size anymore. (grayed out at 1MB). After setting this up I was very anxious to see what would happen.

And yes, yes , yes. After transferring a Windows 10 ISO file from my PC to the TS-453D I received speeds around 400-500Mb/s.

Earlier I think I missed the settings for "read-write" and selected the wrong way to do the caching (Sequential I/O)
Hope you all find it helpful.

Stay healthy and all the best from The Netherlands
Last edited by klaasdj on Sat Jul 18, 2020 11:39 pm, edited 1 time in total.
TS-453D with QXG-10G1T 10GBe card, 2 TB WD RED and 2 1TB Samsung SSD. Still need a RAM upgrade!

P3R
Guru
Posts: 12185
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Slow transfer speed due to SSD cache acceleration

Post by P3R » Sat Jul 18, 2020 7:01 am

klaasdj wrote:
Fri Jul 17, 2020 9:30 pm
First I tried Qtiering. Bad results, just like you all mentioned.
Not everyone report bad results with Qtier. There are senior members of this community that use it and are satisfied with it, as far as I have seen.

Maybe you didn’t give it enough time? The algorithm need to learn what data you use the most and have had the time to move that data to the SSD tier. You won't see any benefits from Qtier at all until all that have happened. Qtier isn’t for the impatient that wan’t immediate results, but it could probably be very good for those that use their storage in similar ways most of the time.
Beware that when you set the cache you have to select READ-WRITE!! And also select it for "Sequential I/O".
Yes if mostly used for sequential workloads (which is what almost all home and SMB users have) and if you want improved write performance for backups and other dumping of larger data volumes onto the storage, that’s a very good recommendation.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

User avatar
klaasdj
New here
Posts: 6
Joined: Sat Apr 18, 2009 1:03 pm
Location: Maassluis

Re: Slow transfer speed due to SSD cache acceleration

Post by klaasdj » Sat Jul 18, 2020 12:54 pm

What do you mean by senior members?
- Age?
- Period over which one is member of this forum?
- Amount of (useless) responses placed here?
-The fact that senior members have had enough time to sit down and wait to see the effect of Qtier?

Please be more precise and explain what you mean.

If Qtier is an algorithm for storing data on different tiers based on usage it should immediately place data that is for the first time written to the NAS on the SSD-tier. As the algorithm does no no yet what the usage will be. Over time, that data can, based on (non) usage, be transferred to a slower/cheaper kind of storage. Data that is used frequently should never leave the fastest tier.
That is my experience with more professional solutions like Hitachi Vantara's DataOpps, Spectrum Protect (former IBM Tivoli Storage Manager), Symantec Enterprise Vault etc. do the job. Most of the time these solutions even have a combination of caching and tiering in one and the same solution. or can be switched on or of on the same time.
Last edited by klaasdj on Sat Jul 18, 2020 11:40 pm, edited 1 time in total.
TS-453D with QXG-10G1T 10GBe card, 2 TB WD RED and 2 1TB Samsung SSD. Still need a RAM upgrade!

MikeLagit
Easy as a breeze
Posts: 299
Joined: Fri Mar 22, 2013 11:40 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by MikeLagit » Sat Jul 18, 2020 2:39 pm

I haven't heard of anyone that has been happy with it and left it in production. Seems like folks try it, get disappointed once it loads up, come here, then disable it. Done...
Model: TS-877-1700 16GB
(HDD): [RAID-5] 4 x 8TB Seagate Ironwolf

Model:TS-569L 3GB
(HDD): [RAID-5] 4 x 3TB WD Red

P3R
Guru
Posts: 12185
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Slow transfer speed due to SSD cache acceleration

Post by P3R » Sat Jul 18, 2020 4:57 pm

klaasdj wrote:
Sat Jul 18, 2020 12:54 pm
What do you mean by senior members?
People that have extensive Qnap experience and that have spent much time in this forum trying to help less experienced users.
If Qtier is an algorithm for storing data on different tiers based on usage it should immediately place data that is for the first time written to the NAS on the SSD-tier.
15 minutes of reading to learn how Qtier actually works before selecting it would have saved you the day you now wasted on testing it having incorrect expectations of it's effect.
That is my experience with more professional solutions like Hitachi Vantara's DataOpps, Spectrum Protect (former IBM Tivoli Storage Manager), Symantec Enterprise Vault etc. do the job.
Unfortunately your experience with those professional solutions seem to have confused you more than help you this time as Qtier is different.
Most of the time these solutions even have a combination of caching and tiering in one and the same solution. or can be switched on or of on the same time.
Qtier can be combined with SSD caching, that can be switched on and off on the fly.

I think that Qtier can still only be switched on but not off. I'm not a Qtier user myself though so I'm not sure about the details.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

dolbyman
Guru
Posts: 18982
Joined: Sat Feb 12, 2011 2:11 am
Location: Vancouver BC , Canada

Re: Slow transfer speed due to SSD cache acceleration

Post by dolbyman » Sat Jul 18, 2020 10:04 pm

I use Qtier and it seems to work well

Via the pool write statistics you can sort of deduct how it works

Writes hit the fast tier first (then depending on usage, get offloaded to the lower tier right away or after a while) reads are coming from the low tier, unless they have been tiered up (scheduled task again) so folders with (e.g.) virtual machine vdisks with constant access get loaded into the tier for fast read

its sadly a very black box ..wish there was more transparency to it

oh also... I am 12 years old ..with 50 years of experience in "painting by numbers" and "selling fine leather jackets", I have 4 kids from 6 wifes a house with monkey and a horse
besides I invented the internet single handed, so there is that

MikeLagit
Easy as a breeze
Posts: 299
Joined: Fri Mar 22, 2013 11:40 pm

Re: Slow transfer speed due to SSD cache acceleration

Post by MikeLagit » Sat Jul 18, 2020 10:18 pm

Now that's funny! Lol. And here I always thought you were the great descendent of the king Earl Milton Dolby of Sound, who came off slave ships in the Africa West Indies....

Anything special with your setup to get it working for you Dolbyman? Anything I've tried the transfer speeds taper off madly once it's in production and I've always reverted back. Though I haven't tried it in 8 or 9 months, I doubt there's been any updates to it since then.
Model: TS-877-1700 16GB
(HDD): [RAID-5] 4 x 8TB Seagate Ironwolf

Model:TS-569L 3GB
(HDD): [RAID-5] 4 x 3TB WD Red

dolbyman
Guru
Posts: 18982
Joined: Sat Feb 12, 2011 2:11 am
Location: Vancouver BC , Canada

Re: Slow transfer speed due to SSD cache acceleration

Post by dolbyman » Sat Jul 18, 2020 11:25 pm

It might have to do with an optimal tier ratio

if you have a 20TB Storage pool and put a 256GB tier on it ..not sure it will do much good

for my 951x I have a 512GB Tier (-25% OP) for a 8TB thick volume ( 5x 4TB spinning RAID6 with some empty space)

for my 853Bu I have a 1TB Tier (-25% OP) for a 16TB thick volume ( 6x 6TB spinning RAID6 with some empty space)

On the 853Bu I hit the SSD write limit of around 500MB\s (10GbE network)
On the 951X I am limited externally by my crappy 5GbE adapter (hitting 370MB\s write)

both times sequential and random

P3R
Guru
Posts: 12185
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Slow transfer speed due to SSD cache acceleration

Post by P3R » Sun Jul 19, 2020 12:13 am

Some resources.

How to use Qtier.

Q: After upgrading a storage pool to use Qtier, why isn’t new data written to the SSD tier?
A: After upgrading a pool to a Qtier storage pool, Qtier does not start moving existing thick volume data to the SSD tier until the first scheduled auto-tiering task starts. This means that new data written to existing thick volumes before the first auto-tiering task is always stored in the HDD capacity tier.

Thin volumes and thick volumes created after upgrading to Qtier do not suffer from this limitation. To utilize the SSD tier immediately, create a new thick volume or convert your existing thick volumes to thin volumes.

Source: https://www.qnap.com/en/how-to/faq/arti ... e-ssd-tier

A review of Qnap Qtier and SSD caching by Storagereview.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

hardmack
New here
Posts: 5
Joined: Sun Jul 19, 2020 6:44 am

Re: Slow transfer speed due to SSD cache acceleration

Post by hardmack » Sat Aug 01, 2020 7:04 pm

Brand new to qnap and the ecosystem... using my TS-332X for file storage, cloud backups and plex/minecraft. Very simple featureset use in other words... Got 3x Ironwolfs and 3x 128gb ssds

I didnt spend enough time researching either I guess.... as a very general end user I thought SSD cache would assist across the board for file transfer and system tasks. Reading through about 50% of this thread and AFTER getting all my network transfer speed back AFTER disabling caching, I was wondering if there is a consensus on if its even worth it for my use case. It was odd seeing 30MB/s filetransfers, while still using 1gbe I was hoping to see a pegged 100+MB/s transfer especially with 400ish MB/s speed tests of the m.2's in QTS

Post Reply

Return to “System & Disk Volume Management”