SSD cache speed degradation

Questions about SNMP, Power, System, Logs, disk, & RAID.
Post Reply
Max1mizer
Starting out
Posts: 32
Joined: Sun Mar 26, 2017 8:54 pm

SSD cache speed degradation

Post by Max1mizer »

Guys, need some help.

Got TVS-872XT with 2x2TB ADATA XPG SX8200 Pro cache in RAID0 READ only. Using weekly performance tests, I got a pop up logging to QNAP about slowed cache (smth like this). Cache got 20% overprovisioning :

https://puu.sh/J8jvQ/a0e287de95.png

I know my QNAP got 2xGen3 x2 ports, but speeds were 1.2GB/s per drive when it was empty up to around 40% allocation (now 55%). Problems started from around 40% of allocation, when speed started slowing down. I use 10gbit, so total 1.2GB/s would be fine to have.

https://puu.sh/J8jvl/ec88454d18.png

Had 2xSamsung 512GB 970 and never had such problems, but with 10% overprovisioning I had like 800GB cache.

Any ideas?
nibbles2bits
Getting the hang of things
Posts: 50
Joined: Fri Aug 23, 2019 10:17 am

Re: SSD cache speed degradation

Post by nibbles2bits »

well it sounds like a case of partially filled blocks, provisioning, TRIM.
What have you found on that front? What filesystem is on those drives?
What sort of data is being read? videos? thumbnails?
dosborne
Experience counts
Posts: 1791
Joined: Tue May 29, 2018 3:02 am
Location: Ottawa, Ontario, Canada

Re: SSD cache speed degradation

Post by dosborne »

I (unfortunately?) don't have SSD caching in my units, but don't really need it for my purposes, but, as mentioned by @nibbles2bits, it may be useful to know what sort of data, apps, usage you are using it for. Maybe your use case is not cache-friendly?
QNAP TS-563-16G 5x10TB Seagate Ironwolf HDD Raid-5 NIC: 2x1GB 1x10GbE
QNAP TS-231P-US 2x18TB Seagate Exos HDD Raid-1
[Deadbolt and General Ransomware Detection, Prevention, Recovery & MORE]
nibbles2bits
Getting the hang of things
Posts: 50
Joined: Fri Aug 23, 2019 10:17 am

Re: SSD cache speed degradation

Post by nibbles2bits »

We could also probably agree that an ADATA is less common than Samsung 970s,
leading to less testing of new versions.

QNAP only lists ADATA model SX8200PNP with firmware S0118C for the TVS-872XT.
Does that match yours exactly?
Max1mizer
Starting out
Posts: 32
Joined: Sun Mar 26, 2017 8:54 pm

Re: SSD cache speed degradation

Post by Max1mizer »

My firmware is like this :

https://puu.sh/J8UCP/f596e808cb.png

It's QTS so ext4 caching around 30TB volume. Using 10gbit I store there lots of sceneries for flight simulation. I think ADATA is just crap... Am I able to optimize this cache? I'm caching static data.
nibbles2bits
Getting the hang of things
Posts: 50
Joined: Fri Aug 23, 2019 10:17 am

Re: SSD cache speed degradation

Post by nibbles2bits »

Absolutely you can optimize it. I'd figure out how to disable the SSD cache, then separate the two drives and wipe them clean, then remove one, then run the QNAP SSD Provisioning checker application on that drive. You will be told it takes like 25% to 30% provisioning to get optimal performance. You will see a graph. You can choose the exact percentage. Then set that and reinstall the both SSDs as a raid cache like you did before.

You'll get good as new performance because you wiped it clean.
You'll get sustained good performance because you left enough space.

That should only cost time and be dangerous. Sounds like a Saturday to me.
Too bad about the firmware revision on your devices. You might double
check that number using a different utility with ADATA's help.
Max1mizer
Starting out
Posts: 32
Joined: Sun Mar 26, 2017 8:54 pm

Re: SSD cache speed degradation

Post by Max1mizer »

Let's see. I removed the cache and made 40% over-provisioning (from 20%). Haven't done this SSD Tool, just doubled over-provisioning. Let's try, so far speed came back to 1.45GB/s and 200000+ IOPS.

https://puu.sh/Ja7ss/2422c9b6b6.png

Let's see how it'll play. Will be testing speeds per each 5% allocation. So far 10% and zero reduction.

https://puu.sh/Ja7sQ/8f2c4523e9.png

During previous setup with 20% over-provisioning, when allocation was 40% filled (3.82TB - 20% overprovisioning = ~3TB cache * 40% = 1,2TB filled), this cache SSD 2x2TB RAID0 Read-Only started to deteriorate super fast. From 1.4GB/s (up to 40% allocation) fast to 0.9GB/s (50%), fast to 0.7GB/s.(55% filled).
nibbles2bits
Getting the hang of things
Posts: 50
Joined: Fri Aug 23, 2019 10:17 am

Re: SSD cache speed degradation

Post by nibbles2bits »

That's good to hear it's working much better.
Max1mizer wrote: Sat Jul 09, 2022 4:11 am Let's see how it'll play. Will be testing speeds per each 5% allocation. So far 10% and zero reduction.
That's what the SSD Provisioning tool does for you while you have snacks :)
I'm not sure if you've investigated how a 512MB cache drive performs versus a 1TB versus a 2TB cache drive of the same make and model given your type of read/writes. There are websites that try to catalog drive stats for you, but you can do the same sort of thing with Amorphus Diskmark afaik. In general the small drives work better for a lot of tasks, and that's why QNAP uses an array of 4 tiny 512 MB SSDs to demonstrate capabilities in their inline videos.

If pure speed is what you need, you'll have to experiment I think beyond just provisioning and wiping your cache drives every one in a while.
I'd ask in the storage review forums, maybe at anand tech, for some insight.
Max1mizer
Starting out
Posts: 32
Joined: Sun Mar 26, 2017 8:54 pm

Re: SSD cache speed degradation

Post by Max1mizer »

Before 2x2TB I had 2x512gb cache. Same raid0 read only. One ssd was samsung 950pro second 960 or 970. Over-provisioning was 10% and they were filled 100% and zero performance drop.
nibbles2bits wrote: Tue Jul 12, 2022 3:12 pm If pure speed is what you need, you'll have to experiment I think beyond just provisioning and wiping your cache drives every one in a while.
I'd ask in the storage review forums, maybe at anand tech, for some insight.
My idea is to keep 10Gbit/s speed on 100% cache usage. If 10Gbit/s is 1.2GB/s I need 600MB/s read per one NVME. This level was filled last time on 20% over-provisioning on 55% cache filling.
Post Reply

Return to “System & Disk Volume Management”