Speed up dir listing or best m.2 SATA SSD for Cache Acceleration?
Posted: Sat Nov 24, 2018 12:33 am
I have a TS-1635AX configured with 7+2+1 (RAID 6+Hot Spare) WD Gold drives, 4 Samsung 860 Pro SSD in RAID 10 (4 TB capacity). These are part of a storage pool using qtier. I use dual 10GbE. Drives are used via NFS.
I have found that when I need to iterate over the files in my system with the linux find command (I have many small files) -- the results take longer than I would like.
On our old SAN, such a search might take 2 minute. With the qnap, it takes more than 20 minutes. The network isn't saturated by this operation.
I have tried the noatime setting on the NFS client but this didn't help.
Is this just a consequence of NFS instead of iSCSI? I would be interested in any opinions on the matter, but NFS is critical for my application.
My next idea: Adding SSD Read-Only Cache might help with the directory reads for files housed on the HDDs.
Along these lines, I have some questions that I think the community might be able to help with:
#1: Will SSD cache also cache my existing SSD RAID Array?
- Presumably caching SSD confers no benefit, and because there would be fewer drives, a potential 50% performance penalty.
#2: Can the cache be set up only for Random, not sequential, IO?
- Sequential performance is already "okay" and I am not seeking to make any big gains here.
#3: I consider to purchase 2 SSDs for the Read-only Cache. Should Raid1 (my default) or RAID0 be used?
#4: What m.2 SATA SSD model is recommended? Considerations:
- Might be most intensively written SSD in the system.
- Intention is to increase speed of directory listings: has to be fast. Do larger capacities with more chips offer greater speed?
- Not looking for capacity particularly. If smaller sizes work and I can save a buck -- then I am happy.
- The Samsung 860 EVO seems like a good compromise in the m.2 form factor, but what about endurance? PM871a -- higher endurance but slower -- do I want slower in a cache?
#5: If a drive in an SSD cache configuration fails how does the QNAP behave? Will it keep working ignoring the failed drive? Will I have to reboot? Anyone experience this?
Thanks!
Aaron
P.S. I also have off-site backup, I know: RAID != Backup.
I have found that when I need to iterate over the files in my system with the linux find command (I have many small files) -- the results take longer than I would like.
On our old SAN, such a search might take 2 minute. With the qnap, it takes more than 20 minutes. The network isn't saturated by this operation.
I have tried the noatime setting on the NFS client but this didn't help.
Is this just a consequence of NFS instead of iSCSI? I would be interested in any opinions on the matter, but NFS is critical for my application.
My next idea: Adding SSD Read-Only Cache might help with the directory reads for files housed on the HDDs.
Along these lines, I have some questions that I think the community might be able to help with:
#1: Will SSD cache also cache my existing SSD RAID Array?
- Presumably caching SSD confers no benefit, and because there would be fewer drives, a potential 50% performance penalty.
#2: Can the cache be set up only for Random, not sequential, IO?
- Sequential performance is already "okay" and I am not seeking to make any big gains here.
#3: I consider to purchase 2 SSDs for the Read-only Cache. Should Raid1 (my default) or RAID0 be used?
#4: What m.2 SATA SSD model is recommended? Considerations:
- Might be most intensively written SSD in the system.
- Intention is to increase speed of directory listings: has to be fast. Do larger capacities with more chips offer greater speed?
- Not looking for capacity particularly. If smaller sizes work and I can save a buck -- then I am happy.
- The Samsung 860 EVO seems like a good compromise in the m.2 form factor, but what about endurance? PM871a -- higher endurance but slower -- do I want slower in a cache?
#5: If a drive in an SSD cache configuration fails how does the QNAP behave? Will it keep working ignoring the failed drive? Will I have to reboot? Anyone experience this?
Thanks!
Aaron
P.S. I also have off-site backup, I know: RAID != Backup.