ES1640DC V2 performance problem

[radcardnel]

ES1640DC V2

Firmware 2.0.0.0543

12 8TB HGST HUH728080AL5204 12Gbps drives

4 400GB Seagate ST400FM0233 SSD (cache)

Spinning disks are configured in two equal size RAID 10 arrays, one on Pool1 one on Pool2.

10Gb connections with jumbo frames (Intel X550 Dual 10Gb NIC for ISCSI).

Cisco switch confirms link speeds are 10Gb and ping -l 9000 confirms good jumbo frames end to end.

I've been troubleshooting performance issues. Created two test targets and LUNs, 200GB each (thick volumes), 128k SAN stripe size, 64k ReFS, direct attached via ISCSI.

Volume1 is on Pool1 - Sustained transfer (large file) of 400MB/s

Volume2 is on Pool2- Transfer starts fast enough but when the write cache fills up it averages about 7MB/s or less.

Driving me crazy! I even brought down SCA and SCB one at a time (failover works fine) thinking it could be a controller issue. Same results.

[storageman]

Please move to QES section

Sounds like a network issue
So pool1/vol1 on SCA
So pool2/vol2 on SCB
Are they connected/tested on same host?
Is dedupe/compression on?

[radcardnel]

Yes, connected to the same host. Compression is on, dedupe is off on both volumes. No network errors shown on SAN, 10Gb switch or server. All link speeds are confirmed at 10Gb full/duplex. I see flow control signals being sent but that would appear to be because data cant be written fast enough.

[storageman]

For the test stop IO on volume1.
So for volume2 is the same poor performance is shown whether using SCA or SCB controller?
The only way to trouble shoot this is to test different scenarios.
Are the disks/RAID choice for vol2 same as vol1?

[radcardnel]

The problem appears to be rectified. Pool2 is still a bit slower (writes will go to zero for a few seconds, then back up to 400 to 500MB/s) but that could be due to other loads on Pool2. There are several registry changes necessary for these units, mostly having to to do with ISCSI timeout for failover but it's possible other adjustments were made. I came across the Qutil_1.0.004.exe utility, ran it, rebooted and I'm assuming that helped with the issue. I had already tried shutting down SCA and SCB individually before running Qutil and got the same poor results on Pool2. I'll continue to monitor and test.
I strongly suggest that the documentation be updated to mention the QUTIL utility to automate registry modifications needed for ISCSI scenarios. Thanks for your input!

[storageman]

The problem appears to be rectified. Pool2 is still a bit slower (writes will go to zero for a few seconds, then back up to 400 to 500MB/s) but that could be due to other loads on Pool2. There are several registry changes necessary for these units, mostly having to to do with ISCSI timeout for failover but it's possible other adjustments were made. I came across the Qutil_1.0.004.exe utility, ran it, rebooted and I'm assuming that helped with the issue. I had already tried shutting down SCA and SCB individually before running Qutil and got the same poor results on Pool2. I'll continue to monitor and test.
I strongly suggest that the documentation be updated to mention the QUTIL utility to automate registry modifications needed for ISCSI scenarios. Thanks for your input!

Good info!
I'll raise the manual issue.