Long time to in initialize new volume - is RAID0 any different?

[carlog]

I just got my first-ever QNAP device, a TVS-672xt with 6 x 16TB (really 14.5TB) HDDs. I'm going to use this to transport large quantities of data physically between two datacenters. I chose this device because it is compact and it has a 10GbE interface.

My first step was to create a RAID-5 static volume, which works out to 72TB in size. I'm am shocked and dismayed that initial synchronization is projected to take >100 hours! This seems completely unnecessary for a brand-new volume. It also makes me suspect that writing to the volume will be slower than I hoped. In principle, I should be able to write 72GB over a 10GbE link in less than 24 hours. Now I suspect writing that amount of data will take 4 times that long.

I am assuming that I would face the same long initial synchronization if I use a Storage pool and thick volume instead of using a Static volume. I chose a Static volume because it's support to offer the best performance.

Since this is for short-term transport of a one-time copy, I am now considering trying RAID0. Anybody know if that also requires synchronization of the entire volume contents on initial creation?

[dolbyman]

that just the initial setup, not the final speed

1GbE is capable to write round about 10TB in 24h (max transfer sustained), so 10GbE could do 10x that much

[holger_kuehn]

This is due to random IO operations with 64k blocks necessary for the first initialisation. Those are nowhere near the sequential write rates you seem to expect here, ap. 200 MB/s or so. Furthermore it is irrelevant if the filesystem contains any actual data or not, as the RAID is placed in deeper levels unaware of the actual file system used. So just wait for it to finish, you can already start transferring you data by the way.

As a small point of reference, my current RAID 6 out of 5x10TiB (27.26TiB) WD Red, took about 22h (with default sync priority) if I remember correctly. You might be able to speed this up a bit by setting this to resync first. I assume RAID 0 not to be much faster as the init should be similar with random IO. And as dolbymam said, do not confuse this with the transfer rates you get later when copying actual (sequential) data.

[carlog]

I cancelled (removed) that volume and created a RAID 0 volume instead. For RAID 0 there seemed to be no synchronization at all, though it spend about 30 minutes "Optimizing."

I then tried copying some large (>10GB) files to the new volume. Performance was limited to about 205GB/sec. The source server showed network traffic at about 1.7Gbps. I am guessing the limiting factor is now the disks. CPU and memory usage remained low on QNAP.

[dolbyman]

GB\s? ..you mean MB\s .. right?

transfer seems low 205MB\s should be any given modern HDD sequential speed..no RAID just single

[carlog]

No, I meant GB/sec.

[dolbyman]

so 205Gigabyte\s ... .. you would need a ~1.6Terrabit\s network .. here is a Wikipedia article for it .. might come in 2025

https://en.wikipedia.org/wiki/Terabit_Ethernet

You might rival facebook and google with those speeds .. is this you Jeff Bezos ?