Hello,
I am using a QNAP TS869L which provides a share as a backup location. The backup Software I am using is Veeam Backup.
With "strict allocation" set to "yes" in the smb.conf file, the "processing rate" of my Full Backup decreases while it runs. After 10 hours, the processing rate has dropped from 100 MB/s to less than 30 MB/s.
After I set "strict allocation" to "no" the performance is just fine.
As I understand it from this article (https://wiki.samba.org/index.php/Linux_Performance), strict allocation in combination with ext4 should improve performance, not make it worse.
Can anyone explain to me why this is happening?
"strict allocation" set to "yes" slows file transfer down although the file system is ext4
-
- New here
- Posts: 3
- Joined: Tue May 18, 2021 5:36 pm
- Location: Germany
-
- Experience counts
- Posts: 1081
- Joined: Thu Aug 24, 2017 10:28 pm
Re: "strict allocation" set to "yes" slows file transfer down although the file system is ext4
So on one hand we have "After 10 hours, the processing rate has dropped from 100 MB/s to less than 30 MB/s" and on the other we have "the performance is just fine".
What does this tell us? Not much.
Give us some real hard numbers so we can actually compare, such as: Total time taken from backup start to finish, and average throughput. With strict allocation ON, vs. strict allocation OFF.
Maybe the kernel or glibc on your NAS is too old to support extent allocation, and glibc or Samba reverts to its respective fallback behavior, which may be worse than with strict allocation OFF.
Are snapshots enabled on the volume where backups are stored?
What does this tell us? Not much.
Give us some real hard numbers so we can actually compare, such as: Total time taken from backup start to finish, and average throughput. With strict allocation ON, vs. strict allocation OFF.
Maybe the kernel or glibc on your NAS is too old to support extent allocation, and glibc or Samba reverts to its respective fallback behavior, which may be worse than with strict allocation OFF.
Are snapshots enabled on the volume where backups are stored?
-
- New here
- Posts: 3
- Joined: Tue May 18, 2021 5:36 pm
- Location: Germany
Re: "strict allocation" set to "yes" slows file transfer down although the file system is ext4
Hi Mousetick,
The file that must be transferred is around 10 TB in size.
The NAS is connected to the server via a 1Gbit connection.
Snapshots are disabled.
The volume on the NAS is a "Thick-Volume".
Unfortunately, I cannot give a direct comparison, as I have never run the backup with strict allocation ON, because it simply takes too long.
With strict allocation OFF the 9.6 TB were transferred in 29 hours 37 minutes with an average Processing rate of 104 MB/s.
The file that must be transferred is around 10 TB in size.
The NAS is connected to the server via a 1Gbit connection.
Snapshots are disabled.
The volume on the NAS is a "Thick-Volume".
Unfortunately, I cannot give a direct comparison, as I have never run the backup with strict allocation ON, because it simply takes too long.
With strict allocation OFF the 9.6 TB were transferred in 29 hours 37 minutes with an average Processing rate of 104 MB/s.
-
- Experience counts
- Posts: 1081
- Joined: Thu Aug 24, 2017 10:28 pm
Re: "strict allocation" set to "yes" slows file transfer down although the file system is ext4
I see
Well I don't know what to tell you then.
On a plain Linux distribution there is a useful command, fallocate, which can be used to test extent allocation, but it's not available on crippled QNAP Linux.
My guess is that, as I've speculated above, your NAS is too old and doesn't really support extent allocation, despite using ext4, so it is emulated by either the C runtime library or Samba, which results in a lot of small disk writes on the NAS, possibly causing timeouts on the Veeam side. Anything noteworthy on the Veeam side, messages in log perhaps?
Either that or there is an incompatibility between Veeam and Samba with strict allocation ON. Have you tried researching this topic on the Veeam side?
I'm not familiar with this product. Does it create one gigantic file on the NAS to hold the backup? Does it pre-allocate 10 TB in one go at the beginning of the backup?
You could try to monitor disk activity on the NAS (Resource Monitor) during the pre-allocation phase. If extent allocation is working as it should, there should be very little disk activity. If it's not working and is emulated instead, there could be a lot of disk thrashing for a while.
Well I don't know what to tell you then.
On a plain Linux distribution there is a useful command, fallocate, which can be used to test extent allocation, but it's not available on crippled QNAP Linux.
My guess is that, as I've speculated above, your NAS is too old and doesn't really support extent allocation, despite using ext4, so it is emulated by either the C runtime library or Samba, which results in a lot of small disk writes on the NAS, possibly causing timeouts on the Veeam side. Anything noteworthy on the Veeam side, messages in log perhaps?
Either that or there is an incompatibility between Veeam and Samba with strict allocation ON. Have you tried researching this topic on the Veeam side?
I'm not familiar with this product. Does it create one gigantic file on the NAS to hold the backup? Does it pre-allocate 10 TB in one go at the beginning of the backup?
You could try to monitor disk activity on the NAS (Resource Monitor) during the pre-allocation phase. If extent allocation is working as it should, there should be very little disk activity. If it's not working and is emulated instead, there could be a lot of disk thrashing for a while.
-
- New here
- Posts: 3
- Joined: Tue May 18, 2021 5:36 pm
- Location: Germany
Re: "strict allocation" set to "yes" slows file transfer down although the file system is ext4
Thank you already for your quick answers!
The NAS is more than 5 years old, so it could be related to that.
In general, I was able to solve the problem in cooperation with Veeam support. I am only interested in why the problem occurs with strict allocation turned ON.
The NAS is more than 5 years old, so it could be related to that.
In general, I was able to solve the problem in cooperation with Veeam support. I am only interested in why the problem occurs with strict allocation turned ON.