Single file size exceeded 15 Terabytes

Questions about SNMP, Power, System, Logs, disk, & RAID.
Post Reply
ruimota
New here
Posts: 7
Joined: Thu Sep 17, 2015 5:27 am

Single file size exceeded 15 Terabytes

Post by ruimota » Wed Aug 28, 2019 5:09 pm

Hello,

One of our storage NAS QNAP got one file that grow up to 15,5 Terabytes in size.

After some research it seems that the maximum single file size for EXT4 is 15 TB.

Couldn't find any oficial information from QNAP regarding maximum single file size, so i am searching for help.

The main questions are:
1) Whats the real maximum single file size for QNAP NAS (TVS-871U-RP FW 4.3.6.0993) ?
2) If it exceeds the maximum data (file) wild be discarded or prevented access to it?

Currently that single file size is increasing but at a lower rate and i am trying to avoid the loss of this big file.

Any info about this limits would be a precious help.

Thanks,

RM

User avatar
OneCD
Ask me anything
Posts: 6389
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: Single file size exceeded 15 Terabytes

Post by OneCD » Thu Aug 29, 2019 9:55 am

ruimota wrote:
Wed Aug 28, 2019 5:09 pm
One of our storage NAS QNAP got one file that grow up to 15,5 Terabytes in size.
Windows runtime error log? :wink:
ruimota wrote:
Wed Aug 28, 2019 5:09 pm
After some research it seems that the maximum single file size for EXT4 is 15 TB.
With the QTS Linux kernel and hardware it's designed for, the maximum single file-size is 16TiB. This limit is determined (in-part) by the memory page-size (4096 bytes), and is hardcoded into the Linux kernel. This size limit can be overcome, but would require some kernel customisation.
ruimota wrote:
Wed Aug 28, 2019 5:09 pm
2) If it exceeds the maximum data (file) wild be discarded or prevented access to it?
Sorry, I don't know. But it would be interesting to see. My guess is there would be a slow crash. :lol:
ruimota wrote:
Wed Aug 28, 2019 5:09 pm
Currently that single file size is increasing but at a lower rate and i am trying to avoid the loss of this big file.
I've got to ask - why do you have a single file of 15.5TB?

production NAS: TS-569 Pro with Debian 9.9 'Stretch' (power on/off times are < 1 minute)
backup NAS: TS-559 Pro+ with QTS 4.2.6 #20190921

one.cd.only@gmail.com

Image Image Image Image

User avatar
Toxic17
Ask me anything
Posts: 5201
Joined: Tue Jan 25, 2011 11:41 pm
Location: Planet Earth
Contact:

Re: Single file size exceeded 15 Terabytes

Post by Toxic17 » Thu Aug 29, 2019 9:31 pm

OneCD wrote:
Thu Aug 29, 2019 9:55 am
I've got to ask - why do you have a single file of 15.5TB?
Indeed, how is this file going to be read or opened? lol
Regards Simon

QTS 4.x User Guidex

QNAP Club Repository
Submit a ticket • QNAP Helpdesk
QNAP Tutorials, User Manuals, FAQs, Downloads, Wiki
When you ask a question, please include the following


NAS: TS-473-32GB QM2-2P QXG-10G1T 4.4.1.1081 • TVS-463-16GB 4.4.1.1081 QM2-2S10G1TB • TS-459 Pro 2GB 4.2.6 • TS-121 4.3.3.0998 • APC Back-UPS ES 700G •
QPKG's: TwonkyServer 8.51 • Apache73 v2441.7310 • QSonarr 3.0.3.644 • QNBZGet 21.0 • phpMyAdmin 4.9.0.1 • Qmono 5.20.1.19 • McAfee 3.0.1 • Lychee 3.2.16 • HBS 3.0.1908029 • LEgo v3.0.0
Network: VM Hub 3.0 <500/35> • UniFi USG Pro 4 • UniFi USW-16-150W • UniFi USW-8-60W • UniFi CloudKey Gen2+• UniFi G3-Flex • UAP AC Pro • UAP AC Lite • SLM2008 • Dell 7050 MFF •

User avatar
OneCD
Ask me anything
Posts: 6389
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: Single file size exceeded 15 Terabytes

Post by OneCD » Thu Aug 29, 2019 10:26 pm

Toxic17 wrote:
Thu Aug 29, 2019 9:31 pm
Indeed, how is this file going to be read or opened? lol
My guess is: very carefully. :DD

production NAS: TS-569 Pro with Debian 9.9 'Stretch' (power on/off times are < 1 minute)
backup NAS: TS-559 Pro+ with QTS 4.2.6 #20190921

one.cd.only@gmail.com

Image Image Image Image

ruimota
New here
Posts: 7
Joined: Thu Sep 17, 2015 5:27 am

Re: Single file size exceeded 15 Terabytes

Post by ruimota » Fri Aug 30, 2019 7:13 am

Hi,

The size of the file is increasing (at a slow pace) and the software process behind it...is running fine.

I guess that if and when this file hits 16 Terabytes, QNAP Storage EXT4 will colapse.

Since the software process behind it is limited to 100 Terabytes, and data volume has 25 Terabytes capacity...i will wait to see what happens 8) .

Took an image from File Station, maybe thats something unusual to see in Storage QNAP.

Thanks for the reply and info.

RM
You do not have the required permissions to view the files attached to this post.

User avatar
OneCD
Ask me anything
Posts: 6389
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: Single file size exceeded 15 Terabytes

Post by OneCD » Fri Aug 30, 2019 7:16 am

What's in that file? It's not a virtual-machine image, is it?

production NAS: TS-569 Pro with Debian 9.9 'Stretch' (power on/off times are < 1 minute)
backup NAS: TS-559 Pro+ with QTS 4.2.6 #20190921

one.cd.only@gmail.com

Image Image Image Image

User avatar
Toxic17
Ask me anything
Posts: 5201
Joined: Tue Jan 25, 2011 11:41 pm
Location: Planet Earth
Contact:

Re: Single file size exceeded 15 Terabytes

Post by Toxic17 » Fri Aug 30, 2019 4:25 pm

OneCD wrote:
Fri Aug 30, 2019 7:16 am
What's in that file? It's not a virtual-machine image, is it?
maybe a Virtual image with a ton of vSphere snapshots perhaps?
Regards Simon

QTS 4.x User Guidex

QNAP Club Repository
Submit a ticket • QNAP Helpdesk
QNAP Tutorials, User Manuals, FAQs, Downloads, Wiki
When you ask a question, please include the following


NAS: TS-473-32GB QM2-2P QXG-10G1T 4.4.1.1081 • TVS-463-16GB 4.4.1.1081 QM2-2S10G1TB • TS-459 Pro 2GB 4.2.6 • TS-121 4.3.3.0998 • APC Back-UPS ES 700G •
QPKG's: TwonkyServer 8.51 • Apache73 v2441.7310 • QSonarr 3.0.3.644 • QNBZGet 21.0 • phpMyAdmin 4.9.0.1 • Qmono 5.20.1.19 • McAfee 3.0.1 • Lychee 3.2.16 • HBS 3.0.1908029 • LEgo v3.0.0
Network: VM Hub 3.0 <500/35> • UniFi USG Pro 4 • UniFi USW-16-150W • UniFi USW-8-60W • UniFi CloudKey Gen2+• UniFi G3-Flex • UAP AC Pro • UAP AC Lite • SLM2008 • Dell 7050 MFF •

Post Reply

Return to “System & Disk Volume Management”