I looking for bottleneck 10Gbe

FTP Server, File Server, DDNS, SAMBA, AFP, NFS
Post Reply
Ivqo77
New here
Posts: 4
Joined: Fri Sep 06, 2019 4:43 am

I looking for bottleneck 10Gbe

Post by Ivqo77 »

Hi. I looking for bottleneck in my LAN.

sys: QNAP TS431x (8GB) 4.4.1.1031 running factory setting,no additional service running. SSD Samsung 850EVO 120GB,2x SSD Apacer AS340 960GB RAID 0
pc: WIN10, I7-5820K (4.5GHz) ASUS X99A,16GB RAM,NVIDIA 1080,SSD7101A-1 4xM.2 Port, 3xSSD Samsung 970Pro,960Pro,950Pro,HDD RAID 0 2xWD 1,5TB,GTX1080,ASUS XG-C100C with active fan
switch: Netgear XS508M 8x10Gb+1 sfp+ 10Gb,NAS connected to SFP+,Port 1 Router ASUS RT-AC68U cat5e,Port 2 pc cat7,Port 3 old pc cat5e

Disk read performance test in QNAP before create volumes:
SSD 850 EVO 529 MB/s,1. AS340 407MB/s,2.AS340 405MB/s

test file: test40GByte.ISO QNAP CPU 30%-90% load
Jumbo set 9000/1500 (PC/QNAP) some result

WRITE from SSD RAID 0 to SSD EVO cca 160MB/s Midnight Commander (PUTTY)
WRITE from SSD EVO to SSD RAID 0 cca 330MB/s Midnight Commander (PUTTY)
WRITE from SSD EVO to SSD RAID 0 cca 300-350MB/s FileStation5



WRITE from SSD RAID 0 to pc (970Pro) cca 250/MBs Total Commander (SMB) QNAP>PC
WRITE from pc(950pro) to SSD RAID 0 cca 285MB/s Total Commander (SMB) PC>QNAP
WRITE from pc(960pro) to SSD EVO cca 160MB/s Total Commander (SMB) PC>QNAP

Iperf3: Transfer 4.49GBytes, Bandwidth 3.86GBits/s sender/receiver

QNAP says: TS-431X delivers up to 956 MB/s throughput (4x Intel SSD)
My 2xSSD gave 1/3 performance :-(


May be bottleneck in my switch? Currently I dont have ability connect NAS to PC directly. I looking for Netgear performance I found nothing. I newbie in 10GB networking and English. Thx for ansfer :-)
User avatar
Moogle Stiltzkin
Guru
Posts: 11448
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: I looking for bottleneck 10Gbe

Post by Moogle Stiltzkin »

How to get the most from 10GbE networking: a practical guide
https://blog.qnap.com/10gbe-walkthrough-en/

:'
NAS
[Main Server] QNAP TS-877 (QTS) w. 4tb [ 3x HGST Deskstar NAS & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A (Truenas Core) w. 4x 2TB Samsung F3 (HD203WI) RaidZ1 ZFS + 8gb ddr3 Crucial
[^] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) 2x 4TB Seagate Ironwolf, Raid5
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-253D (Truenas Scale)
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Qotom Pfsense|100mbps FTTH | Win11, Ryzen 5600X Desktop (1x2tb Crucial P50 Plus M.2 SSD, 1x 8tb seagate Ironwolf,1x 4tb HGST Ultrastar 7K4000)


Resources
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin
User avatar
Toxic17
Ask me anything
Posts: 6469
Joined: Tue Jan 25, 2011 11:41 pm
Location: Planet Earth
Contact:

Re: I looking for bottleneck 10Gbe

Post by Toxic17 »

You could get an idea of the current setup speeds of your drives using the following commands in an SSH session

Code: Select all

qcli_storage -d

Code: Select all

qcli_storage -T force=1

Code: Select all

qcli_storage -t force=1
Regards Simon

Qnap Downloads
MyQNap.Org Repository
Submit a ticket • QNAP Helpdesk
QNAP Tutorials, User Manuals, FAQs, Downloads, Wiki
When you ask a question, please include the following


NAS: TS-673A QuTS hero h5.1.2.2534 • TS-121 4.3.3.2420 • APC Back-UPS ES 700G
Network: VM Hub3: 500/50 • UniFi UDM Pro: 3.2.9 • UniFi Network Controller: 8.0.28
USW-Aggregation: 6.6.61 • US-16-150W: 6.6.61 • 2x USW Mini Flex 2.0.0 • UniFi AC Pro 6.6.62 • UniFi U6-LR 6.6.62
UniFi Protect: 2.11.21/8TB Skyhawk AI • 3x G3 Instants: 4.69.55 • UniFi G3 Flex: 4.69.55 • UniFi G5 Flex: 4.69.55
Ivqo77
New here
Posts: 4
Joined: Fri Sep 06, 2019 4:43 am

Re: I looking for bottleneck 10Gbe

Post by Ivqo77 »

login as: admin
admin@192.168.1.250's password:
[~] # qcli_storage -d
Enclosure Port Sys_Name Type Size Alias Signature Partitions Mode l
NAS_HOST 2 /dev/sda SSD:data 111.79 GB -- QNAP FLEX 5 Sams ung SSD 850 EVO 120GB
NAS_HOST 3 /dev/sdb SSD:data 894.25 GB -- QNAP FLEX 5 Apac er AS340 960GB
NAS_HOST 4 /dev/sdc SSD:data 894.25 GB -- QNAP FLEX 5 Apac er AS340 960GB
[~] # qcli_storage -T force=1
fio test command for physical disk: /sbin/fio --filename=test_device --direct=1 --rw=read --bs=1M --runti me=15 --name=test-read --ioengine=libaio --iodepth=32 &>/tmp/qcli_storage.log
fio test command for RAID: /sbin/fio --filename=test_device --direct=0 --rw=read --bs=1M --runtime=15 --n ame=test-read --ioengine=libaio --iodepth=32 &>/tmp/qcli_storage.log
Start testing!
Performance test is finished 100.000%...
Enclosure Port Sys_Name Throughput RAID RAID_Type RAID_Throughput Pool
NAS_HOST 2 /dev/sda 528.12 MB/s /dev/md2 Single 530.43 MB/s 289
NAS_HOST 3 /dev/sdb 508.43 MB/s /dev/md1 RAID 0 1.01 GB/s 288
NAS_HOST 4 /dev/sdc 508.47 MB/s /dev/md1 RAID 0 1.01 GB/s 288
[~] # qcli_storage -t force=1
fio test command for LV layer: /sbin/fio --filename=test_device --direct=0 --rw=read --bs=1M --runtime=15 --name=test-read --ioengine=libaio --iodepth=32 &>/tmp/qcli_storage.log
fio test command for File system: /sbin/fio --filename=test_device/qcli_storage --direct=0 --rw=read --bs =1M --runtime=15 --name=test-read --ioengine=libaio --iodepth=32 --size=128m &>/tmp/qcli_storage.log
Start testing!
Performance test is finished 100.000%...
VolID VolName Pool Mapping_Name Throughput Mount_Path FS_Throughput
1 Ivqo 288 /dev/mapper/cachedev2 1.00 GB/s /share/CACHEDEV2_DATA 859.06 MB/s
2 BackUp 289 /dev/mapper/cachedev3 529.03 MB/s /share/CACHEDEV3_DATA 524.59 MB/s
[~] #
Ivqo77
New here
Posts: 4
Joined: Fri Sep 06, 2019 4:43 am

Re: I looking for bottleneck 10Gbe

Post by Ivqo77 »

Set jumbo to 9000 reading from NAS increase to 460 MB/s write not change, some as previous.
How get 1GB/s to PC? :roll: :lol: :roll:
User avatar
Toxic17
Ask me anything
Posts: 6469
Joined: Tue Jan 25, 2011 11:41 pm
Location: Planet Earth
Contact:

Re: I looking for bottleneck 10Gbe

Post by Toxic17 »

when setting MTU it needs to be 9000 on NAS, Switch and PC.

make sure any 10GbE cables are CAT6a or higher.
Regards Simon

Qnap Downloads
MyQNap.Org Repository
Submit a ticket • QNAP Helpdesk
QNAP Tutorials, User Manuals, FAQs, Downloads, Wiki
When you ask a question, please include the following


NAS: TS-673A QuTS hero h5.1.2.2534 • TS-121 4.3.3.2420 • APC Back-UPS ES 700G
Network: VM Hub3: 500/50 • UniFi UDM Pro: 3.2.9 • UniFi Network Controller: 8.0.28
USW-Aggregation: 6.6.61 • US-16-150W: 6.6.61 • 2x USW Mini Flex 2.0.0 • UniFi AC Pro 6.6.62 • UniFi U6-LR 6.6.62
UniFi Protect: 2.11.21/8TB Skyhawk AI • 3x G3 Instants: 4.69.55 • UniFi G3 Flex: 4.69.55 • UniFi G5 Flex: 4.69.55
Ivqo77
New here
Posts: 4
Joined: Fri Sep 06, 2019 4:43 am

Re: I looking for bottleneck 10Gbe

Post by Ivqo77 »

switch unmanaged jumbo 9K support,pc and qnap set to 9000,qnap to switch sfp+ fibre,pc to switch cat7 cable. any next idea
User avatar
Trexx
Ask me anything
Posts: 5393
Joined: Sat Oct 01, 2011 7:50 am
Location: Minnesota

Re: I looking for bottleneck 10Gbe

Post by Trexx »

Not all SSD's are equal in performance so brand/model/etc. matters. Also difference SSD's will drop in performance faster/slower depending on when you overrun the cache buffer in them (hence SSD over-provisioning). Also make sure you have SMB set to 3.x on QNAP.

CPU also matters, I know Bob Zelin our resident video editing guru doen't recommend ANY Arm based models for 10GbE usage (other than backup) as he wasn't able to get full 10GbE throughput out of them even with much bigger models.
Paul

Model: TS-877-1600 FW: 4.5.3.x
QTS (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
Data (HDD): [RAID-5] 6 x 3TB HGST DeskStar
VMs (SSD): [RAID-1] 2 x1TB SK Hynix Gold
Ext. (HDD): TR-004 [Raid-5] 4 x 4TB HGST Ultastor
RAM: Kingston HyperX Fury 64GB DDR4-2666
UPS: CP AVR1350

Model:TVS-673 32GB & TS-228a Offline[/color]
-----------------------------------------------------------------------------------------------------------------------------------------
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq
Bob Zelin
Experience counts
Posts: 1374
Joined: Mon Nov 21, 2016 12:55 am
Location: Orlando, FL.
Contact:

Re: I looking for bottleneck 10Gbe

Post by Bob Zelin »

Ivqo77 writes
"Set jumbo to 9000 reading from NAS increase to 460 MB/s write not change, some as previous.
How get 1GB/s to PC?"

REPLY - you should be jumping up and down with happiness with these results. Stop looking at the specs on the QNAP website. I only do 10G systems. With a TS-453BT3 setup perfectly, if you get 450 MB/sec you are doing great. You want to do 1000 MB/sec - well, first, you are not going to do 1000 MB/sec on a Mac (unless you get lucky). If you get 800 - 900 MB/sec with an EIGHT BAY QNAP with an Intel or AMD Ryzen processor, you are lucky. Even with a 16 bay TS-1683XU-RP top of the line QNAP, with 16 drives, and a 10G connection, if you use a Win 10 PC with the QNAP QXG-10G1T NIC card, THEN you will get 1000 MB/sec (1Gb/sec). It's not going to happen with a 4 bay, and a cheap 4 bay at that.

And lets go one step further than the TS-453BT3 - what if you buy the best 4 bay that QNAP is making - the TVS-482XT. NOPE - only 4 drives.

So now you will say "but, but, but - it says it can do it on the QNAP website". Welcome to the real world. Synology and Netgear say a lot of things too.

And to end this (for those that do research) - how do you achieve 1250 MB/sec on a 10G line (SFP+ or 10Gbase-T) - simply answer - you CANT DO IT, and you can't exceed these speeds. 1250 MB/sec is the THEORETICAL bandwidth, and in REAL LIFE, you don't get these speeds - just like you can't get 125 MB/sec on a 1G ethernet line - if you get 110 MB/sec its a miracle - in real life, it's more like 90 - 100 MB/sec. Everyone keeps looking at these specs on the company websites, and expects a miracle. Well, you are not getting it. No matter what you do.

When the world is all 40G with all NVMe's, then things will be different.

Bob Zelin
Bob Zelin / Rescue 1, Inc.
http://www.bobzelin.com
Post Reply

Return to “File Sharing”