I looking for bottleneck 10Gbe

FTP Server, File Server, DDNS, SAMBA, AFP, NFS
Post Reply
Ivqo77
New here
Posts: 4
Joined: Fri Sep 06, 2019 4:43 am

I looking for bottleneck 10Gbe

Post by Ivqo77 » Tue Sep 10, 2019 2:30 am

Hi. I looking for bottleneck in my LAN.

sys: QNAP TS431x (8GB) 4.4.1.1031 running factory setting,no additional service running. SSD Samsung 850EVO 120GB,2x SSD Apacer AS340 960GB RAID 0
pc: WIN10, I7-5820K (4.5GHz) ASUS X99A,16GB RAM,NVIDIA 1080,SSD7101A-1 4xM.2 Port, 3xSSD Samsung 970Pro,960Pro,950Pro,HDD RAID 0 2xWD 1,5TB,GTX1080,ASUS XG-C100C with active fan
switch: Netgear XS508M 8x10Gb+1 sfp+ 10Gb,NAS connected to SFP+,Port 1 Router ASUS RT-AC68U cat5e,Port 2 pc cat7,Port 3 old pc cat5e

Disk read performance test in QNAP before create volumes:
SSD 850 EVO 529 MB/s,1. AS340 407MB/s,2.AS340 405MB/s

test file: test40GByte.ISO QNAP CPU 30%-90% load
Jumbo set 9000/1500 (PC/QNAP) some result

WRITE from SSD RAID 0 to SSD EVO cca 160MB/s Midnight Commander (PUTTY)
WRITE from SSD EVO to SSD RAID 0 cca 330MB/s Midnight Commander (PUTTY)
WRITE from SSD EVO to SSD RAID 0 cca 300-350MB/s FileStation5



WRITE from SSD RAID 0 to pc (970Pro) cca 250/MBs Total Commander (SMB) QNAP>PC
WRITE from pc(950pro) to SSD RAID 0 cca 285MB/s Total Commander (SMB) PC>QNAP
WRITE from pc(960pro) to SSD EVO cca 160MB/s Total Commander (SMB) PC>QNAP

Iperf3: Transfer 4.49GBytes, Bandwidth 3.86GBits/s sender/receiver

QNAP says: TS-431X delivers up to 956 MB/s throughput (4x Intel SSD)
My 2xSSD gave 1/3 performance :-(


May be bottleneck in my switch? Currently I dont have ability connect NAS to PC directly. I looking for Netgear performance I found nothing. I newbie in 10GB networking and English. Thx for ansfer :-)

User avatar
Moogle Stiltzkin
Ask me anything
Posts: 7611
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: I looking for bottleneck 10Gbe

Post by Moogle Stiltzkin » Tue Sep 10, 2019 2:37 am

How to get the most from 10GbE networking: a practical guide
https://blog.qnap.com/10gbe-walkthrough-en/

:'
NAS
[Main Server] QNAP TS-877 w. 4tb [ 3x HGST Deskstar NAS (HDN724040ALE640) & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 + 16gb ddr4 Crucial + QWA-AC2600 wireless adapter.
[Backup] QNAP TS-653A w. 5x 2TB Samsung F3 (HD203WI) EXT4 Raid5
[^] QNAP TS-659 Pro II 1x 4TB HGST Deskstar NAS
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-228 w. 1x 1TB WD RE3 (WD1002FBYS)
[^] QNAP TS-128
Mobile NAS TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Asus AC68U Router|100dl/50ul MBPS FTTH Internet | Windows 10, WC PC-Intel i7 920 Ivy bridge desktop (1x 512gb Samsung 850 Pro SSD + 1x 4tb HGST Ultrastar 7K4000)


Guides & articles
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review

https://www.patreon.com/mooglestiltzkin

User avatar
Toxic17
Ask me anything
Posts: 5158
Joined: Tue Jan 25, 2011 11:41 pm
Location: Planet Earth
Contact:

Re: I looking for bottleneck 10Gbe

Post by Toxic17 » Tue Sep 10, 2019 2:57 am

You could get an idea of the current setup speeds of your drives using the following commands in an SSH session

Code: Select all

qcli_storage -d

Code: Select all

qcli_storage -T force=1

Code: Select all

qcli_storage -t force=1
Regards Simon

QTS 4.x User Guidex

QNAP Club Repository
Submit a ticket • QNAP Helpdesk
QNAP Tutorials, User Manuals, FAQs, Downloads, Wiki
When you ask a question, please include the following


NAS: TS-473-32GB QM2-2P QXG-10G1T 4.4.1.1064 • TVS-463-16GB 4.4.1.1064 QM2-2S10G1TB • TS-459 Pro 2GB 4.2.6 • TS-121 4.3.3.0998 • APC Back-UPS ES 700G •
QPKG's: TwonkyServer 8.51 • Apache73 • QSonarr 3.0.1.503 • QNBZGet 21.0 • phpMyAdmin 4.9.0.1 • Qmono 5.20.1.19 • McAfee 2.2.1 • Lychee 3.2.15 • HBS 3.0.190802 • LEgo v3.0.0
Network: VM Hub 3.0 <500/35> • UniFi USG Pro 4 • UniFi USW-16-150W • UniFi USW-8-60W • UniFi CloudKey Gen2+• UniFi G3-Flex • UAP AC Pro • UAP AC Lite • SLM2008 • Dell 7050 MFF •

Ivqo77
New here
Posts: 4
Joined: Fri Sep 06, 2019 4:43 am

Re: I looking for bottleneck 10Gbe

Post by Ivqo77 » Tue Sep 10, 2019 3:27 am

login as: admin
admin@192.168.1.250's password:
[~] # qcli_storage -d
Enclosure Port Sys_Name Type Size Alias Signature Partitions Mode l
NAS_HOST 2 /dev/sda SSD:data 111.79 GB -- QNAP FLEX 5 Sams ung SSD 850 EVO 120GB
NAS_HOST 3 /dev/sdb SSD:data 894.25 GB -- QNAP FLEX 5 Apac er AS340 960GB
NAS_HOST 4 /dev/sdc SSD:data 894.25 GB -- QNAP FLEX 5 Apac er AS340 960GB
[~] # qcli_storage -T force=1
fio test command for physical disk: /sbin/fio --filename=test_device --direct=1 --rw=read --bs=1M --runti me=15 --name=test-read --ioengine=libaio --iodepth=32 &>/tmp/qcli_storage.log
fio test command for RAID: /sbin/fio --filename=test_device --direct=0 --rw=read --bs=1M --runtime=15 --n ame=test-read --ioengine=libaio --iodepth=32 &>/tmp/qcli_storage.log
Start testing!
Performance test is finished 100.000%...
Enclosure Port Sys_Name Throughput RAID RAID_Type RAID_Throughput Pool
NAS_HOST 2 /dev/sda 528.12 MB/s /dev/md2 Single 530.43 MB/s 289
NAS_HOST 3 /dev/sdb 508.43 MB/s /dev/md1 RAID 0 1.01 GB/s 288
NAS_HOST 4 /dev/sdc 508.47 MB/s /dev/md1 RAID 0 1.01 GB/s 288
[~] # qcli_storage -t force=1
fio test command for LV layer: /sbin/fio --filename=test_device --direct=0 --rw=read --bs=1M --runtime=15 --name=test-read --ioengine=libaio --iodepth=32 &>/tmp/qcli_storage.log
fio test command for File system: /sbin/fio --filename=test_device/qcli_storage --direct=0 --rw=read --bs =1M --runtime=15 --name=test-read --ioengine=libaio --iodepth=32 --size=128m &>/tmp/qcli_storage.log
Start testing!
Performance test is finished 100.000%...
VolID VolName Pool Mapping_Name Throughput Mount_Path FS_Throughput
1 Ivqo 288 /dev/mapper/cachedev2 1.00 GB/s /share/CACHEDEV2_DATA 859.06 MB/s
2 BackUp 289 /dev/mapper/cachedev3 529.03 MB/s /share/CACHEDEV3_DATA 524.59 MB/s
[~] #

Ivqo77
New here
Posts: 4
Joined: Fri Sep 06, 2019 4:43 am

Re: I looking for bottleneck 10Gbe

Post by Ivqo77 » Tue Sep 10, 2019 3:50 am

Set jumbo to 9000 reading from NAS increase to 460 MB/s write not change, some as previous.
How get 1GB/s to PC? :roll: :lol: :roll:

User avatar
Toxic17
Ask me anything
Posts: 5158
Joined: Tue Jan 25, 2011 11:41 pm
Location: Planet Earth
Contact:

Re: I looking for bottleneck 10Gbe

Post by Toxic17 » Tue Sep 10, 2019 4:16 am

when setting MTU it needs to be 9000 on NAS, Switch and PC.

make sure any 10GbE cables are CAT6a or higher.
Regards Simon

QTS 4.x User Guidex

QNAP Club Repository
Submit a ticket • QNAP Helpdesk
QNAP Tutorials, User Manuals, FAQs, Downloads, Wiki
When you ask a question, please include the following


NAS: TS-473-32GB QM2-2P QXG-10G1T 4.4.1.1064 • TVS-463-16GB 4.4.1.1064 QM2-2S10G1TB • TS-459 Pro 2GB 4.2.6 • TS-121 4.3.3.0998 • APC Back-UPS ES 700G •
QPKG's: TwonkyServer 8.51 • Apache73 • QSonarr 3.0.1.503 • QNBZGet 21.0 • phpMyAdmin 4.9.0.1 • Qmono 5.20.1.19 • McAfee 2.2.1 • Lychee 3.2.15 • HBS 3.0.190802 • LEgo v3.0.0
Network: VM Hub 3.0 <500/35> • UniFi USG Pro 4 • UniFi USW-16-150W • UniFi USW-8-60W • UniFi CloudKey Gen2+• UniFi G3-Flex • UAP AC Pro • UAP AC Lite • SLM2008 • Dell 7050 MFF •

Ivqo77
New here
Posts: 4
Joined: Fri Sep 06, 2019 4:43 am

Re: I looking for bottleneck 10Gbe

Post by Ivqo77 » Tue Sep 10, 2019 11:58 am

switch unmanaged jumbo 9K support,pc and qnap set to 9000,qnap to switch sfp+ fibre,pc to switch cat7 cable. any next idea

User avatar
Trexx
Experience counts
Posts: 4907
Joined: Sat Oct 01, 2011 7:50 am
Location: Minnesota
Contact:

Re: I looking for bottleneck 10Gbe

Post by Trexx » Wed Sep 11, 2019 3:17 am

Not all SSD's are equal in performance so brand/model/etc. matters. Also difference SSD's will drop in performance faster/slower depending on when you overrun the cache buffer in them (hence SSD over-provisioning). Also make sure you have SMB set to 3.x on QNAP.

CPU also matters, I know Bob Zelin our resident video editing guru doen't recommend ANY Arm based models for 10GbE usage (other than backup) as he wasn't able to get full 10GbE throughput out of them even with much bigger models.
Paul

Model: TS-877-1600 FW: 4.4.1.x
QTier (HDD): [RAID-5] 6 x 3TB HGST DeskStar NAS QTier (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
(SSD): [RAID-1] 2 x 500GB Evo 860
RAM: Kingston HyperX Fury 64GB DDR4-2666
GPU: EVGA GTX 1060, ACX 2.0(1 Fan), 6GB
UPS: CyberPower AVR1350 Ext. Backup: USB 3.0 Seagate 5TB
Media Boxes: Nvidia ShieldTV Pro, AppleTV 4k, Various Roku's

Model: TVS-673 32GB FW: 4.4.1.x Test/Backup Box
-----------------------------------------------------------------------------------------------------------------------------------------
NAS RAID Rebuild Times | Live QTS Videos | | QNAP NAS Guide | Information needed when you ask for HELP | QNAP Links, Tutorials, etc.
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq

Bob Zelin
Been there, done that
Posts: 590
Joined: Mon Nov 21, 2016 12:55 am
Location: Orlando, FL.
Contact:

Re: I looking for bottleneck 10Gbe

Post by Bob Zelin » Wed Sep 11, 2019 4:10 am

Ivqo77 writes
"Set jumbo to 9000 reading from NAS increase to 460 MB/s write not change, some as previous.
How get 1GB/s to PC?"

REPLY - you should be jumping up and down with happiness with these results. Stop looking at the specs on the QNAP website. I only do 10G systems. With a TS-453BT3 setup perfectly, if you get 450 MB/sec you are doing great. You want to do 1000 MB/sec - well, first, you are not going to do 1000 MB/sec on a Mac (unless you get lucky). If you get 800 - 900 MB/sec with an EIGHT BAY QNAP with an Intel or AMD Ryzen processor, you are lucky. Even with a 16 bay TS-1683XU-RP top of the line QNAP, with 16 drives, and a 10G connection, if you use a Win 10 PC with the QNAP QXG-10G1T NIC card, THEN you will get 1000 MB/sec (1Gb/sec). It's not going to happen with a 4 bay, and a cheap 4 bay at that.

And lets go one step further than the TS-453BT3 - what if you buy the best 4 bay that QNAP is making - the TVS-482XT. NOPE - only 4 drives.

So now you will say "but, but, but - it says it can do it on the QNAP website". Welcome to the real world. Synology and Netgear say a lot of things too.

And to end this (for those that do research) - how do you achieve 1250 MB/sec on a 10G line (SFP+ or 10Gbase-T) - simply answer - you CANT DO IT, and you can't exceed these speeds. 1250 MB/sec is the THEORETICAL bandwidth, and in REAL LIFE, you don't get these speeds - just like you can't get 125 MB/sec on a 1G ethernet line - if you get 110 MB/sec its a miracle - in real life, it's more like 90 - 100 MB/sec. Everyone keeps looking at these specs on the company websites, and expects a miracle. Well, you are not getting it. No matter what you do.

When the world is all 40G with all NVMe's, then things will be different.

Bob Zelin
Bob Zelin / Rescue 1, Inc.
http://www.bobzelin.com

Post Reply

Return to “File Sharing”