TVS-h1288x / TVS-h1688x

Interested in our products? Post your questions here. Let us answer before you buy.
Post Reply
ashhz
New here
Posts: 2
Joined: Sun Jun 26, 2022 6:47 am

Re: TVS-h1288x / TVS-h1688x

Post by ashhz »

Hi all,

I'm testing the new tvs-h1688X with 25gbe card QXG-25G2SF-CX6 with 11x18to/raid6 and 6xssd 870evo/raid5 and 2xnvme 2to 970evo for cache and result is not good only 1,4go/s write and 1,6go/s read. done the same result with thubderbolt 3 connexion other a mac.
i made the same test with tvs1282t with aprox the same disk configuration, it's an old generation of qnap nas but i get 1,4 go/s write and 2,5go/s read speed

its like there is a bottleneck on the serveur with sata/nvme transmission or pcie lan gen?

is there some else here that get the same bad result?

thanks
nibbles2bits
Getting the hang of things
Posts: 50
Joined: Fri Aug 23, 2019 10:17 am

Re: TVS-h1288x / TVS-h1688x

Post by nibbles2bits »

Great question. Sorry I edited it to be more clear about lane usage.

Let's try to figure out whether you have enough PCIe lanes to run everything.

The TVS-1688x web page says the NAS has three PCIe exapansion slots on the back
that are laid out the usual way, 1x8, 2x4, 2x4.

The Xeon W-1250 CPU in that NAS has 16 PCIe lanes directly to the CPU.

The QNAP website says the dual 10 GbE card takes up the top PCIe Gen 3x8 slot.
The website also says the 25 GbE card QXG-25G2SF-CX6 uses a PCI Gen 4x8 connection.
It's my understanding that NVMe M.2 PCIe usage is Gen 3x4.
Thunderbolt 3 does 40 Gbps using Gen 3x4 lanes. Only have 2 lanes? Your max is 20 Gbps.

So given that a 3x8 is used by the 10 GbE,
given that a 3x4 is used by Thunderbolt
given that a 3x4 is used by NVMe

I think you had no lanes left for the 25 GbE card that wanted 8 lanes.
If your layout matches what I guessed, the 25 GbE card
had to take lanes from the Tb3 and the NVMe.

When consider things like this, everyone says to look at your user manual for the block diagram
showing which PCIe slots and devices go directly to the CPU's 16 available lanes as versus going
to the chipset.

I have no idea whether the Tb3 and the NVMe use the CPU's PCIe lanes or some of the 4 lanes saved
for the chipset, DMI, and SSDs. But what a great question.

You might check your Jumbo Frames, test the network with iPerf3,
and test the disks with `dd` or Black Magic Speed Test.
Last edited by nibbles2bits on Sun Jun 26, 2022 11:56 am, edited 2 times in total.
ashhz
New here
Posts: 2
Joined: Sun Jun 26, 2022 6:47 am

Re: TVS-h1288x / TVS-h1688x

Post by ashhz »

<<<<The tvs-1688x web page says the NAS has a PCIe Gen 3x8 slot that's populated with a dual 10 GbE card.
It also says the other two PCIe slots are Gen 3x4.>>>>

<<<The 25 GbE card QXG-25G2SF-CX6 web page says it is made for PCI Gen 4x8.>>>>

My workstation got a QXG-25G2SF-CX6 on a pcie gen3 x4 and i ve got the full bandwidth with the tvs1282t with blackmagic speedtest 2.5go/s readspeed

<<<<So is the dual 10 GbE card removed? >>>>

the 10gbe card from the pcie gen3 x8 was remove and replaced with the QXG-25G2SF-CX6.
the switch is a QSW-M5216-1T server and switch configured in lacp 50gbe

<<<<Have you tested your network with an iPerf3 server and multiple iPerf3 clients?>>>
<<<<Have you tested just the disk speed with Blackmagic Speed Test or `dd`?>>>>

i ve tested with multiple client and got the same result not with iperf3 but with multiple blackmagic speedtest. the serveur only get 1,5go/s readspeed
the QXG-25G2SF-CX6 is configured in lacp serveur and switch.

the only thing i m thinking about is perhaps all the pcie expension card share the same bandwidth. perhaps the 25 gbe card, the 2 nvme for cache and the thunderbolt pcie card share the same pcie lanes and bottleneck the server speed?
nibbles2bits
Getting the hang of things
Posts: 50
Joined: Fri Aug 23, 2019 10:17 am

Re: TVS-h1288x / TVS-h1688x

Post by nibbles2bits »

yes a bit too much sharing lanes I guessed, but I was editing my post when you replied.

If you took out the 10G card, in theory you're okay with lanes.
(We still would need a block diagram to know how QNAP used the 16 lanes to be sure.)

iPerf3 only tests your network, which I would also do, and use ping to verify my jumbo frame size
isn't too high given my LAN packet overhead (think vlans etc). This is testing your network only,
not using drives. Lets verify one high speed element at a time.

The drive speed tests I'm talking about running them from an App on the server. No clients.
This would let us know how your drives are performing. Or you can use `dd` for that like I did
in the 1900s.

I would also check that my RAM is running as expected, as it's important
to have identical chips in dual channel and verify that they are being recognized
at the rated speed.
User avatar
marcoi
Starting out
Posts: 11
Joined: Fri May 21, 2021 3:29 am

Re: TVS-h1288x / TVS-h1688x

Post by marcoi »

FYI - h5.0.0.2069 build 20220614 been released. I am in the process of trying it now. Hopefully it isnt as bad as the prior version release. If it is ill have to go back two version again. The biggest reason to apply is the security fixes
- Fixed the security issue CVE-2022-0778. For details, see https://www.qnap.com/en/security-advisory/qsa-22-11
- Fixed the security issue CVE-2019-11043. For details, see https://www.qnap.com/en/security-advisory/qsa-22-20

Edited - if you run HDP there seems to be an issue with app and newer FW past .1986 that will cause the NAS to crash. I have tickets opened for the issue.
Edit 2 - well the system became unresponsive after a day of being up. I cant log into the NAS, either via webpage, qfinder, cell phone app or ssh. network storage seems to still be accessible but slow. This version 5.x Firmware has been horrible. I cannot even keep the unit updated and online running basic nas features.
Last edited by marcoi on Fri Jul 01, 2022 12:05 pm, edited 2 times in total.
Model name TVS-h1688X
CPU Intel(R) Xeon(R) W-1250 CPU @ 3.30GHz (6 cores, 12 threads)
Total memory 128 GB (126 GB usable)
QNAPDanielFL
Easy as a breeze
Posts: 488
Joined: Fri Mar 31, 2017 7:09 am

Re: TVS-h1288x / TVS-h1688x

Post by QNAPDanielFL »

ashhz wrote: Sun Jun 26, 2022 11:20 am
i ve tested with multiple client and got the same result not with iperf3 but with multiple blackmagic speedtest. the serveur only get 1,5go/s readspeed
the QXG-25G2SF-CX6 is configured in lacp serveur and switch.

the only thing i m thinking about is perhaps all the pcie expension card share the same bandwidth. perhaps the 25 gbe card, the 2 nvme for cache and the thunderbolt pcie card share the same pcie lanes and bottleneck the server speed?
Are you running QuTS Hero?
Do you have an SSD system pool?
If you use L2ARC cache, is it set to Random Only? Have you tried the speed with cache turned off?
What is your block size on the folder you used for the speed test?
cacakaka
New here
Posts: 8
Joined: Tue Aug 17, 2021 8:53 pm

Re: TVS-h1288x / TVS-h1688x

Post by cacakaka »

marcoi wrote: Mon Jun 13, 2022 9:56 am anyone try QuTS hero h5.0.1.2045 build 20220526 Public Beta yet? The last QuTS 5.0.0.x firmware put my unit into loop where it could not load correctly. when it did load up, it only lasted a few mins before it hung. I wish i never went to v5.x.x.x
I was updated to QuTS hero h5.0.1.2045 build 20220526。It has been running continuously for more than 50 days。Everything looks stable and functioning properly.
QuTS hero h5.0.1.2045 was added many new features。
User avatar
marcoi
Starting out
Posts: 11
Joined: Fri May 21, 2021 3:29 am

Re: TVS-h1288x / TVS-h1688x

Post by marcoi »

cacakaka wrote: Sun Aug 14, 2022 10:02 pm
marcoi wrote: Mon Jun 13, 2022 9:56 am anyone try QuTS hero h5.0.1.2045 build 20220526 Public Beta yet? The last QuTS 5.0.0.x firmware put my unit into loop where it could not load correctly. when it did load up, it only lasted a few mins before it hung. I wish i never went to v5.x.x.x
I was updated to QuTS hero h5.0.1.2045 build 20220526。It has been running continuously for more than 50 days。Everything looks stable and functioning properly.
QuTS hero h5.0.1.2045 was added many new features。
Good to know, maybe I'll try it at some point. Hopefully the non beta gets released soon.
Model name TVS-h1688X
CPU Intel(R) Xeon(R) W-1250 CPU @ 3.30GHz (6 cores, 12 threads)
Total memory 128 GB (126 GB usable)
Bob Zelin
Experience counts
Posts: 1374
Joined: Mon Nov 21, 2016 12:55 am
Location: Orlando, FL.
Contact:

Re: TVS-h1288x / TVS-h1688x

Post by Bob Zelin »

always fascinated by these posts. A 10G system - more than enough for video editing, over a 10G network, will give you 1000 MB/sec. So if you have a TVS-h1688X with all 12 drives in a single RAID group, you will get 1000 MB/sec from your client computer if you have a modern Mac or Win 10 PC with a 10G NIC card in a x4 lane slot.

For 7200 RPM SATA drives, if you have lots of multiple users, and want to get more people working on an ENTERPRISE box, like the TS-h1683XU-RP 16 bay or TS-h2483XU-RP, THEN you can add a 25G card into one of these systems, plug this into a 10G switch with 25G uplink ports (Ubiquiti, FS), and now have lots of 10G clients working at great speeds (but still not over 1000 MB/sec) for more than 4 - 5 client computers.

But now you say. - NO - I WANT 25G SPEEDS ON THE CLIENTS !. Well, this is not happening with 7200 RPM SATA drives. You now purchase a QNAP that supports all M.2 or all U.2 NVMe drives (the TS-h1290FX, or the TS-h2490FX), and load this thing up with all U.2 NVMe drives (expensive). Now, when you create your RAID group, and plug the 25G port into a 25G switch (like The QNAP QSW-M5216-1T), and plug this into a 25G card in your Win 10/11 computer (Mellanox CX6 SFP28 card), or Mac with an ATTO Thunderlink thunderbolt 3 to 25G SFP28 adapter - now you will get 2200 MB/sec.

How do I know this ? Because I have done it. So how do you get 2200 MB/sec on a TVS-h1688X ? You don't. (now, I have not tried this, but in theory, if you put in a QNAP QM2-4P-384 card, with four M.2 NVMe drives on this card, AND a 25G card into the TVS-h1688X - and now plug this into a SINGLE 25G client - then MAYBE you will get 2200 MB/sec).

These conversations all remind me of people that bought a 4 bay, like the TVS-472XT, or TS-453BT3, get crappy speeds, and say "well - what if I put some M.2 drives inside as a cache - will it now do 1000 MB/sec ?". You bought the wrong box.

Bob Zelin
Bob Zelin / Rescue 1, Inc.
http://www.bobzelin.com
User avatar
marcoi
Starting out
Posts: 11
Joined: Fri May 21, 2021 3:29 am

Re: TVS-h1288x / TVS-h1688x

Post by marcoi »

I run my two onboard 10G nics in LACP mode to get 20GB connection to the switch. I also added a Mellanox MT27500 Family [ConnectX-3] card running at 10GB just to handle ISCSI traffic. Each server also has 20GB LACP setup and 1 ConnectX3 card. I run most of my pools as raid10 setup with 4 drives and also have two m.2 drives setup for read and write cache. I havent benchmarked the system, but i often see 500-900 MB read/write speeds, which for me is good enough for my use case.
Model name TVS-h1688X
CPU Intel(R) Xeon(R) W-1250 CPU @ 3.30GHz (6 cores, 12 threads)
Total memory 128 GB (126 GB usable)
Bob Zelin
Experience counts
Posts: 1374
Joined: Mon Nov 21, 2016 12:55 am
Location: Orlando, FL.
Contact:

Re: TVS-h1288x / TVS-h1688x

Post by Bob Zelin »

if you did not use LACP, if you did not create a LAG, if you did not use iSCSI -
and simply connected your computer with 10G Ethernet from your computer to a TVS-h1688X directly to the 10G port, and enabled jumbo frames -
even with a cheap 10G card in your computer (TP Link, ASUS XG-C100C, etc.) you would be getting 1000 MB/sec to the QNAP with all 12 drives configured in a simple RAID 6 group.

Everyone tries to be SO smart, and so fancy. But you can be dumb (just like me), and do a nice simple connection over SMB - no iSCSI, no LAG, and get KILLER 1000 MB/sec (unless you are not in a x4 lane slot in your PC, and you don't have 12 drives in your TVS-h1688X).

Everyone is a big shot.

Bob Zelin
Bob Zelin / Rescue 1, Inc.
http://www.bobzelin.com
Post Reply

Return to “Presales”