TVS-h1288x / TVS-h1688x

[QNAPDanielFL]

What I know of deduplication is that for all deduplicated data, there should be metadata stored in the RAM. Metadata that can be used to Un-deduplicate, so that you don't need to wait to retrieve the metadata from the drives whenever you want to read a deduplicated file. So I would expect this metadata to always take RAM space at all times. I don't know why the RAM usage increased during the Night.

This can't be accurate. RAM gets erased if the machine is powered off or loses power. There is a directory entry that says what block a file starts at. All that happens with deduplication is that two different directories reference the same starting block. That information is saved with other data on the nonvolatile storage media.

When you power down the NAS, the RAM does lose it's data since it is volatile memory. But when you power back up, the metadata for deduplication is written back to the RAM and takes up space in the RAM. It is important that the metadata for deduplicated data be in the RAM. Otherwise, if you try to open a file that is deduplicated, it will take longer to open because it would first have to fetch metadata from the drives and only then open the file. It is my understanding that to avoid terrible performance opening data that is deduplicated, that metadata is written to your RAM when you turn on your NAS.

[SR22pilot]

I like answering all questions relating to 10G, 40G and video editing - and now especially with the new QuTS products, that I am installing all the time - but BOY, am I happy I did not get involved with this crazy thread.
When you purchase a QNAP 10G card, they give you ALL the brackets for every model. One bracket comes installed, and there are two more available that you can adapt. IF you lose these brackets, well - you are screwed. If you buy it used from someone, well - you are screwed. If you throw out your stuff - and then try to move it to a different model (I just moved a QNAP LAN-10G2T-X550 card from a TVS-EC1680U to a QNAP TS-1677XU-RP) - well - I kept those brackets, because I knew that the client would never know where he put that stuff (in a safe place). And as you (the readers of this thread) are saying - "that is terrible" - I see posts on all the Netgear, and Ubiquiti UniFi and Synology Forums, about "where can I get the rack mounting brackets - where can I get those little screws", etc.

There is NO reason for QNAP or anyone else to include the LP brackets with the NAS for countless cards. They COME with the cards, and if the card does not come with the brackets, then send the card back to the dealer, and say "this is not a new product -this is a re-pack". I have purchased plenty of things (from places like B&H Photo) that I can see are repacks - with no cable ties on the power cords, the plastic bags removed, and loose pieces floating around inside the box. I know that this was a "return" and the dealer tried to resell it.

Bob Zelin

The card is a standard part of the product and not an optional purchase. It was placed in the x8 PCI slot by Qnap to keep the two x4 slots free for Thunderbolt cards. However, I have no desire to put a Thunderbolt card in. I do have an nVidea graphics card that will give me 4K 60 Hz video. It will require the x8 PCI slot. Qnap admits the 10 GbE card will work fine in the x4 slot but will need a new bracket. Personally, I think it should have come with the unit to make moving the card easy. It's not a huge deal but this unit will have the fastest processor of any computer I own. Qnap has designed it to take a reasonably power graphics card (550 watt PS, 8 pin and 6 pin PSU cables). I plan on updating to a 10 GbE network as soon as I can find someone to pull the wiring. I would rather not do that myself. Hence, I don't want to have to buy another 10 GbE card just to get a different bracket.

[SR22pilot]

When you power down the NAS, the RAM does lose it's data since it is volatile memory. But when you power back up, the metadata for deduplication is written back to the RAM and takes up space in the RAM. It is important that the metadata for deduplicated data be in the RAM. Otherwise, if you try to open a file that is deduplicated, it will take longer to open because it would first have to fetch metadata from the drives and only then open the file. It is my understanding that to avoid terrible performance opening data that is deduplicated, that metadata is written to your RAM when you turn on your NAS.

Any idea why this is done this way? It makes no sense to me. When you reference a regular file, the system reads the directory and is pointed to the first block of data. In a regular system where there are two copies of a file (C1 and C2) there are two directory entries with Entry1 --> C1 and Entry2 --> C2. It seems to me you just change Entry2 so that Entry1 --> C1 and Entry2 --> C1 since C1 and C2 are exactly the same. What am I missing?

[matthewoliver]

I like answering all questions relating to 10G, 40G and video editing - and now especially with the new QuTS products, that I am installing all the time - but BOY, am I happy I did not get involved with this crazy thread.
When you purchase a QNAP 10G card, they give you ALL the brackets for every model. One bracket comes installed, and there are two more available that you can adapt. IF you lose these brackets, well - you are screwed. If you buy it used from someone, well - you are screwed. If you throw out your stuff - and then try to move it to a different model (I just moved a QNAP LAN-10G2T-X550 card from a TVS-EC1680U to a QNAP TS-1677XU-RP) - well - I kept those brackets, because I knew that the client would never know where he put that stuff (in a safe place). And as you (the readers of this thread) are saying - "that is terrible" - I see posts on all the Netgear, and Ubiquiti UniFi and Synology Forums, about "where can I get the rack mounting brackets - where can I get those little screws", etc.

There is NO reason for QNAP or anyone else to include the LP brackets with the NAS for countless cards. They COME with the cards, and if the card does not come with the brackets, then send the card back to the dealer, and say "this is not a new product -this is a re-pack". I have purchased plenty of things (from places like B&H Photo) that I can see are repacks - with no cable ties on the power cords, the plastic bags removed, and loose pieces floating around inside the box. I know that this was a "return" and the dealer tried to resell it.

Bob Zelin

Well, it seems to me you ARE pretty well involved in this thread, seeing your posts... I'm not really sure what's your point regarding the "repacks" though. What are you trying to say?
Anyways, Qnap advertises the TVS-h1288X as ready for graphics cards. But since there is no lp bracket wether included or available to purchase separately. What should I do ? Ditch the 10GbE card?

[QNAPDanielFL]

When you power down the NAS, the RAM does lose it's data since it is volatile memory. But when you power back up, the metadata for deduplication is written back to the RAM and takes up space in the RAM. It is important that the metadata for deduplicated data be in the RAM. Otherwise, if you try to open a file that is deduplicated, it will take longer to open because it would first have to fetch metadata from the drives and only then open the file. It is my understanding that to avoid terrible performance opening data that is deduplicated, that metadata is written to your RAM when you turn on your NAS.

Any idea why this is done this way? It makes no sense to me. When you reference a regular file, the system reads the directory and is pointed to the first block of data. In a regular system where there are two copies of a file (C1 and C2) there are two directory entries with Entry1 --> C1 and Entry2 --> C2. It seems to me you just change Entry2 so that Entry1 --> C1 and Entry2 --> C1 since C1 and C2 are exactly the same. What am I missing?

It is more complicated than what I said. It and I will try to get more information on this. I have some more things to learn about how dedup works as well. But here is one thing to consider. "ZFS dedupe will discard any data block that is identical to an already written block, while keeping a reference so that it can always reproduce the same block when read". But if you write a block of data, how will it know if the block is identical to another block of data? Will it scan every block in your folder every time you write a block so see if your new block is identical to any other block? That would make the write performance terrible. Instead, it stores the checksum of all the blocks in your RAM so it can right away know if the checksum of the new block is identical to the checksum of any other block.

If RAM is not enough, I think there are some limits on how much RAM ZFS will let dedup take because other things need RAM too. But imagine if dedup was not able to take all the RAM it needed. Imagine if it could not store the checksums of all the blocks in the RAM and the result was that every time you write, you have to fetch the checksums of all blocks to compare it to the checksum of the new block.

Someone who knows more than me can likely explain this better. But what I want to get across is that if you don't have enough RAM, dedup will either take up too much RAM, or fail to have all the RAM it needs to not greatly slow down your system. I would say that compression can be for everyone. Dedup is for specific situations and only if you have enough RAM.

[dawsonkm]

Well, it seems to me you ARE pretty well involved in this thread, seeing your posts... I'm not really sure what's your point regarding the "repacks" though. What are you trying to say?
Anyways, Qnap advertises the TVS-h1288X as ready for graphics cards. But since there is no lp bracket wether included or available to purchase separately. What should I do ? Ditch the 10GbE card?

Matt. I think Bob just meant not to expect to have the brackets, cables, etc. with the Qnap purchase.
From my experience the hole alignments are the same on the cards that use these LP and HP brackets. The hard part is finding the correct cuts on the bracket you need.
So you have to try and find a LP bracket for the vid card. You can buy generic brackets but modifying them for the cutout is the fun part.

Hope this help you

Dawson

[matthewoliver]

Well, it seems to me you ARE pretty well involved in this thread, seeing your posts... I'm not really sure what's your point regarding the "repacks" though. What are you trying to say?
Anyways, Qnap advertises the TVS-h1288X as ready for graphics cards. But since there is no lp bracket wether included or available to purchase separately. What should I do ? Ditch the 10GbE card?

Matt. I think Bob just meant not to expect to have the brackets, cables, etc. with the Qnap purchase.
From my experience the hole alignments are the same on the cards that use these LP and HP brackets. The hard part is finding the correct cuts on the bracket you need.
So you have to try and find a LP bracket for the vid card. You can buy generic brackets but modifying them for the cutout is the fun part.

Hope this help you

Dawson

I know, I guess I just didn't like the tone. I was thinking of doing something even more simpler. Take the existing PCiE lp bracket and cut it out for the 10GBe connections

[dawsonkm]

I know, I guess I just didn't like the tone. I was thinking of doing something even more simpler. Take the existing PCiE lp bracket and cut it out for the 10GBe connections

I would totally do that if you can. Blank brackets are easy to purchase if you need one later

[QNAPDanielFL]

I had previously said that to reclaim the RAM deduplication took you need to delete the deduplicated folder. Just turning off deduplication won't reclaim the RAM deduplication has already taken. It will just stop it from taking even more RAM.
I now see here has an option in Storage&Snapshots, Storage/Snapshots. Then click on the storage pool and click manage. Then under action, there is the option to "Cleen Dedup Table".

That should reclaim the RAM by un-deduplicating. So if someone uses Dedup and they are not happy with the performance, that is how you can get the performance back.

[SR22pilot]

Well, my TVS-h1688x should be here today. I seem to have sneaked in under the price increase. However, as I have learned more, I have some questions.

1) What is the best way to set up the 2.5" SSD array? I have 4X 2TB SATA SSDs. I am thinking RAID10. I would have thought 2 VDEVs, each RAID1, together in a pool. That way I could upgrade 2 at a time if I ever wanted to. However, it seems Qnap's ZFS is different. Certainly they don't use the term VDEV and the manual seems to say I would wind up with RAID 10 anyway. This will be the initial pool together with running virtual machines.

2) I was originally planning RAID 10 for the 3.5" drives. However, I saw some tests on this forum with RAID 6 being as fast in large file use cases. Again, on regular ZFS, I could add drives in groups of 2 as mirror VDEVs allowing easy upgradability either by adding 2 drives at a time or by upgrading current drives 2 at a time. Since I have 7 drives, one would be a hot spare. However, apparently Qnap (am I wrong?) would merge the mirror VDEVs into one RAID 10 requiring all drives be changed to upgrade. I could still add in groups of 2 until hitting 12 total. Now I am leaning RAID 6 since you get so much more capacity from 7 drives. The problem is upgradability. I won't be able to add drives. All I can do is add another RAID 6 array to the pool. Is that correct? If I understand correctly (big if) that would be slower than one large RAID 6 array. This will be document, video and photo storage.

[casw1000]

Must say, as much as I was hoping for a nice multi core Ryzen 3 generation cpu (tower / desktop model), I have given up that hope after waiting over a year. So it seems the 'best' we can get for home use, is the Xeon 6 core cpu. Now that I figure that's the best option out there, I will simply wait for the crazy high prices to return back to what they was on release, then buy a TVS-h1288x. My current 4 bay TS-453 just cannot handle the CCTV, Plex and other apps that I am running. So I will leave the TS-453b for cctv, then build a nice nas for my home labs, vmware, iscsi, plex, network backups etc. Just sharing my thoughts.

Oh and I know its been mentioned, but I would also want to add a GPU into this nas, therefore having the correct bracket to move the 10GbE card into the x4 slot is important to me as well.

[SR22pilot]

I'm not sure what you are really looking for but I don't see any desktop Synology that is as fast. Qnap sells a very fast tower if money is no object. If you build your own then you can get faster for the same amount of money. You can also get faster in a Qnap or Synology rack mount but again it will take a lot more money. Qnap has a less expensive Ryzen based desktop machine (TS-877) but it is slower. Then there are the TS-1277 and TS-1677. They are Ryzen 7 based but older designs. There must be a reason Qnap went with the Xeon. for the TVS-h1x88X series. If you look at the NAScompares Plex tests the new Qnap units seem top handle whatever you throw at them.

[casw1000]

I'm not sure what you are really looking for but I don't see any desktop Synology that is as fast. Qnap sells a very fast tower if money is no object. If you build your own then you can get faster for the same amount of money. You can also get faster in a Qnap or Synology rack mount but again it will take a lot more money. Qnap has a less expensive Ryzen based desktop machine (TS-877) but it is slower. Then there are the TS-1277 and TS-1677. They are Ryzen 7 based but older designs. There must be a reason Qnap went with the Xeon. for the TVS-h1x88X series. If you look at the NAScompares Plex tests the new Qnap units seem top handle whatever you throw at them.

As i mentioned, I was hoping for some time now, that Qnap would release updated versions of the (no longer available) Ryzen models. A number of members here was hoping that they would upgrade the first generation Ryzen tower's, whilst that did happen, it was really a full product update, more of a silent update on some models. I was looking for the 1277/1677 model to have a second gen or better still 3rd gen Ryzen. But here, IMO, is why it didn't happen. The Ryzen processors are a better processor (in many ways) than the Intel products, but, I think for Qnap, the margins and value in selling Ryzen is much less than what Intel makes them. Yes there was and are some Ryzen processors, but these are mainly rack models, again, this is about who they are selling them to. Also, another point, if you compare these Ryzen's to their Intel competition, the Ryzen's walk all over them, I think this is another reason why Qnap is not pushing these AMD's. The embedded ryzen's are not the same, again, in my view.

The Xeon supports the ECC ram and decent ish amount of ram. I have watched most of Rob's videos especially about those 12/16 bay devices. But the recent price increase of many hundred of pounds is making the purchase difficult. When Rob did those videos of the 12/16 bay, the prices were around £2K. Now your looking at £2600 (cheapest 1288x model) or £3180 for the 1688x.

For business, this can be acceptable, but for the home Plex and techie, thats a lot to run past the board of directors (The Wife). I wish, I got in quicker.

[SR22pilot]

My main wish is that the W-1250 supported 4K 60 Hz over HDMI or that Qnap had put in a Displayport connection which would support 4K at 60 Hz. Still I think the W-1250 choice aids transcoding although I haven't tried it. I am still copying files from one Synology. It is taking days since the Synology is so slow. I do find Qnap software more confusing and less well integrated than Synology. Right now I am dealing with mail authorization errors yet test emails are just fine. I also got told the NAS name was already registered which is strange since it is fairly unique. I had to add a number at the end of the name. Tech support has been unresponsive. I submitted a ticket with a simple question. A lot of references are made to Photo Station but I only see Music Station and Video Station. No answer from Qnap. On the performance side, the system seems fast. The most I have pushed the CPU is 10% and that was briefly with three streams running. It quickly dropped back to 3%. It is noisy but that I expected considering the large number of Exos drives. I had actually expected it to be noisier. It is still quieter than rack mounts I have dealt with in the past. The noise is almost entirely drive noise and not fan noise. An error message on boot was poor. The system said it was checking memory but was hung there. It should have said there was a memory error. I powered down, opened the case and reseated a memory stick and that fixed it. I had removed the stick to photograph the label to document exactly the memory in the machine. I thought I had reseated it properly but obviously not. I need to update my network to 10GbE. A 1 GbE network doesn't push the device at all.

[casw1000]

I think you will certainly have a good system and one of the reasons I have not built a home brew NAS is that I really like the qnap software. I used to sell synology systems but stopped with their products and moved to selling Qnap. I have opened a number of support tickets with Qnap and they normally respond within 24/48 hours, if I cannot find the answers myself. So, I personally believe qnap is the market leader with its products and operating system package. I hope you have fund building and using your system, having used both platforms quite a bit, I am sure you will find peace with the same but slightly different interface that qnap offers. As for the Hero side of things, I just dont know if its for me or not. I hope that I can jump on this product soon. Have fun.