QTS Hero... ZFS? What? When?

Interested in our products? Post your questions here. Let us answer before you buy.
occamsrazor
Know my way around
Posts: 217
Joined: Tue Mar 30, 2010 8:30 pm

Re: QTS Hero... ZFS? What? When?

Post by occamsrazor » Sat Nov 02, 2019 3:33 pm

@TRexx - Near the end of the video he says they are aiming for “the end of next month”.
At 39:00 he also refers to the need to purchase a license, but price is still TBD. I do hope it’s affordable...

Also interesting are the minimum compatibility standards which is basically 4GB RAM, while recommending a LOT more if using deduplication, etc etc.
I'm looking to upgrade my TS-451 to something like a TS-873 or TVS-872XT, but in the meantime I wonder if this will run on my old and lowly TS-451 with Celeron J1800 and 8GB RAM if I just want the core ZFS functionality such as self-healing, and I don't use deduplication etc.
TS-451 [4 x 10TB WD Reds in Raid-5]
TS-239 Pro II [2 x 3TB in Raid-0]
pfSense router and Ubiquiti Unifi switches
Mac Minis, MacBook Pro, iPhones

User avatar
Moogle Stiltzkin
Ask me anything
Posts: 7872
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: QTS Hero... ZFS? What? When?

Post by Moogle Stiltzkin » Sat Nov 02, 2019 5:44 pm

QNAPDanielFL wrote:
Sat Nov 02, 2019 3:47 am
If anyone wants to know more about QTS Hero, this might answer some of your questions.https://www.youtube.com/watch?v=LQtqahHh4d4

ty daniel, i only noticed this late becauz i was busy with lost ark russia.

anyway watched the video and a lot of important questions and answers.

it's interesting that though the meta data is saved in the ram, it uses some type of mechanism to ensure it doesn't get lost in the ram when the power is turned off?

i assume this protection also occurs in the event of a sudden power loss scenario? for users that don't have a UPS.

i was a bit dissapointed that nobody at that event brought up the question of ecc vs non ecc ram :S

anyway zfs on qts is definitely worth paying for. i just hope that the price is something reasonable.
NAS
[Main Server] QNAP TS-877 w. 4tb [ 3x HGST Deskstar NAS (HDN724040ALE640) & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 + 16gb ddr4 Crucial + QWA-AC2600 wireless adapter.
[Backup] QNAP TS-653A w. 5x 2TB Samsung F3 (HD203WI) EXT4 Raid5
[^] QNAP TS-659 Pro II 1x 4TB HGST Deskstar NAS
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-228 w. 1x 1TB WD RE3 (WD1002FBYS)
[^] QNAP TS-128
Mobile NAS TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Asus AC68U Router|100dl/50ul MBPS FTTH Internet | Windows 10, WC PC-Intel i7 920 Ivy bridge desktop (1x 512gb Samsung 850 Pro SSD + 1x 4tb HGST Ultrastar 7K4000)


Guides & articles
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review

https://www.patreon.com/mooglestiltzkin

occamsrazor
Know my way around
Posts: 217
Joined: Tue Mar 30, 2010 8:30 pm

Re: QTS Hero... ZFS? What? When?

Post by occamsrazor » Fri Nov 08, 2019 6:11 pm

TS-451 [4 x 10TB WD Reds in Raid-5]
TS-239 Pro II [2 x 3TB in Raid-0]
pfSense router and Ubiquiti Unifi switches
Mac Minis, MacBook Pro, iPhones

User avatar
Moogle Stiltzkin
Ask me anything
Posts: 7872
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: QTS Hero... ZFS? What? When?

Post by Moogle Stiltzkin » Fri Nov 08, 2019 7:39 pm

ZIL / L2ARC provide improved caching performance

QTS Hero lets you safely and reliably use Memory/RAM as large scale Cache. Using L2ARC in conjunction with SSDs means that you will have fantastically high throughput, as well as working will with hybrid storage where data is moved around constantly internally. Not a new ideal and good in the short term, but flushes/empties constantly with power on/off. ZIL works with the Memory Cache, logging the data that is held there.

This helps with maintain the data record there and preventing corruption with that log. QTS has one-off memory/ram caching, but it cannot hold it safely or has the log record to ensure it is complete. Additionally, there are some high-end enterprise QNAP NAS (ES series) available with QES that support battery backup units (BBU) for long term memory caching options, but QTS Hero and the ZFS system using ZIL should allow very similar levels of memory caching performance, or as close as to make little difference in the real world.

Image

in layman terms, is UPS a must if you wish to use features like deduplication that store the meta in the ram? and this new ssd caching feature? or are these features mechanisms durable enough to survive a sudden power loss without UPS? i'm confused :S
NAS
[Main Server] QNAP TS-877 w. 4tb [ 3x HGST Deskstar NAS (HDN724040ALE640) & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 + 16gb ddr4 Crucial + QWA-AC2600 wireless adapter.
[Backup] QNAP TS-653A w. 5x 2TB Samsung F3 (HD203WI) EXT4 Raid5
[^] QNAP TS-659 Pro II 1x 4TB HGST Deskstar NAS
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-228 w. 1x 1TB WD RE3 (WD1002FBYS)
[^] QNAP TS-128
Mobile NAS TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Asus AC68U Router|100dl/50ul MBPS FTTH Internet | Windows 10, WC PC-Intel i7 920 Ivy bridge desktop (1x 512gb Samsung 850 Pro SSD + 1x 4tb HGST Ultrastar 7K4000)


Guides & articles
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review

https://www.patreon.com/mooglestiltzkin

User avatar
storageman
Ask me anything
Posts: 5096
Joined: Thu Sep 22, 2011 10:57 pm

Re: QTS Hero... ZFS? What? When?

Post by storageman » Fri Nov 08, 2019 7:52 pm

If HBS is anything to go by, I'm waiting for QTS Hero to go horribly wrong, both in terms of customer performance expectations and reliability.

And why they are trying to get people off QES to QTS Hero? Is this admittance that their FreeBSD/ZFS implementation was a mistake?

ZFS need massive amounts of RAM/SSD cache to dedupe properly and without damaging performance.

Time will tell...

QNAPDanielFL
Know my way around
Posts: 208
Joined: Fri Mar 31, 2017 7:09 am

Re: QTS Hero... ZFS? What? When?

Post by QNAPDanielFL » Sat Nov 09, 2019 1:15 am

"is UPS a must if you wish to use features like deduplication that store the meta in the ram? and this new ssd caching feature? or are these features mechanisms durable enough to survive a sudden power loss without UPS?"

The metadata for deduplication is stored in the RAM for faster access. But it is also on the drives. So if the RAM lost power, you would not lose the metadata. You would just have to wait a small amount of time for the NAS to copy the metadata back to the RAM.

As for using the RAM as cache, normally if you lost the data that was still in the RAM cache, the NAS would just re-request the data. Much like if a packet is lost it can just be requested again. The Zill Logging logs what data is in the RAM cache so it knows what to request if the RAM loses power. Zil Logging makes it safer to use RAM as a cache, and I don't think the risk is very high.

Actually even with a windows, Linux, or Mac PC, when you transfer a file to the computer, it often first goes into the RAM to increase the speed. If the computer loses power, the file can just be resent if the transfer was not complete. So the risk might not be as great as you expect. And the Zil logging adds safety.

You also have the option of setting "Write Though" which will stop the NAS from using the RAM as a cache.
ZFS was built with safety in mind and I think overall, this is safer than QTS.

User avatar
storageman
Ask me anything
Posts: 5096
Joined: Thu Sep 22, 2011 10:57 pm

Re: QTS Hero... ZFS? What? When?

Post by storageman » Sat Nov 09, 2019 1:24 am

QNAPDanielFL wrote:
Sat Nov 09, 2019 1:15 am
"is UPS a must if you wish to use features like deduplication that store the meta in the ram? and this new ssd caching feature? or are these features mechanisms durable enough to survive a sudden power loss without UPS?"

The metadata for deduplication is stored in the RAM for faster access. But it is also on the drives. So if the RAM lost power, you would not lose the metadata. You would just have to wait a small amount of time for the NAS to copy the metadata back to the RAM.

As for using the RAM as cache, normally if you lost the data that was still in the RAM cache, the NAS would just re-request the data. Much like if a packet is lost it can just be requested again. The Zill Logging logs what data is in the RAM cache so it knows what to request if the RAM loses power. Zil Logging makes it safer to use RAM as a cache, and I don't think the risk is very high.

Actually even with a windows, Linux, or Mac PC, when you transfer a file to the computer, it often first goes into the RAM to increase the speed. If the computer loses power, the file can just be resent if the transfer was not complete. So the risk might not be as great as you expect. And the Zil logging adds safety.

You also have the option of setting "Write Though" which will stop the NAS from using the RAM as a cache.
ZFS was built with safety in mind and I think overall, this is safer than QTS.
I agree "Copy On Write" ZFS is safer than EXT4 writing but the COW feature is at the expense of some performance.
ZFS was designed for scaling large volumes safety rather than out and out performance.

ZFS or not, UPS is always good to have if you're serious about protecting your system (and data).

User avatar
Moogle Stiltzkin
Ask me anything
Posts: 7872
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: QTS Hero... ZFS? What? When?

Post by Moogle Stiltzkin » Sat Nov 09, 2019 4:12 am

QNAPDanielFL wrote:
Sat Nov 09, 2019 1:15 am
"is UPS a must if you wish to use features like deduplication that store the meta in the ram? and this new ssd caching feature? or are these features mechanisms durable enough to survive a sudden power loss without UPS?"
The metadata for deduplication is stored in the RAM for faster access. But it is also on the drives. So if the RAM lost power, you would not lose the metadata. You would just have to wait a small amount of time for the NAS to copy the metadata back to the RAM.

As for using the RAM as cache, normally if you lost the data that was still in the RAM cache, the NAS would just re-request the data. Much like if a packet is lost it can just be requested again. The Zill Logging logs what data is in the RAM cache so it knows what to request if the RAM loses power. Zil Logging makes it safer to use RAM as a cache, and I don't think the risk is very high.

Actually even with a windows, Linux, or Mac PC, when you transfer a file to the computer, it often first goes into the RAM to increase the speed. If the computer loses power, the file can just be resent if the transfer was not complete. So the risk might not be as great as you expect. And the Zil logging adds safety.

You also have the option of setting "Write Though" which will stop the NAS from using the RAM as a cache.
ZFS was built with safety in mind and I think overall, this is safer than QTS.
hi daniel,

ty, this is exactly what i wanted to know. so it does sound like i'm over worrying (because it's tech i'm unfamiliar with). i'll give it a try when it comes out, but from what was explained it does seem like something i can use without much worry :)
NAS
[Main Server] QNAP TS-877 w. 4tb [ 3x HGST Deskstar NAS (HDN724040ALE640) & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 + 16gb ddr4 Crucial + QWA-AC2600 wireless adapter.
[Backup] QNAP TS-653A w. 5x 2TB Samsung F3 (HD203WI) EXT4 Raid5
[^] QNAP TS-659 Pro II 1x 4TB HGST Deskstar NAS
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-228 w. 1x 1TB WD RE3 (WD1002FBYS)
[^] QNAP TS-128
Mobile NAS TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Asus AC68U Router|100dl/50ul MBPS FTTH Internet | Windows 10, WC PC-Intel i7 920 Ivy bridge desktop (1x 512gb Samsung 850 Pro SSD + 1x 4tb HGST Ultrastar 7K4000)


Guides & articles
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review

https://www.patreon.com/mooglestiltzkin

dolbyman
Guru
Posts: 15245
Joined: Sat Feb 12, 2011 2:11 am
Location: Vancouver BC , Canada

Re: QTS Hero... ZFS? What? When?

Post by dolbyman » Sat Nov 09, 2019 4:18 am

QNAPDanielFL wrote:
Sat Nov 09, 2019 1:15 am
ZFS was built with safety in mind and I think overall, this is safer than QTS.
I know I know I am splitting hairs here but shouldn't it be:

ZFS was built with safety in mind and I think overall, this is safer than EXT.
or
QTS Hero was built with safety in mind and I think overall, this is safer than QTS.

:mrgreen:

User avatar
Moogle Stiltzkin
Ask me anything
Posts: 7872
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: QTS Hero... ZFS? What? When?

Post by Moogle Stiltzkin » Sat Nov 09, 2019 2:02 pm

doesn't seem to mention much new, but just another video explaining about the qts hero coming soon
https://www.youtube.com/watch?v=LvarvZAf2t8
NAS
[Main Server] QNAP TS-877 w. 4tb [ 3x HGST Deskstar NAS (HDN724040ALE640) & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 + 16gb ddr4 Crucial + QWA-AC2600 wireless adapter.
[Backup] QNAP TS-653A w. 5x 2TB Samsung F3 (HD203WI) EXT4 Raid5
[^] QNAP TS-659 Pro II 1x 4TB HGST Deskstar NAS
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-228 w. 1x 1TB WD RE3 (WD1002FBYS)
[^] QNAP TS-128
Mobile NAS TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Asus AC68U Router|100dl/50ul MBPS FTTH Internet | Windows 10, WC PC-Intel i7 920 Ivy bridge desktop (1x 512gb Samsung 850 Pro SSD + 1x 4tb HGST Ultrastar 7K4000)


Guides & articles
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review

https://www.patreon.com/mooglestiltzkin

occamsrazor
Know my way around
Posts: 217
Joined: Tue Mar 30, 2010 8:30 pm

Re: QTS Hero... ZFS? What? When?

Post by occamsrazor » Sun Nov 10, 2019 4:13 pm

Seeing some conflicting information on likely supported models between the info on NAS Compares and QNAP's own info....
@QNAPDanielFL - Do you think QTS Hero will be able to run on a TS-451 (Celeron J1800 @ 2.41Ghz) with 4x10TB that has 8GB RAM (without any deduplication or compression etc)
TS-451 [4 x 10TB WD Reds in Raid-5]
TS-239 Pro II [2 x 3TB in Raid-0]
pfSense router and Ubiquiti Unifi switches
Mac Minis, MacBook Pro, iPhones

User avatar
Trexx
Experience counts
Posts: 4975
Joined: Sat Oct 01, 2011 7:50 am
Location: Minnesota
Contact:

Re: QTS Hero... ZFS? What? When?

Post by Trexx » Mon Nov 11, 2019 4:57 am

I would say it MAY run, but I would guess NOT WELL. It likely will fall into the could be SHOULD you issue.

The j1x00 series celeron processors are no where fast enough nor do they have all the embedded hw accelerating features that will mitigate the compression and deduplication impacts.

Stick with regular QTS unless you are willing to sacrifice performance and higher cpu loads.


Sent from my iPhone using Tapatalk
Paul

Model: TS-877-1600 FW: 4.4.1.x
QTier (HDD): [RAID-5] 6 x 3TB HGST DeskStar NAS QTier (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
(SSD): [RAID-1] 2 x 500GB Evo 860
RAM: Kingston HyperX Fury 64GB DDR4-2666
GPU: EVGA GTX 1060, ACX 2.0(1 Fan), 6GB
UPS: CyberPower AVR1350 Ext. Backup: USB 3.0 Seagate 5TB
Media Boxes: Nvidia ShieldTV Pro, AppleTV 4k, Various Roku's

Model: TVS-673 32GB FW: 4.4.1.x Test/Backup Box
-----------------------------------------------------------------------------------------------------------------------------------------
NAS RAID Rebuild Times | Live QTS Videos | | QNAP NAS Guide | Information needed when you ask for HELP | QNAP Links, Tutorials, etc.
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq

occamsrazor
Know my way around
Posts: 217
Joined: Tue Mar 30, 2010 8:30 pm

Re: QTS Hero... ZFS? What? When?

Post by occamsrazor » Mon Nov 11, 2019 1:22 pm

Trexx wrote:
Mon Nov 11, 2019 4:57 am
I would say it MAY run, but I would guess NOT WELL. It likely will fall into the could be SHOULD you issue.
Thanks - that was my guess also, but you never know.
Trexx wrote:
Mon Nov 11, 2019 4:57 am
The j1x00 series celeron processors are no where fast enough nor do they have all the embedded hw accelerating features that will mitigate the compression and deduplication impacts.
For sure I wouldn't be CPU-intensive functions such as deduplication or compression, so the question is without those turned on... is QTS hero/ZFS still much more CPU-intensive than QTS/Ext4? My unit does have 8GB RAM which should be sufficient. I'm guessing the checksumming still uses a fair amount of CPU cycles but I don't know.

In any case I'm looking to upgrade the unit to something more modern in the short/medium term future, like a TS-873 or TVS-872N/XT but was curious anyway as I find the move to ZFS really exciting....
TS-451 [4 x 10TB WD Reds in Raid-5]
TS-239 Pro II [2 x 3TB in Raid-0]
pfSense router and Ubiquiti Unifi switches
Mac Minis, MacBook Pro, iPhones

QNAPDanielFL
Know my way around
Posts: 208
Joined: Fri Mar 31, 2017 7:09 am

Re: QTS Hero... ZFS? What? When?

Post by QNAPDanielFL » Tue Nov 12, 2019 4:45 am

occamsrazor wrote:
Sun Nov 10, 2019 4:13 pm
@QNAPDanielFL - Do you think QTS Hero will be able to run on a TS-451 (Celeron J1800 @ 2.41Ghz) with 4x10TB that has 8GB RAM (without any deduplication or compression etc)

We should know for sure when QTS Hero is actually released. but the TS-451+ with 8GB of ram would seem to meet the minimum requirements. If you are not using deduplication or compression, then you can get by with a less powerful CPU and less RAM. I don't expect the self-healing to take that much CPU power. But paying for QTS Hero on a NAS that is low powered and can't take advantage of the performance-enhancing and space-saving features, may not be worth the money.

If you were referring to the dual-core TS-451 without the +, my advice is not to use QTS Hero whether or not it is available for that NAS.

occamsrazor
Know my way around
Posts: 217
Joined: Tue Mar 30, 2010 8:30 pm

Re: QTS Hero... ZFS? What? When?

Post by occamsrazor » Tue Nov 12, 2019 5:00 am

QNAPDanielFL wrote:
Tue Nov 12, 2019 4:45 am
If you were referring to the dual-core TS-451 without the +, my advice is not to use QTS Hero whether or not it is available for that NAS.
Yes it's the TS-451 not the later + version. Oh well, I can't complain, it's an old machine. Looks like I will have to wait until I get a new unit....
Thanks for the information, appreciated.
TS-451 [4 x 10TB WD Reds in Raid-5]
TS-239 Pro II [2 x 3TB in Raid-0]
pfSense router and Ubiquiti Unifi switches
Mac Minis, MacBook Pro, iPhones

Post Reply

Return to “Presales”