Feature Request Hybrid RAID

Tell us your most wanted features from QNAP products.
User avatar
Toxic17
Ask me anything
Posts: 5664
Joined: Tue Jan 25, 2011 11:41 pm
Location: Planet Earth
Contact:

Re: Feature Request Hybrid RAID

Post by Toxic17 » Thu Mar 18, 2021 11:05 pm

OneCD wrote:
Thu Mar 18, 2021 1:22 pm

If so, it's EOL. https://www.phpbb.com/community/viewtopic.php?t=2534536 :(
So are a lot of threads and subforums. they need archived.
Regards Simon

QTS 4.x User Guidex

QNAP Club Repository
Submit a ticket • QNAP Helpdesk
QNAP Tutorials, User Manuals, FAQs, Downloads, Wiki
When you ask a question, please include the following


NAS: TS-473-32GB QM2-2P QXG-10G1T 4.5.3.1652 • TVS-463-16GB 4.5.3.1652 QM2-2S10G1TB • TS-459 Pro 2GB 4.2.6 • TS-121 4.3.3.1624 • APC Back-UPS ES 700G
Network: VM Hub3 • UniFi UDM Pro 1.10-0.9 • Controller: 6.2.23 • UniFi US-16-150W/US-8-60W 5.60.3 • USW Mini Flex 1.8.4 • UniFi G3-Flex • AP: AC Pro 5.60.3 • U6-LR 5.60.3

User avatar
OneCD
Ask me anything
Posts: 8973
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: Feature Request Hybrid RAID

Post by OneCD » Fri Mar 19, 2021 12:57 am

Toxic17 wrote:
Thu Mar 18, 2021 11:05 pm
So are a lot of threads and subforums. they need archived.
Agree, that's also something I've asked about... it was sometime in 2017. :roll:

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage

User avatar
ETHREAL1
Starting out
Posts: 14
Joined: Sun Mar 14, 2021 7:46 pm

Re: Feature Request Hybrid RAID

Post by ETHREAL1 » Mon Mar 22, 2021 12:20 am

Well that was a waste of time in my opinion. Though posting here apparently won’t get results either.... QNAP got back to me about the feature request. This was there reply;

“Thank you for contacting QNAP. QNAP does not support hybrid raid. Please go to the link below for supported QNAP raid types.”

Guess I’m not getting a QNAP NAS. At least not for a few years..... Thank so much for listening to your customers QNAP....

User avatar
dolbyman
Guru
Posts: 22733
Joined: Sat Feb 12, 2011 2:11 am
Location: Vancouver BC , Canada

Re: Feature Request Hybrid RAID

Post by dolbyman » Mon Mar 22, 2021 12:48 am

so you actually thought your request would immediately cause a multi month multi milion dollar rewrite of the whole storage subsystem...how many units did you offer to buy?

User avatar
jaysona
Been there, done that
Posts: 662
Joined: Tue Dec 02, 2008 11:26 am
Location: Somewhere in the Great White North

Re: Feature Request Hybrid RAID

Post by jaysona » Mon Mar 22, 2021 2:29 am

ETHREAL1 wrote:
Mon Mar 22, 2021 12:20 am
...
Thank so much for listening to your customers QNAP....
But you're not a customer... :p

....and there are other options for Hybrid RAID. If you're really hel lbent on using QNAP hardware, then check the moderz area for ideas on how to install other distros and OS' on QNAP hardware.
H/W: TS-219 Pro / TS-269 Pro / TS-253 Pro (8Gig) / TS-509 Pro x2 / TS-569 Pro
H/W: TS-670 Pro (i7-3770S 16Gig) x2 / TS-853 Pro (8Gig) / TVS-871 Pro (i7-4790S 16Gig)
O/S: Slackware 14.2 / MS Windows 7-64 (x5)
Router1: Asus RT-AC86U - Asuswrt-Merlin - 384.19
Router2: Asus RT-AC68U - DD-WRT v3.0-r39960M kongac
Router3: Linksys WRT1900AC - DD-WRT v3.0-r43028 std
Router4: Asus RT-AC66U - FreshTomato v2020.7
Misc: Popcorn Hour A-110/WN-100, Pinnacle Show Center 250HD, Roku SoundBridge Radio (all retired)

User avatar
ETHREAL1
Starting out
Posts: 14
Joined: Sun Mar 14, 2021 7:46 pm

Re: Feature Request Hybrid RAID

Post by ETHREAL1 » Mon Mar 22, 2021 3:29 am

dolbyman wrote:
Mon Mar 22, 2021 12:48 am
so you actually thought your request would immediately cause a multi month multi milion dollar rewrite of the whole storage subsystem...how many units did you offer to buy?
No. But I would think they would at least be accepting of the request. The way that reply was worded was along the line line of “We don’t support this and don’t plan to add it in any time soon” vibe. That and it’s not like it’s just me on the planet that want this. If they literally worded it differently like “We don’t support hybrid raid at the moment but your request as been noted.” It would be a different story altogether.

User avatar
ETHREAL1
Starting out
Posts: 14
Joined: Sun Mar 14, 2021 7:46 pm

Re: Feature Request Hybrid RAID

Post by ETHREAL1 » Mon Mar 22, 2021 3:43 am

jaysona wrote:
Mon Mar 22, 2021 2:29 am
ETHREAL1 wrote:
Mon Mar 22, 2021 12:20 am
...
Thank so much for listening to your customers QNAP....
But you're not a customer... :p

....and there are other options for Hybrid RAID. If you're really hel lbent on using QNAP hardware, then check the moderz area for ideas on how to install other distros and OS' on QNAP hardware.
I would dare say that potential customers are just as important. I would dare say that anyone weather they know it or not that are in the mat key for a NAS are just as relevant.

As for the latter comments. No. I HATE Linux with a passion. The only times Linux has not been a total nightmare for me is when there a pre done image. Like the image bundled with things like a QNAP NAS. And even then..... and this is all coming from a Network Administrator.... I have to deal with Linux on a near daily basis. Linux does have its perks. Such as being light weight and reliable (when packages aren’t updated) for long periods of time without reboots. When I first became a Network Administrator Linux was not super prominent. It was mostly proprietary stuff like Cisco and Windows server.

Sorry if I sounded mad. I’m not. What I am trying to get across is this is my home setup. I want it to be easy and as stress free as possible. Yes I could even make my own distro if I wanted to.... But I don’t. That’s the point.

P3R
Guru
Posts: 12626
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Feature Request Hybrid RAID

Post by P3R » Mon Mar 22, 2021 5:38 am

ETHREAL1 wrote:
Mon Mar 22, 2021 3:29 am
The way that reply was worded was along the line line of “We don’t support this and don’t plan to add it in any time soon” vibe.
And the vibe you felt is the position I would expect Qnap to have on hybrid RAID. If they had really thought it was worthwhile, they would have followed Syno many years ago.

If it was me I would have been very happy for a polite and honest reply rather than the "we have noted your request" and then bury it along with all the other requests that doesn't fit into the company strategy for a foreseeable future. In what way would that response have been better for you?
That and it’s not like it’s just me on the planet that want this.
No but since Syno and others have already offered it for years, Qnap would be very late into a niche market. Also, those that want this are mostly price sensitive home users so it's low cost products and not a lot of profit to make.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

User avatar
ETHREAL1
Starting out
Posts: 14
Joined: Sun Mar 14, 2021 7:46 pm

Re: Feature Request Hybrid RAID

Post by ETHREAL1 » Mon Mar 22, 2021 6:17 am

P3R wrote:
Mon Mar 22, 2021 5:38 am
ETHREAL1 wrote:
Mon Mar 22, 2021 3:29 am
The way that reply was worded was along the line line of “We don’t support this and don’t plan to add it in any time soon” vibe.
And the vibe you felt is the position I would expect Qnap to have on hybrid RAID. If they had really thought it was worthwhile, they would have followed Syno many years ago.

If it was me I would have been very happy for a polite and honest reply rather than the "we have noted your request" and then bury it along with all the other requests that doesn't fit into the company strategy for a foreseeable future. In what way would that response have been better for you?
That and it’s not like it’s just me on the planet that want this.
No but since Syno and others have already offered it for years, Qnap would be very late into a niche market. Also, those that want this are mostly price sensitive home users so it's low cost products and not a lot of profit to make.
I get where your coming from. But that’s not entirely true. They have entire marketing points about home user features. One such feature is them boasting about Plex hardware encoding. Another such example is the HDMI and media remotes.

As I originally stated Hybrid RAID would be the idea solution with it being automated. But I would be totally content with Manually Definable Nested RAID. There lineup already supports nested RAID. RAID 10, 50, 60, and so on are examples of nester RAID. The difference would be that the user can define what RAID methods are used where. So with my disk configuration I would take the two 6TB drives and RAID them with RAID 0. Then use RAID 5 to RAID the two 6TB serving as one 12TB and the other two 12TB drives. This can be done with Linux though it is annoying without tools to do so. But the tools would not be to difficult to produce and integrate into the QNAP GUI.

User avatar
ETHREAL1
Starting out
Posts: 14
Joined: Sun Mar 14, 2021 7:46 pm

Re: Feature Request Hybrid RAID

Post by ETHREAL1 » Wed Mar 31, 2021 10:52 am

Now that I have done some research into ZFS I am very disappointed in QNAP. ZFS or rather OpenZFS supports software defined RAID by default. It also supports adding any number of disks of any size effectively as its object based. So the fact that QuTS Hero does not support the adding of individual disks at all tells me something is going on. From the research I have done I've come to the conclusion that there is no good reason why these futures where not included other than some other motivation even if that motivation was laziness.

P3R
Guru
Posts: 12626
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Feature Request Hybrid RAID

Post by P3R » Wed Mar 31, 2021 5:37 pm

ETHREAL1 wrote:
Wed Mar 31, 2021 10:52 am
ZFS or rather OpenZFS supports software defined RAID by default.
Yes ZFS is software "RAID" as is Linux md RAID in QTS.

In QuTS Qnap have tried to keep a similar terminology as is used in QTS to lower the threshold for administrators upgrading to QuTS from QTS. This may however cause some confusion when you read standard ZFS documentation. I will try to translate below.
It also supports adding any number of disks of any size effectively as its object based.
Actually ZFS only support nesting additional vdevs (what Qnap call RAID groups) into a ZFS vdev (what Qnap call a Storage pool). A vdev could be a single disk (I think, I never even considered the possibility) or multiple disks in a redundant configuration. You can never add individual disks into an existing ZFS mirror (RAID 1) or RAIDz (RAID 5) vdev though so you can't add a fifth disk into an existing 4-disk RAIDz.

So yes, you can add any size of disk to ZFS and it will actually be handled efficiently but if you add a vdev with less redundancy than the other vdevs in the ZFS array you effectively lower the total disk redundancy to that lower level. If you add multiple single disk vdevs to a zfs you end up with the eqivalent of a JBOD, concatenated single disks without any disk redundancy at all.

To keep disk redundancy in your ZFS array you have to add vdevs with at least two mirrored disks at the same time. The same size rules as with md RAID applies, so the smallest disk of the two determine the usable size of the resulting vdev.
So the fact that QuTS Hero does not support the adding of individual disks at all tells me something is going on.
What QuTS doesn't support is to "Add an individual disk to an existing RAID group". That's exactly the same as the ZFS rule that you can never add disks to an existing vdev.

What Qnap does support is to "Add a RAID group to a Storage Pool". That's the same as the ZFS rule that you can nest additional vdevs into an existing ZFS vdev.

If you think ZFS is closer to your dream of a hybrid RAID, then I'm afraid you will be disappointed as ZFS are in most cases more restricting than standard Linux md RAID when it come to expanding existing storage. ZFS is great for other reasons though. :D
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

User avatar
ETHREAL1
Starting out
Posts: 14
Joined: Sun Mar 14, 2021 7:46 pm

Re: Feature Request Hybrid RAID

Post by ETHREAL1 » Thu Apr 01, 2021 4:06 am

P3R wrote:
Wed Mar 31, 2021 5:37 pm
ETHREAL1 wrote:
Wed Mar 31, 2021 10:52 am
ZFS or rather OpenZFS supports software defined RAID by default.
Yes ZFS is software "RAID" as is Linux md RAID in QTS.

In QuTS Qnap have tried to keep a similar terminology as is used in QTS to lower the threshold for administrators upgrading to QuTS from QTS. This may however cause some confusion when you read standard ZFS documentation. I will try to translate below.
It also supports adding any number of disks of any size effectively as its object based.
Actually ZFS only support nesting additional vdevs (what Qnap call RAID groups) into a ZFS vdev (what Qnap call a Storage pool). A vdev could be a single disk (I think, I never even considered the possibility) or multiple disks in a redundant configuration. You can never add individual disks into an existing ZFS mirror (RAID 1) or RAIDz (RAID 5) vdev though so you can't add a fifth disk into an existing 4-disk RAIDz.

So yes, you can add any size of disk to ZFS and it will actually be handled efficiently but if you add a vdev with less redundancy than the other vdevs in the ZFS array you effectively lower the total disk redundancy to that lower level. If you add multiple single disk vdevs to a zfs you end up with the eqivalent of a JBOD, concatenated single disks without any disk redundancy at all.

To keep disk redundancy in your ZFS array you have to add vdevs with at least two mirrored disks at the same time. The same size rules as with md RAID applies, so the smallest disk of the two determine the usable size of the resulting vdev.
So the fact that QuTS Hero does not support the adding of individual disks at all tells me something is going on.
What QuTS doesn't support is to "Add an individual disk to an existing RAID group". That's exactly the same as the ZFS rule that you can never add disks to an existing vdev.

What Qnap does support is to "Add a RAID group to a Storage Pool". That's the same as the ZFS rule that you can nest additional vdevs into an existing ZFS vdev.

If you think ZFS is closer to your dream of a hybrid RAID, then I'm afraid you will be disappointed as ZFS are in most cases more restricting than standard Linux md RAID when it come to expanding existing storage. ZFS is great for other reasons though. :D
Intriguing. This has sparked more research and at the moment you are right. Though some things to note are that you can actually add single disks to the pool its just not recommended as it creates a redundancy hole. Now also digging around places like github and TrueNAS forums I have discovered something. Apparently the ability to add disks to RAIDZ vdevs and even converting 2 disk mirrors into RAIDZ has been in the works for some time. Since like 2017 in fact. But other issues keep coming up and it gets pushed back. The status can actually be monitored here: https://github.com/openzfs/zfs/pull/8853 The problem at the moment appears to be transient errors becoming permanent. But the two big questions I have are;

1. How does ZFS handle multiple vdevs of differing sizes in a pool? So if you have one vdev that has a usable capacity of say 1TB and a vdev of 2TB and if you copy 1TB of files to the pool does approximately 333GBs (or 1/3) get put on the 1TB vdev with the remaining 666GBs going to the 2TB vdev? and how would expanding a vdev affect that?

2. What is the likelihood and or time scale that QNAP would implement the feature when it becomes available?

Post Reply

Return to “Features Wanted”