Migrate settings, QPKGs, NOT data to new disks on TS-451

[occamsrazor]

Hi, I currently have a TS-451 with 4x4TB WD Red in Raid-5. I now have a new set of 4x10TB WD Red which I plan to swap in and remove the 4x4TB.

First question:

I'm wondering what is the best way to migrate all the settings, QPKGs, etc. BUT because all the main data is just backups of other machines it would seem cleaner to start afresh with a clean volume with no data. But I don't want to have to redo all the settings. I posted similar question on Reddit and was advised to do a disk-by-disk storage expansion but that doesn't seem a clean way to me.

So what exactly does System > Control Panel > Backup/Restore save, and what does it not?

Second question:

Regarding the disk pool/volume setup... I currently just have one big storage pool with one big thick volume of a similar size. Is there a better way to do this? Most of my data is:

- Time Machine backups from various Macs.
- Music and Movie backups cloned across network from Macs
- A few other archived files like software installers

...all of which are just organized in subfolders under the /Multimedia folder.

This has worked OK for me for a number of years, but am wondering if there is a better storage organisation strategy.... Thanks

[P3R]

I posted similar question on Reddit and was advised to do a disk-by-disk storage expansion...

That was a good answer.

So what exactly does System > Control Panel > Backup/Restore save, and what does it not?

It backup only the basic NAS configuration. It's completely unaware of any apps (QPKGs)!

Regarding the disk pool/volume setup... I currently just have one big storage pool with one big thick volume of a similar size. Is there a better way to do this?

In most cases one large volume holding everything is the best as it mean less administration and a minimum of storage overhead.

Thick volumes are my recommendation as they aren't limiting any features and at the same time are easy to understand. Very experienced administrators know themselves when they need thin volumes but if you need to ask, you don't need them (and you can convert later anyway).

There can be reasons to use multiple volumes in special cases but it's often a waste of time and resources on a Qnap. People with a background in administering other systems are often overdoing their Qnaps with multiple volumes because they're used to doing it that way but that doesn't mean it's a good idea.

What's important for the future is the inode setting but hopefully you have at least the default setting of 16384. That will allow future volume expansion up to 63.99 TB.

Also now that you expand your storage pool significantly, my advice is to not expand the volume to the maximum immediately. Expand the volume successively when needed with some TB at a time. That way you have the possibility to add more volumes, snapshots or anything else in the future should you one day feel the need.

This has worked OK for me for a number of years....

That tell me a single thick volume is very good for you. Don't try to fix something that isn't broken!

[occamsrazor]

It backup only the basic NAS configuration. It's completely unaware of any apps (QPKGs)!

Hmm, that's not very smart really. I see why I was recommended the disk-by-disk expansion in this case. I don't know, it just seems an over-complicated way to do things and from what I've read is going to take a really long time for 4x4TB. Also does it result in as clean and reliable a volume as if I initialize it from scratch? I'd be willing to start from a restored basic settings if it's likely to be more reliable long-term.

In most cases one large volume holding everything is the best as it mean less administration and a minimum of storage overhead.
Thick volumes are my recommendation as they aren't limiting any features and at the same time are easy to understand. Very experienced administrators know themselves when they need thin volumes but if you need to ask, you don't need them (and you can convert later anyway).

Thanks, that is very helpful...

What's important for the future is the inode setting but hopefully you have at least the default setting of 16384. That will allow future volume expansion up to 63.99 TB.

Thanks. It's quite likely I will upgrade to a 6/8-bay in the near future so I do want to plan for future possibilities. Is that a setting of the pool or the volume, and where do I find this setting?
EDITED: OK I found it for the existing volume and it is set to 16K.

Also now that you expand your storage pool significantly, my advice is to not expand the volume to the maximum immediately. Expand the volume successively when needed with some TB at a time. That way you have the possibility to add more volumes, snapshots or anything else in the future should you one day feel the need.

Ah I see... so when I replace the disks one by one it would expand the storage pool to fill the 30TB (4x10TB Raid5), but the existing thick volume would stay the same size for now? Then I can manually increase it as I need?
One thing I'd like to do is use snapshots which I don't at the moment because my volume is so full. But I find the way these work really utterly confusing in terms of how to set them up with guaranteed snapshot space etc... Any recommendations?

Thanks for your help... I have attached some screenshots showing current storage setup.

[P3R]

Hmm, that's not very smart really. I see why I was recommended the disk-by-disk expansion in this case. I don't know, it just seems an over-complicated way to do things and from what I've read is going to take a really long time for 4x4TB.

Yes it'll probably take a week or so.

A reinstall, personal fine tune and restore of all data will probably in total take as long but it'll require much more hands on work and active time from you.

Also does it result in as clean and reliable a volume as if I initialize it from scratch?

Impossible to say for sure but most likely yes if you don't have problems with it today.

Thanks. It's quite likely I will upgrade to a 6/8-bay in the near future so I do want to plan for future possibilities.

It's recommended to switch to RAID 6 when wanting to add more disks than 4 and In my opinion, that would be a much better time to redo the configuration from scratch.

Ah I see... so when I replace the disks one by one it would expand the storage pool to fill the 30TB (4x10TB Raid5), but the existing thick volume would stay the same size for now? Then I can manually increase it as I need?

Correct.

One thing I'd like to do is use snapshots which I don't at the moment because my volume is so full. But I find the way these work really utterly confusing in terms of how to set them up with guaranteed snapshot space etc... Any recommendations?

No, I see very little value in local snapshots so I've only enabled it in a small scale for fun as I have some free space but I really don't have any experience or configuration recommendations to share.

I use backups on external systems to protect my data.

[occamsrazor]

Yes it'll probably take a week or so.
A reinstall, personal fine tune and restore of all data will probably in total take as long but it'll require much more hands on work and active time from you.

Wow... I guess no big deal as long as it works OK. I need to have a closer look and see what exactly I'd need to restore manually if I did a config backup/restore. Assuming it backs up all the settings in the control panel... then it probably wouldn't be too big a deal.

It's recommended to switch to RAID 6 when wanting to add more disks than 4 and In my opinion, that would be a much better time to redo the configuration from scratch.

It's likely to be 4-drive Raid5 for quite a while, as the 4x10tb was a significant expense for me, so I'll stick with that for now.
Regarding the inode setting of 16k... if I did do a clean install would you advise to increase that? It seems unlikely I'd go higher than a 60tb storage pool in the future, but am not sure if there are other advantages/disadvantages.

No, I see very little value in local snapshots so I've only enabled it in a small scale for fun as I have some free space but I really don't have any experience or configuration recommendations to share.

I played with it briefly but then stopped as I couldn't really spare the space. That said, I recently had the (direct-attached) drive with all my movies on my Mac die. No problem I thought, I had a cloned backup on my QNAP so ordered another drive for the Mac intending to copy it all back. The night before the new drive was delivered I somehow (!!) managed to manually delete the backup folder on the QNAP... resulting in permanent loss of all my data. In this scenario if I'd had local snapshots enabled I could have simply restored that whole deleted folder.... You've always got to plan for user stupidity

[occamsrazor]

Thanks for all the advice... I've taken onboard everything said and reviewed what apps I'd really need to re-install and configure and it's not actually so much. I can definitely see the arguments for a disk-by-disk expansion... but am also thinking I will use this as an opportunity to maybe set things up a bit better than I did before.
So planning to do a full reinitialization and manually set everything up from scratch, having taken a config backup and screenshotted every settings screen.
Will do one storage pool of the whole 4-disk Raid-5 with inode 16384, then create a thick volume in that.
One question... if I really mess things up for some reason, I should be able to just put back the original 4-disk set and it should boot exactly the same as before I pulled them... correct?

[P3R]

So planning to do a full reinitialization and manually set everything up from scratch, having taken a config backup and screenshotted every settings screen.

Excellent! You don't follow my recommendation strictly but I like it even better that you've thought it over well and is taking an informed decision. Doing it like that will also give you more exposure to the setup and configuration of the system, which is always a useful experience for the future.

Will do one storage pool of the whole 4-disk Raid-5 with inode 16384, then create a thick volume in that.

The inode setting is on the volume (it affects the file system) not the pool.

One question... if I really mess things up for some reason, I should be able to just put back the original 4-disk set and it should boot exactly the same as before I pulled them... correct?

I don't think you need to worry but if you've done a normal controlled shutdown and then replaced all disks, you should be able to put them all back in and return to the current configuration.

Good luck!

[occamsrazor]

Thanks... I've now installed all the new drives and am setting up QTS.
One odd thing - the first 2 bays are reporting "current" SATA speed as 3gbps and "maximum" as 6gbps, whereas the second 2 bays are reporting 6gbps for each. I read a thread or two:
viewtopic.php?t=117467
viewtopic.php?t=127890
...that suggested this could be fixed by swapping the drives around and back again... will try that later.
Currently am doing a bad blocks scan on all the drives before I initialize them into a Raid-5 storage pool, going to take quite a while... does this order of doing that make sense, or better to scan for bad blocks after creating the pool?
Assuming all goes to plan I will reserve 20% of the pool for guaranteed snapshot space and create a thick volume filling all the rest.

[P3R]

Currently am doing a bad blocks scan on all the drives before I initialize them into a Raid-5 storage pool, going to take quite a while... does this order of doing that make sense, or better to scan for bad blocks after creating the pool?

Personally I would probably do it the other way around but neither approach is wrong and the way you do it is more logical.

[occamsrazor]

Hi, Just wanted to say thanks again for all the help.... I got it all set up, one large storage pool, one thick volume (32kb bytes-per-inode in case I were to go above 64TB in future), 20% guaranteed storage space, etc. All good. Been cloning my files onto it for a couple of days now....

The only issue was I experienced was same as this: viewtopic.php?f=25&t=133717&p=622407&hilit=3gbps+6gbps

Opened a ticket and they basically said it wasn't anything to worry about. Not so sure I'm confident with that, but as my unit is out of warranty anyway and I haven't seen any issues I will live with it.