[Review] TL-D400S by Moogle

[Moogle Stiltzkin]

I've missed your point @Moogle.

i'm not sure that just doing scheduled scrubbing is sufficient. i leave bitmap enabled as well because i have no ups :}

based on some of the sources i read, the scrub will check if things are right (it also checks that nothing wrong so later when you do need to raid rebuild, it doesn't find out then which will cause the raid to fail because you didn't a scrub to find out sooner). and the bitmap helps in the situation where improper shutdown (or you can potentially incur a write hole situation). hence why i use both

this was all i was suggesting.


When i move to quts hero i don't have to worry about write hole. Is bitmap still a thing for zfs linux? i still have to study about on zfs linux configurations for raid later when i am using it

[OneCD]

Scrubbing assumes that RAID metadata and bitmaps may be inaccurate, so it verifies all RAID metadata, LV data, and parity blocks.

OK, so that disagrees with the earlier statement:

Linux MDADM software RAID also is protected against the 'write hole' phenomenon by using a bitmap (which is enabled by default4).

We need to lock these bloggers in a room together with a few hand tools and let them fight-it-out.

[Moogle Stiltzkin]

We need to lock these bloggers in a room together with a few hand tools and let them fight-it-out.

or have disqus on those articles so people like you can point out any errors. in case some of the facts are wrong i usually also refer to disqus in case somebody does point out something odd.

anyway since you brought it up, i'll carefully double check that blog against other sources next time just in case

[OneCD]

I'm not a RAID guru. But it does seem there are differing opinions as to what these operations actually do.

[Moogle Stiltzkin]

One odd scenario. (couple of oddities actually)

so for some reason, a hard drive in slot 1 got disconnected. this then led to a raid rebuild. this coincidentally happened when i inserted the new seagate hard drive in the 4th slot, so i wonder if that is a bug.

but the 2nd odd issue is, after i pulled out and reinserted hdd 1, and it rebuilds raid. then later i expanded pool by including the new 4th drive and it migrated from 3 x 4tb hdd to 4 x 4tb hdd raid5, when that was done, i checked shares and only 2 out of 5 of my shares were there. the missing 3 were encrypted shares.

and there was an odd Data volume 4 share that is greyed out inaccessible there.


the solution is to create the missing shares. i remembered what the shares names were, but i'm not sure what you can do if u don't quite remember, so maybe someone else has a solution for that.

anyway if you create the shares for the missing ones which are share encrypted, you also have to enable share encryption and enter the same password as previously. otherwise it will say you can't add the share because it already exists.

anyone did all that, shares are back, i can access them, data seems intact.


but that Data volume 4 in shares is still there. it's not something accessible, and when i did refresh it says there is 0 stuff on it afaik.

So i removed it and now it's gone. I double checked my shares on the tl-d400s, and i can't tell whether any data is missing, i presume there isn't any issue. but i can't explain why was there a data vol4 entry in shares, was that a bug?


why did this happen. was this normal that after ejecting, that the shares may go missing and you had to re-add them? or was this due to that raid rebuild earlier? or was this just simply explained that it was a bug? , especially that data volume 4 that got shown in shares?


anyway i reported these odd quirks and possible bugs to helpdesk and see if they can make heads and tails of it, cause i can't

anyway now i am proceeding with my backup job

when i the job finishes i'll post the update results here. but the initial result is blazing fast. this is the fastest i've gotten so far


raid5 x 4tb HDDs

Capacity :12 TB
Speed gain: 3x read speed, no write speed gain
Fault tolerance: 1-drive failure

http://www.raid-calculator.com/default.aspx


both the ts-877 and the tld-400s are configured similarly probably why i can hit this kind of speeds over the tl-d400s connection via the QXP interface

[Moogle Stiltzkin]

Just to clarify a bit more with the recovery process which i mentioned here already
viewtopic.php?f=45&t=155142&start=15#p755820


so when you click the "do you want to recover" click yes (this prompt occurs when you power on the tl-d400s and the QNAP NAS detects it's presence, then it will prompt you this). it then adds back the raid pool/volume. You can see this in storage manager. The status for the tl-d400s raid pool/volume will be optimizing. Let that finish before you begin using the storage space there.

Things to keep in mind. After you eject, and system tells you it finished ejecting, so you then use the power switch to shut off. Then later you power on, qnap nas detects and you ask it to recover. The shares that were share encrypted will dissapear. I ran this recovery process and both times this was the result. So if you use encrypted shares like i do, keep that in mind, but don't panic.

You just have to re-add the missing shares, and ALSO select encrypted share and add back the same password you used.

For non encrypted shares, they will recover just fine without resulting with any missing shares. only encrypted shares will require you to re-add them.

The next thing i did was to verify whether the contents are in all my shares were intact.

I use hybrid backup sync to verify. to do this i run my sync job with the following settings

- remove additional files at destination

- check file contents

- exclude system generated temp files

- do not take snapshots

for the verification i highlighted in red the settings that matter.


run the job and look at the results. here is my example


so there were 20322 files, and out of those 20322 files were skipped. What does this mean? simple. source had that many files, yet that exact number of files were skipped from moving to destination. This is because no changes occured, so no transfer required. (speaking of which, whenever you do a backup, and if you want to be sure, you can run the job with these settings again just to verify everything is there and intact. up to you)

So this verified that the data on the TL-D400S is intact with no errors and can survive just fine through ejection, and later by recovery process.

notes: the filtered files is probably things like the system generated files like thumbnails by QNAP indexing and etc. This is why when you see files/folders in qts > shares, they don't match between your source and destination (backup). That is probably because of those system generated files. So the only way i know to get a side to side comparison for files is via HBS sync

So it works, and now you can confidently rely on it as your backup if you eject and later recover.




What i haven't yet determined is, if after ejecting the TL-D400S, and you reinitialize your NAS, can it still recover?

I think it should work.


But can it recover if you reinitialize by switching from QTS (EXT4 linux where you initially created mdam raid5) to QUTS HERO (zfs linux where they use raidz instead).

I doubt it, but i will test how that works when quts hero comes out I will however have to keep a second backup just in case recovery from the TL-D400S isn't possible in that scenario.

[Moogle Stiltzkin]

tested another eject and recover.

but i did something a bit different.


step1: eject. after it says ejected, power off. then remove the hdds from tl-d400s. then one of the drives, insert into a different hdd slot

step2: power on. it asks to recover, but select NO! Then at this point i tested storage manager disks, was checking a disk on different hdd slots.

step3: eject. then power off. remove the hdd, then reinsert back all hdds. i believe the order matters for raid? so i put them all back in same order.

step4: power on. then in qts it will say do you want to recover, click yes. then you go through the recovery process. But this time something different happened.

all 5 shares were recovered, so i didn't have to recreate them. But i couldn't access them. So i tried to add encryption password, but it says one already exists and wouldn't let me.

the solution was to remove the share but DO NOT select the delete data as well. leave that unticked. Next click add share, and type the same share name again, and ALSO enter the encryption password. and this time it worked. I believe this is a bug, but no data loss, just an inconvenience.

step5: i sync with HBS again from main nas to tl-D400S, just to reverify that there is no data loss. Sure enough all the file numbers match and are skipped.

So just to confirm, even after all that, data remains intact. So i'm pretty confident that ejecting raid5 is a safe procedure as i managed to recover the raid5 into qts without any data loss whatsoever. I can then use this as a reliable backup. However i don't think leaving the tl-d400s connected to the main nas 24/7 is a good idea for a backup. so i will just run my backup, then eject when done. then power it on once a month to do the scrub, then power off again, until the next time i do a backup or i need to recover.

[Moogle Stiltzkin]

another test.

power on tl-d400s.

qnap detects it and asks whether to recover, i select no. because i need it on, so i could merely check for the 2 new hard drive serial numbers via storage & snapshots manager. I was gonna shut it down when i was done, so i didn't want to waste time recovering it when i'm not using it and just gonna boot down anyway.

so after i finished getting my serials and registering at seagate, i click eject and power down.

then next is to wait a minute, then power on again and then recover. i'm just checking whether the tl-d400s can be recovered if you rejected it the first time, while the system is live. it should work, but i am merely finding out just to be sure. i got back ups elsewhere so now is the best time to do this kind of testing while i am able to without worry about data loss :}

anyway i recovered, then used hbs to verify, no issues.


conclussion: so even if you boot up and it asks you, do you want to recover but you click no. You can later recover without issue. but afaik, you need to eject first, then reconnect it again to get that recover prompt, OR perhaps you merely just need to relogin to qts for a detect external device do you want to recover prompt (may work, not sure, didn't try this).


so yes, ejecting the raid5 built with qts, is pretty safe.

even when the raid is recovered, you can still pull out 1 drive for raid5, and all that does is put it into degraded mode as you would expect as per usual. and reinserting the hdd, or a different hdd back will initiate the raid rebuild. the problem only happens if more than 1 hdd gets pulled out before the tl-d400s is ejected, then your raid is probably toasted so keep that in mind.

in events where you do an improper shutdown (either nas or the tl-d400s), or you pull the sff cable (it requires a good yank to even do this by mistake), or a powercut, then i assume it will result in either

1. file system unclean need to check. no raid rebuild required?
2. raid rebuild (may or may not rebuild success, but over the years i haven't seen a raid fail to rebuild, since i use proper raid hdds, so odds are low but is possible to fail).
3. or perhaps it will just boot up and report no issues, although i'm doubtful.


basically standard behaviour even on your nas. just remember, if you want to power of the tl-d400s, don't forget to eject first, and WAIT for the "it's safe to remove now" prompt. In manual they recommend 20-30 ish seconds in between power ons and offs, so best heed that.

[Moogle Stiltzkin]

competition

Unboxing the QNAP TL-D400S 4 Bay JBOD Expansion Enclosure
https://www.youtube.com/watch?v=t5Ky2eufMrg


was watching in case i missed out on something i did not noticed previously. but i don't reckon i did. although the one thing i haven't figured out is how the hardware raid cards work, since i don't have a raid card to try it on

i assume that the hardware raid controller is installed on the desktop pc. but for some of the higher end tl models, it was implied that you could probably install the hardware raid controller on the device itself since it had pcie slots. but nobody has demonstrated that afaik

despite the jbod mo-nik-er, for 3-4 hard drives, i rather just use raid5.

jbod spanning is out of the question. single disk volumes? ideally i don't want to split up my main nas raid5 into separate disk volumes, bit tedious to manage, hence why i rather just use the same raid5 setup. Different story if the single disk volumes could fit the entire raid5 from the main nas onto 1 hard disk because of a high capacity drive, i think that would be ok for a backup, but if i was going that route, i'd probably just go for a wd elements even if it's not as fast (or opt for the 2bay TR model). but because i am using raid5 4 hdds, i opted for the tl-d400s which also has better speed connectivity and can do raid5 for a lan backup.


like rob pointed out, most people would typically use hdds over the ssds for price per gig reasons, so they would normally want the results based on that scenario (which is why i posted my results using 4 hdds in raid5, though my hdds were low RPM and not the best hdds in terms of performance keep in mind. you would need bigger hdd capacities with higher rpm for better results, maybe someone else will provide their results for that setup). still interesting to see what is the max speeds using a ssd setup, if only to see the max possible speeds for this device

that said, speed wise, i don't have complaints. it's better than what i had before, and the speeds is more or less expected based on the hdds i used for raid5.

[Moogle Stiltzkin]

picking up where i left off. was gonna wait till quts hero came around for my ts-877 but it's been quite long.... so gonna go ahead and start fresh because of lingering bugs on nas specifically hbs. also from doing a lot of stuff on nas, just want to clear out all the unecessary junk


few things to do

1. can the tl-d400s recover your data after reinitializing the qnap nas it's attached to?
2. does my backup recovery plan actually work?
3. verify the supposedly new initialization for qts. what's changed?

and yes in the event it doesn't work out, i also have another separate backup on a different nas, just in case



So the first thing to do is backup first.

Then reinitialize
viewtopic.php?f=45&p=788344#p788344

next to go through all the steps to setup again. create the raid, do basic settings, create the shares, do the hbs jobs.

check if mount tl-d400s will retain the data still, then recover


PS: although i am testing tl-d400s recovery ability after a reinitialize, i am at the same time testing about the reinitialization process as a whole, as well as whether all recovery processes works and whether it is smooth or not.

[Moogle Stiltzkin]

I posted gallery pics for the new qts reinitialization setup steps
https://imgur.com/a/zK4yvoj


the process worked seemlessly when reinitializing and using the smart setup (when prompted by qfinder).


reinitialization + initial setup took around 1-2 hours

When setting up the raid, this says it will take 7-8 hours to sync 4x 4tb raid5. So before i can create the shares and attempt at recovering from my backup, i have to wait that long for it to complete.

fyi, you should immediately change the sync priority to highest asap, because by default it isn't set to the fastest speed.


one thing i managed to restore is the Unifi controller qpkg hosted on the qnap. prior to reinitializing the qnap nas, i made a backup of the settings config. so i restore the config, wait a while, and had to stop the controller, and start it again, before it worked. config restoration success.




thing i have yet to do

- turn on tl-d400s and hope i can recover my data (i'm confident it will work, but this is my first time testing it this way)
- confirm that hbs is now without bugs. before i got some error pop up when navigating part of the UI, hopefully reinitializing the nas fixes it
- complete recovery and setting up nas, back to production level status

so it will probably be done tomorrow cauz i have to wait for sync to finish first.

[Moogle Stiltzkin]

Question: Can you restore your data if you backup to a TL-D400S which had raid5 previously setup on it using QTS from a QNAP NAS, but later you reinitialize the same QNAP NAS?

Answer: Okay i'm happy to report that YES YOU CAN! The process is very simple too.


Basically i suggest you backup to the TL-D400S first.

disclaimer: i take no responsibility for any potential data loss. i highly recommend having backups prior to doing any unfamiliar actions or processes that entail risk

Next, detach the TL-D400S. When it says it is safe to remove, then power off the TL-D400S.

Then you can reinitialize your QNAP Nas at this point.


Once your QNAP NAS has reinitialized and setup the raids, the shares, the basic settings etc, you can then power on the TL-D400S. at this point the QNAP will say that it detects it, and ask if you want to mount the device, click "yes".

At this point the shares on the TL-D400S should be visible.

note: QNAP says this about the tl device "To recover your data, connect the device to a NAS that supports roaming, go to Storage and snapshots ? disks/VJBOD, then select recover > attach and recover storage pool". The thing is, i didn't have to do that. Instead, i had the pop up saying that the tl device was detected, do i want to recover it? i click yes. So you can avoid navigating through the sub menu to do that manually yourself, as long as you didn't disable these pop ups notifications for when you power on the tl device and it gets detected, and you did not click "no" when asked about it. fyi.

You can at this point procede to create the restore jobs in hybridbackupsync to restore the backups from the TL-D400S to your QNAP NAS shares.

i've tested throughout the entire process and it works fine on the latest QTS firmware.


The only hiccup, is that for encrypted shares on the TL-D400S, i had to REMOVE those encrypted shares (BUT DO NOT DELETE DATA). Then add back the same shares using the exact same names, and also the same encryption password. This then correctly loads and unlocks the encrypted shares, whereas before it didn't. This is one annoyance in the entire process, but regardless it worked.




So yes, the TL-D400D can definitely be used as a backup. However, there are some caveats (i understood them so i was okay with the tl device despite knowing, but it might not be acceptable to some others)

1. it's dependent on being installed and accessed from a supporting qnap NAS model. If your qnap nas is broke and it's the only one available that supports it, then your only choice is to fix the qnap nas first, or replace it with another qnap nas that supports the tl device.

Note: this is only if you setup the TL device by setting it via QTS, and also connecting directly to the QNAP NAS. BUT, if you set it up to a desktop PC, then issue 1 isn't a problem. However i do not like setting up raid using windows storage spaces which is honestly awful in terms of performance, hence why i don't recommend that mode of use. As for the ubuntu setup method, i don't use ubuntu. There is also a method where you can install a hardware raid controller, but i don't have that either so i couldn't test that.

I have no clue whether recovering the data from the TL device using ubuntu on a desktop pc is possible or not
viewtopic.php?f=25&t=159692&p=780520

I leave that to others to confirm if it's possible or not


2. it connects directly to the QNAP NAS. so it's by no means considered a remote backup because it isn't. You need a separate backup plan if you intend to have a remote backup plan.



On the plus side, the backups and restore performance speed was very fast, because of the direct high speed connection between the nas and the tl device, as i am able to go above the 1gbe bottleneck








also random spot check for checksum which matches, no corruption



and just to be sure, i also ran hbs restore job (the same one) twice. with conflict policy, if it exists, skip. Just to verify that the restore did not miss anything.


3. backing up to a 2nd nas is good in the sense that, if your main nas is down, you can then use the backup nas for use, while waiting for the main nas to get back into working order. You can't do the same with the TL device, at least in the mode where you set it up to connect direct to your qnap nas and setup the raid via qts.

note: you can use your tl device content while you are restoring to your qnap nas and setting it up. however this is totally dependent on whether the qnap nas is functional that you can boot it up to access qts to begin with. if not, then your out of luck, unless you setup the TL where it's not dependent on running from a qnap nas in the other modes of setup that did not go through QTS to do so

[Moogle Stiltzkin]

and then after recovering, to then setup a backup job. i used one way sync though for my backup. i run them to verify that the backup also works

note: after finish recovering data, make sure to disable the recovery job. In case of mistakes where you accidentally run a recovery that was not intended and it may overwrite your main nas data

In total, backing up, reinitialization, qts settings configuration setup, recovering data from backup, setting up and confirming backup works, and finally back to production level daily driver usage took 2 days to accomplish for a 4x 4tb raid5 setup.

most of the delay was due to waiting for the raids to complete sync which took 7 hours roughly.

i proved that the tl-d400s can be used as a backup (with some limitations which i did mention), and that the backup and recovery of my data has been validated as functional (meaning that if disaster does strike *exception being physical theft or environmental disaster, i do not have to wonder whether my backup will work).

it's not enuff to just backup. you should at least test at least once (and perhaps every now and then, annualy?), that your backup plan actually works.... just a friendly reminder

[kapper67]

Hi, thank you for the great review.
Even though the unit clearly says JBOD, I have still a doubt (or rather, hope): in mode 3 (connecting to a pc) would it be possible to make each disk be displayed as a separate unit?
Or would it just bunch them all together like a normal JBOD?
I know you can view them as individual units when connecting to a NAS, but so far haven't found anything clear for mode 3.

Edit: nevermind, happy to discover that the disk are indeed seen as separate units, as seen here:

screen.PNG

Source: https://youtu.be/l7fqvmdf23M?t=503

Another doubt that came up now: the adapter is listed as PCI-E Gen 3 x2, but on my motherboard i only have a PCI-E Gen 2 slot.
Theoretically they should be compatible, just that the bandwith would be cut in half, but I wanted to hear your thoughts.

[Moogle Stiltzkin]

i had to do a clean nas reinstall so i just wanted to take the opportunity for others curious about this TL box whether it can restore data, if it's a reliable backup or not.

There a few modes you can setup this box as. the one i opted for is raid5 created using QTS by connecting to a TS-877 QNAP NAS.



these are the steps for a nas reset (it wipes the data and everything)




step1:

preparation

BACKUP first. In my case i back up to the TL-D400S.

And equally important, once you backup to the TL-D400S, EJECT IT, then POWER OFF. Because if you don't, when you reinitialize, it may try to delete everything on the tl box as well, because it treats the tl box connected to the qnap as part of the system, this is why you need to carefully make sure you remove it after backup BEFORE you factory reset/reinitialize (because that wipes data)



step2:

then factory reset. reinitialize.

default password is the 1st mac address. usually it will be listed in qfinder or the sticker on your nas


remove your pools/volumes e.g. i had a pair of ssds for system volume, so i removed that, then i went to use SSD secure erase. This removes all the previous qts stuff.

For the hdds, they all got wiped during the factory reset/reintialize earlier. You still also have to remove the pool/volume for it as well.

Then run reinitialize

step3:

open qfinder it should tell you a new nas detected and ask if you want to do smart install, click yes.

Then go through the setup procedure.


The first volume you create will be labelled for the system use so create accordingly.

Myself i set system volume to be static volume. Then for hdd volume i did thick (have to create a pool for this).

then setup the shares i had before


step4:

power on the TL box. qnap detects it, mounts the existing volume.

So yes the data is still there. Now i can use hybrid backup sync to recover from it to my brand new volume created after the reinitialize process. so yes i was able to successfully restore my data, and use this tl box as a backup



So in this whole backup process, the weak point is, you need a QNAP nas to connect the TL box to. So if the nas has a hardware failure and can't be boot up, then you can't access your backup unless you get a replacement qnap nas that supports this box.


But this only applies to this particular setup using qts qnap nas for setting up raid5.

If you setup the tl box as single disks, it will be considered an external storage, and this should be able to be accessed from a desktop pc just fine, although installing the addon for the connector is a bit tedious to do.