Using 3D Raid 1 in a 3-2-1 backup strategy

Backup, Restore, Netbak Replicator, Cloud Storage Services
Post Reply
Joeypeoples
New here
Posts: 4
Joined: Wed Sep 12, 2018 9:32 am

Using 3D Raid 1 in a 3-2-1 backup strategy

Post by Joeypeoples »

I’m soliciting critiques of my plan to improve my 3-2-1 backup strategy by implementing what can be described as a 3 dimensional Raid 1 strategy using two QNAP NAS devices. I’m considering this in order to address the desire to take advantage of the benefits of a Raid 1 configuration while having two separated on-site copies of data that are always the same at all times. This is intended to compliment my monthly manual on-site backup to an external HDD that is only powered up and connected to the NAS during the backup process. I’m also doing a monthly manual backup to an external HDD that is then taken off-site. Based on past experience, I’m not confident that the monthly manual backups will actually be performed on a “monthly” basis over the long-term. The 3 dimensional Raid 1 strategy is meant to automate the process of having two on-site copies of data that are always current and the same.

The primary NAS device will have two 4TB Seagate Ironwolf HDDs in a Raid 1 configuration. The backup NAS device will have single 4TB Ironwolf HDD. Both NAS devices will have the same CPU processor and amount of RAM memory. A real time two-way synchronization (using RTRR?) will be established between the Raid 1 on the primary NAS and the single HDD on the backup NAS. The intent is to simulate as close as possible a Raid 1 configuration between the two NAS devices. This is the “3D” part of the 3D Raid 1 configuration. The backup NAS will be kept in another part of the building, have its own UPS power supply, and will be on a different building electrical circuit.

The Raid 1 configuration on the primary NAS will ensure that the most current data is maintained if one of the two HDDs fails in the primary NAS. The real time two-way synchronization will ensure that the most current data is available on the backup NAS if the entire primary NAS device itself should fail. The 3D Raid 1 configuration is meant to ensure that the most current data is available if either one of the two NAS devises entirely fails. I chose to use a single HDD on the backup NAS because the Raid 1 configuration on the primary NAS constitutes the first drive of the 3D Raid 1 and the single HDD on the backup NAS constitutes the second drive of the 3D Raid 1 configuration. I think that creating a Raid 1 configuration on the backup NAS is unnecessary and overkill.

In order to address the data transfer rate bottleneck typically found with 1Gbps Ethernet, both NAS devices will be connected to a 2.5Gbps unmanaged switch by CAT 6 or 7 cable. According to the Seagate Ironwolf datasheet, the 4TB drive has a maximum sustained transfer rate OD of 180MBps. If my math is correct, 180MBps equals 1,440Mbps and 2.5Gbps equals 2,560Mbps. Therefore the data should be transferred between the two NAS devices as fast as the HDDs can transfer it. Since HDDs are being used there’s really no need to use a faster switch. The fastest Ironwolf drives have a maximum sustained transfer rate OD of 210MBps (1,680Mbps) so a 2.5Gbps switch is sufficient.

I went back and forth between using raids 1 or 6 before settling on raid 1 as the simplest and least complicated raid to implement. I’m comfortable with having only 4TB of usable storage across three HDDs. I do all of my internet browsing and downloading with a virtual machine, am using antivirus/malware software on the virtual machine, all of my PCs, and the NAS devises so malware contamination is not a high priority for me. My NAS devises will not be connected to the internet so a cloud backup solution is not available for the off-site backup component of the 3-2-1 strategy. Also, the option to locate the backup NAS to a different building is not available.

I’m interested in the critiques of you learned and more experienced QNAP users.
User avatar
dolbyman
Guru
Posts: 35007
Joined: Sat Feb 12, 2011 2:11 am
Location: Vancouver BC , Canada

Re: Using 3D Raid 1 in a 3-2-1 backup strategy

Post by dolbyman »

I think you are over-complicating this .. you basically want a real-time replication to a separate device , a separate NAs with a single disk. Has noting to do with a "3D RAID" (you cannot swap drives around between units). That in itself is a valid redundancy setup but do not confuse this with a "backup" as any accidental or malicious files deletion will be replicated to all parties (the RAID1 and the replication target).
Joeypeoples
New here
Posts: 4
Joined: Wed Sep 12, 2018 9:32 am

Re: Using 3D Raid 1 in a 3-2-1 backup strategy

Post by Joeypeoples »

Thank you for your promt reply dolbyman. After reading your reply I should have said 2D Raid 1. Using the terms "3D" and "2D" is primarily to help me visualize my idea. I was considering one dimensional Raid 1 to be between the two HDDs on the primary NAS. Two dimensional Raid 1 would be simulating Raid 1 between the two NAS devices using real time replication. I understand that I can't swap drives between the units. My goal is to have the most complete and current data on two devices so that if one fails the complete and current data still exists on the other device. I agree that this is not a backup. But unless you know of another solution, it appears to be my best option for preserving the data for in between the monthly backups to external hard drives. Backups to external hard drives on a more frequent basis is not realistic given my schedule. I also understand that any accidental or malicious files deletion will be replicated to all parties (the RAID1 and the replication target). However, for the vast majority of the time, I will be adding/modifying files and not deleting files. This is for my personal setup at home and not for a work/business environment. I've lost data in the past due to not following a sound backup strategy so I'm trying to improve.


"1D Raid 1" = HDD1 <--> HDD2 on the Primary NAS

"2D Raid 1" = "1D Raid 1" on the primary NAS (primary NAS)
^
| = simulated Raid 1 using real time replication
v
HDD on the backup NAS (secondary NAS)
User avatar
Moogle Stiltzkin
Guru
Posts: 11448
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: Using 3D Raid 1 in a 3-2-1 backup strategy

Post by Moogle Stiltzkin »

if all the capacity you require is 4tb then you could do 2x 4tb hdds in a raid1 configuration on your NAS. Then back that up to an external usb storage (or since you claim you got a 2nd nas, you can back it up to that 2nd nas).

Then off site you could go for cloud storage subscription service? or setup a nas at a remote premises to backup to there as well


Probly a good idea to time gap your backup, or at least one of them. reason being so you have time to react in case of malware etc, so you can accidentally backup whatever corruption from main nas to the backup.

example, is you have a 1 day scheduled backup each day, your other backup is a weekly backup or monthly.

with cloud backup services, they may do like versioning which is useful.



how many hard drives were you testing using raid 6 for ? usually that is something used for the 8hdd 8bay nas models. but for something like 3-6 hdds raid5. raid isn't scary. but regardless you would still need a backup
https://www.reddit.com/r/qnap/comments/ ... _a_backup/

but if all you are requiring is 4tb, probly just stick to raid1 :' is sufficient.
NAS
[Main Server] QNAP TS-877 (QTS) w. 4tb [ 3x HGST Deskstar NAS & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A (Truenas Core) w. 4x 2TB Samsung F3 (HD203WI) RaidZ1 ZFS + 8gb ddr3 Crucial
[^] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) 2x 4TB Seagate Ironwolf, Raid5
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-253D (Truenas Scale)
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Qotom Pfsense|100mbps FTTH | Win11, Ryzen 5600X Desktop (1x2tb Crucial P50 Plus M.2 SSD, 1x 8tb seagate Ironwolf,1x 4tb HGST Ultrastar 7K4000)


Resources
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin
User avatar
Moogle Stiltzkin
Guru
Posts: 11448
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: Using 3D Raid 1 in a 3-2-1 backup strategy

Post by Moogle Stiltzkin »

Joeypeoples wrote: Wed Jun 02, 2021 1:24 am
The primary NAS device will have two 4TB Seagate Ironwolf HDDs in a Raid 1 configuration. The backup NAS device will have single 4TB Ironwolf HDD. Both NAS devices will have the same CPU processor and amount of RAM memory. A real time two-way synchronization (using RTRR?) will be established between the Raid 1 on the primary NAS and the single HDD on the backup NAS.

The intent is to simulate as close as possible a Raid 1 configuration between the two NAS devices. This is the “3D” part of the 3D Raid 1 configuration. The backup NAS will be kept in another part of the building, have its own UPS power supply, and will be on a different building electrical circuit.

The Raid 1 configuration on the primary NAS will ensure that the most current data is maintained if one of the two HDDs fails in the primary NAS. The real time two-way synchronization will ensure that the most current data is available on the backup NAS if the entire primary NAS device itself should fail. The 3D Raid 1 configuration is meant to ensure that the most current data is available if either one of the two NAS devises entirely fails. I chose to use a single HDD on the backup NAS because the Raid 1 configuration on the primary NAS constitutes the first drive of the 3D Raid 1 and the single HDD on the backup NAS constitutes the second drive of the 3D Raid 1 configuration. I think that creating a Raid 1 configuration on the backup NAS is unnecessary and overkill.
my advice is to not use non standard terms like 3rd raid which nobody knows what the heck is since you just made that up. you can keep that self reference for your ownself if you think it helps you :' (i don't judge)

simply just explain what it is you are doing and intend to do, that's all. otherwise it just confuses others.


are you trying to use 2 way sync? honestly i never touched that because i don't trust it will work for my own setup.
https://www.skysync.com/one-way-vs-two-way-data-sync/

i only use rtrr or rsync in 1 direction only for any single job at a time. Either a push aka backup, or a pull aka restore.

if i'm not mistaken a 2 way sync means any change in both direction gets replicated to the other side automatically. i don't want that because it's risky if one side replaces the other by mistake :/ also to mitigate risks, i also disable my pull/recover job after i create. so i have manually enable and run it whenever i want to recover. so i don't accidental trigger a recover that may overwrite new data on my main nas by accident. also my backup job (i don't use the hbs backup, i use the hbs one way sync), also has "delete files at destination that don't match the source". Because my main nas will have the current stuff i want backed up, whereas the old backup may have stale files/folders that i had previously intentionally deleted from my main nas, so i don't want to keep them around, hence why i enable that setting.


also make sure you test your backup and restore functions work. what i do is create a new share i call "test", to a destination at the backup location also "test". Then i test the backup and restore settings to ensure they work as i imagined it would, before i create the jobs for my actual shares once it's fully understood how it works, and that it actually works.

and when using raids, make sure you automate a scheduled raid scrub for once a month usually running during non usage hours e.g. over midnight
NAS
[Main Server] QNAP TS-877 (QTS) w. 4tb [ 3x HGST Deskstar NAS & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A (Truenas Core) w. 4x 2TB Samsung F3 (HD203WI) RaidZ1 ZFS + 8gb ddr3 Crucial
[^] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) 2x 4TB Seagate Ironwolf, Raid5
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-253D (Truenas Scale)
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Qotom Pfsense|100mbps FTTH | Win11, Ryzen 5600X Desktop (1x2tb Crucial P50 Plus M.2 SSD, 1x 8tb seagate Ironwolf,1x 4tb HGST Ultrastar 7K4000)


Resources
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin
Post Reply

Return to “Backup & Restore”