NAS to NAS or RTRR

[Spider99]

mdm is the disk manager is it not?

as for the caching yes have been keeping an eye on a few threads about this - my thought was its optimised for small files and they forgot about big ones or though it would be ok - needs a switch to say ignore anything over xMB

is this r/w or just a read cache issue?

[cryptochrome]

mdm (or mda?) is the linux raid process.

well they actually have a setting for the SSD cache to ignore anything larger than N. If you set it higher than 1 MB, problems start to kick in, and the higher you go, the worse the problems. QNAP uses lvm-cache for SSD caching and that works perfectly well even with unlimited cache object size under normal conditions. My guess is they either use a buggy lvm version or their implementatin is just borked. Needless to say that QNAP support staff are utterly useless.

[Steerpike]

mdm (or mda?) is the linux raid process.

well they actually have a setting for the SSD cache to ignore anything larger than N. If you set it higher than 1 MB, problems start to kick in, and the higher you go, the worse the problems. QNAP uses lvm-cache for SSD caching and that works perfectly well even with unlimited cache object size under normal conditions. My guess is they either use a buggy lvm version or their implementatin is just borked. Needless to say that QNAP support staff are utterly useless.

Can you clarify what 'normal conditions' means in this context? The way I read the above, SSD caching is great under 'normal conditions' but goes south if you set the SSD cache > 1MB. So it works fine at a setting < 1 MB?

[cryptochrome]

With normal conditions I meant that lvm-cache works just fine with large cache block sizes outside of QNAP, e.g. in any Linux distribution (like Ubuntu). Meaning, it is a supported scenario to use large cache block sizes. QNAP expose that in their GUI (you can even set "unlimited") so it's supposedly supported. It just doesn't work.

When setting cache block size to 1 MB or lower the erratic behavior is still visible but to a much lesser extent. It really starts to go south if you set it to 4 MB or higher.

[Steerpike]

I finally got my two NAS devices in their respective locations (one in AZ, one in CA) and am now doing testing between the two locations. Last night I was able to sync some large files (1GB+ ISO files) using RTRR at about 3.4 Mbps, and my typical home upload speed it about 6Mbps; I now plan to test with NAS to NAS.

But one question arose that people may be able to answer. By accident, I left a bunch of files in the '@Recycle' folder. Typically, I don't plan to use that, and if used, I don't need to replicate it. I found, in the job settings for RTRR job, 'Policy' tab, a setting 'Filter System-Generated Temporary Files'. The 'help' for that item says 'Temporary files created by the system (including thumbnails and @recycle) will be filtered'. So that seems pretty clear; checking that box should stop replicating the @recycle folder; but what else will it stop? 'temporary files created by the system ...' suggests files created by QNAP itself, perhaps, but '...including thumbnails ...' ... does the QNAP create thumbnails? I just don't want to prevent replication of (eg) thumbnails created by my apps and stored on the NAS.

[Spider99]

thumbnails would not be temporary files

these are created by the media apps usually and file manager if you have generate media files still turned on in the preferences

they are stored in hidden directories @__thumb - search on the forum about them - great way to double the number of files on your nas!

[Moogle Stiltzkin]

i recently did a backup for 6-7tb worth of data from my TS-653A to TS-659 Pro II using NAS to NAS in back up station. Eta was 2-3 days :X

So initial backup does take a while, but subsequent backup using the same, should be faster as i can set the option not to replace if file already at destination. It should then copy only files that changed :X

if i'm not mistaken the transfer speed i was getting was between 25-60 ish MB/s or something

Spider99 says rtrr is better so maybe i should have used that

[Spider99]

rtrr - good on the lan - but use the more powerful cpu to do the work - push or pull

rsync better over wan (or more reliable as thats what it was design for originally)

[Moogle Stiltzkin]

rtrr - good on the lan - but use the more powerful cpu to do the work - push or pull

rsync better over wan (or more reliable as thats what it was design for originally)

ty

[Moogle Stiltzkin]

rtrr - good on the lan - but use the more powerful cpu to do the work - push or pull

rsync better over wan (or more reliable as thats what it was design for originally)

thx.

by the way only reason i did nas to nas because it had this option which rtrr did not from settings.

nas to nas p1.jpg

also real time is as implied always on. but i wanted to rely on a scheduled backup rather than 24/7 because if anything happens with the production server, i did not want it to auto backup changes that went bad. so only NAS to NAS had an option for scheduling backup to stagger over a timeline e.g. daily, weekly, monthly etc.

But looking at it another way, maybe initial backup using rtrr then disabling it, to later use NAS to NAS for future scheduling may have worked? But i did not want to backup everything, only specific shares. Not sure if RTRR could do that or not

[Spider99]

rtrr can be done to a time schedule over hours days etc as well as real -time sync (dangerous as you pointed out)

clues in the name real time remote replication

[P3R]

by the way only reason i did nas to nas because it had this option which rtrr did not from settings.

If you mean the option "Only copy files that differ from files at the destination", RTRR always do that. The difference is that you can't disable it in RTRR.

But i did not want to backup everything, only specific shares. Not sure if RTRR could do that or not

Of course it can, shares or even subfolders of shares.

RTRR have been constantly developed and have more features and advantages than NAS to NAS:

• Both real time and scheduled modes
• Two way synchronization (I'm not a fan and don't know how well it handles conflicts but it's available for the many users asking for it)
• Choice of protocols; RTRR, FTP, CIFS/Samba
• Versioning (this alone make it a killer app!)
• Best performance for bulk copying on fast, LAN-like, connections (disable; Check file contents, Compress files during transmission, Delete extra files)

I'd say that today RTRR is almost always preferable and the only reasons to use Nas to NAS are when you want:

• Rsync protocol
• Block based mode (only the parts of a large file that have changed will be transferred)
• Best performance on very slow connections, like <5 Mbit/sec

[Moogle Stiltzkin]

rtrr can be done to a time schedule over hours days etc as well as real -time sync (dangerous as you pointed out)

clues in the name real time remote replication

i was looking at the gui but didn't see it. maybe it only shows up when added first ? i'll check next time xd.

@p3r hm thx for explanation

[Spider99]

yes you need to go through the various screens and the options p3r listed appear on those along with others including the schedule - one of the later screens - just setup a dummy job you can always delete it afterwards

[Moogle Stiltzkin]

yes you need to go through the various screens and the options p3r listed appear on those along with others including the schedule - one of the later screens - just setup a dummy job you can always delete it afterwards

found the resource which includes the step by step procedure
https://www.qnap.com/en/tutorial/con_sh ... one&cid=41

yes u guys were right there was a scheduling option my bad



and some finer settings similar to what NAS to NAS had


and additional settings





So i guess what i should have done is use the schedule mode from the selection. Then once it was set up, i can then click run now (looking at the last screenshot it looks like you would be able to run right now). And when it was completed, i can just stop the scheduling then only run it manually when i need to do it. I don't trust automation, i prefer a hands on approach but now i know the steps for next time so thx.



since i'm doing a review i'll run a rtrr and take some screenshots of the procedure and see if it works or not