Rsync replication with correct file & folder permission

[snoopstaa]

I have tried to set up rsync to backup files from my windows laptop to my backup folder on my Qnap 219.

I have managed to get the rsync backup to work correctly and the files appear in the correct folder on my Qnap, however, the files do not appear to have the right permissions so I am unable to manage them using Windows.

The only way of accessing the files and folders is via the Web File Manager which is inconvenient. I have tried reading other peoples similar problems with this issue but still cannot work out how to resolve.

Could somebody please provide a step by step guide of resolving this so that I can access and manage the backup files from my laptop.

I think I need to hack the rsync config file on the Qnap, I've downloaded Putty and logged in, but from that point I don't know what to do.

Thanks heaps in advance,

Snoop

[GrahamM]

I'm having something very similar. There are a couple of threads hereabouts that might be worth a look:

http://forum.qnap.com/viewtopic.php?f=22&t=20038
and
http://forum.qnap.com/viewtopic.php?f=32&t=19163

The latter one isn't directly related but it made me think about file/directory permissions and chmod. Directories need to be 777, files need to be 666 - if you set both to 777 then files are "Hidden" as far as Windows is concerned (can't re-find the thread that had that gem but thanks, whoever it was).

So, for an extremely quick and dirty hack, turn on SSH, log in using PuTTY or your preferred ssh client
cd /share/MyShareName
chmod -R 777 *

Then in Windows, turn on the bit in Folder Options to show hidden files.

It should be possible to use some combination of find and something, e.g. xargs, to recursively set the files to 666 but working it out is beyond me since none of the combinations I've tried quite want to work (unless you don't have any file or folder names with spaces in, which I did manage to work out!)

If someone can tell us how to set the file permissions properly on a QNAP box (rather than generic Unix), I'm sure they'll be very popular!

Regards,
Graham.

[GrahamM]

Thanks Bob, that's great. That solves the current permissions problem at least. I need to wait until the next rsync run tonight and see what happens with the permissions on new files then. Don't want to be having to run this manually every day!

I am wondering if the default owner in rsyncd.conf might need changing from admin / administrators but I'm rather hoping QNAP support will get back to me with that before I start on the random fiddling...

Thanks,
Graham.

[Freppa]

I'll add to the last question of this old thread:

I am wondering if the default owner in rsyncd.conf might need changing from admin / administrators

The trouble I'm having is that a friend of mine is going to use rsync to backup to my Qnap 219p. On the Qnap I've created a user and set a user disk quota. After some web surfing we manage to get rsync to backup files to my Qnap. But all files that were backed up had 'admin' as owner, thus not adding to my friends user quota.

How is it possible to specify which user who shall have ownership of the files created with a rsync remote replication? In the rsyncd config file, 'admin' and 'administrator' is listed which made me believe that it is this user who actually is performing the creating of files on the nas? Should we somehow make sure there are same users on both ends (sending and recieving) both names and/or user ID?

I'm feeling that I've only seen the tip of the iceberg here, but with some guidiance I'm sure this can be solved.

[Freppa]

After a little researching I've learned that if the rsync daemon (UID & GID) runs as root (or admin, in the Qnap), the files that are replicated should maintain its owners/permission.

But if the rsync daemon is changed to a user (which of course have to have read/write permission on the destination folder), the files that are copied get this user as owner.

Someone with better insight in the world of Linux can hopefully fill in and explain this in a better way...

[Jonk]

I've just bought 2 x TS119s and am playing with rsync on my home LAN. I noticed the same thing, I was expecting the backup file's owner to be the rsync username and password I set up, but I'm also getting admin as the owner.

I will eventually move one of tyhe NASes to another location and automatically rsync over the internet, but I'm not sure I want to do this until I fully understand little things like this.

Would be interesting to see how this can be changed / explained?

[Freppa]

I am probably not the one to explain this, given that I don't fully understand all concepts of the Linux environment, but...

I logged on to the nas using SSH and opened the rsyncd.conf file.

http://everythinglinux.org/rsync/

In this I used another 'uid' and 'gid' for the module I wanted to change. Everytime this module is used, the owner of the files are changed into the chosen 'uid' for that particular module. (I.e. file owner is NOT perserved, but instead changed. For me this was a good way, since I that way can force all files that my friend want to backup to his user and thus his user share quota.)

Editing the rsyncd.conf file has to be done everytime you change or create a user share in the qnap user interface, since qnap rewrites the config file accordingly.

[JunT]

I'm still having this problem of file ownership and folder permission changing after running a backup from one qnap box to another using Rsync.
Both Qnap boxes are within the same network and under the same domain.

What I notice is that for example, all files under the ownership of "user1" in the original file when rsynced to another box gets the ownership of for example "user2". Meaning the change in ownership from one user to another is consistent. What I need is to retain all file ownership and permissions on the rsync destination.

Any help on this matter is greatly appreciated. thanks.

[ajft]

I'm still having this problem of file ownership and folder permission changing after running a backup from one qnap box to another using Rsync.
Both Qnap boxes are within the same network and under the same domain.

What I notice is that for example, all files under the ownership of "user1" in the original file when rsynced to another box gets the ownership of for example "user2". Meaning the change in ownership from one user to another is consistent. What I need is to retain all file ownership and permissions on the rsync destination.

Any help on this matter is greatly appreciated. thanks.

I suspect that what you're seeing is that the user IDs on the two QNAPs are not matching. Unix (and linux) systems use the UID numbers internally and in the filesystems to show who owns what, the /etc/passwd file will show you the mapping between the user numbers and the user names for local accounts. You'll probably see that on QNAP1 user1 has a UID of "502" and on QNAP2 user2 has a UID of "502". Run "ls -ln" to see the UID numbers rather than user names on both source and destination.

rsync copies the files across, "filex, owned by user id 502, permissions rw-rw-r--"

If you want both your QNAPs to have the same UIDs for local accounts then you can do it by judicious editting of /etc/passwd on the destination box. For AD accounts you need to configure Samba to use UIDs from LDAP lookups in the AD, and as far as I know, QNAP won't let you, it uses auto-generated UIDs.

[JunT]

For AD accounts you need to configure Samba to use UIDs from LDAP lookups in the AD, and as far as I know, QNAP won't let you, it uses auto-generated UIDs.

Thank you for the reply ajft. You have correctly understood my problem.
I have already checked the /etc/passwd but as you said, it's only for the local QNAP accounts. I hope there is a way to configure the UIDs to match that from the source QNAP to the destination QNAP for AD accounts. If there is any work-around on the auto-generated UIDs to match even if it needs to be done manually, that would be of great help.

[ajft]

For AD accounts you need to configure Samba to use UIDs from LDAP lookups in the AD, and as far as I know, QNAP won't let you, it uses auto-generated UIDs.

Thank you for the reply ajft. You have correctly understood my problem.
I have already checked the /etc/passwd but as you said, it's only for the local QNAP accounts. I hope there is a way to configure the UIDs to match that from the source QNAP to the destination QNAP for AD accounts. If there is any work-around on the auto-generated UIDs to match even if it needs to be done manually, that would be of great help.

As far as I can tell there is no work-around with the QNAP implementation of Samba; you're stuck with standalone winbind and auto-generated UID/GID pairs on each QNAP. There are a few requests in the enhancement forum, but it's outside QNAP's target audience so they don't seem to have done anything about it.