Possibility to set UID and GID in User management

Tell us your most wanted features from QNAP products.
daffyd
Starting out
Posts: 11
Joined: Sun Mar 14, 2010 6:53 am

Possibility to set UID and GID in User management

Post by daffyd »

I would love to be able to specify uid and gid when creating new users and groups, to make sure the uid and gid matches what I have already. I'm running three ubuntu-machines, all with uid's beginning with 1000. My Qnap TS-439 ProII starts with 500. I also manually tried changing one user from uid=502 to uid=1000, and after reboot, suddenly ownership of all files created by the 1000-user had been changed to 501.

It's going to be a slow and bothersome process changing uid's and gid's on my three ubuntu machines. (Which I've been using for two years on average.) So I'm not gonna do it, and just continue hating the fact that it's not easy to use the same uid/gid as on an existing network.

Needless to say: I mount via nfs.
gbl
Starting out
Posts: 11
Joined: Sat Mar 20, 2010 11:45 pm

Re: Possibility to set UID and GID in User management

Post by gbl »

If you use *only* nfs then there's no reason for identical user IDs anyway, in fact, you don't even need to define users on the nas, Set up shares and you're ok. Of course, it's different if you want your users to be able to use ftp, or samba, like i do.

If somebody has an existing network that mixes windows and linux/unix, you *need* a way to make the NAS user names/user IDs match the IDs/names on your existing machines, so please please please qnap: give us an option to state the uid/gid when defining a new user.

On my new TS-210, i created the users, then adjusted /etc/passwd and /etc/smbpasswd, rebooted the machine, made sure the user ids hadn't gotten overwritten, then chown()ed the directories and went on happily. (daffyd, maybe you missed the /etc/smbpasswd file and created some files via samba?) But this really should be a GUI option instead of requiring users to ssh to the machine.
daffyd
Starting out
Posts: 11
Joined: Sun Mar 14, 2010 6:53 am

Re: Possibility to set UID and GID in User management

Post by daffyd »

Thank you, gbl, I wasn't aware that I also had to change smbpasswd! :)

I have a heterogenous environment, one machine with Win Vista, and the rest with various versions of ubuntu.

I would guess that it should be really easy to add the gui-option to change uid/gid when creating a user... But I'm pretty sure that Qnap's users mainly uses windows.
john_duff
Getting the hang of things
Posts: 89
Joined: Mon Jun 02, 2008 2:34 am
Location: Ireland

Re: Possibility to set UID and GID in User management

Post by john_duff »

+1
And maybe an easy way of resetting file permission if things get out of hand. In a mixed Ubuntu/Windows environment I find I can write to some folders but not others, depending on whether they were created by ubuntu or windows. Usernames are the same on all machines. Any advice on a tutorial to understand how U & Gids are used in a mixed environment?
TS209 Pro II Version 3.3.0 Build 0924T
2 x 1TB Raid 1 (WDC WD1002FBYS-05A6B00.0 and SAMSUNG HD103UJ 1AA0)
Ubuntu WIN7&8
NFS, Samba, Twonky
gbl
Starting out
Posts: 11
Joined: Sat Mar 20, 2010 11:45 pm

Re: Possibility to set UID and GID in User management

Post by gbl »

(Short answer to john_duff: Mount your share from a linux server. Start the command line. Use chmod/chown/chgrp, possibly using numerical user IDs instead of names, and probably use -R).


I don't know about a tutorial, but it's not that complicated. Explaining it here has the nice side effect of keeping this thread on page 1, to increase the chance of someone at qnap reading it.

1. If you use more than one unix machine, you should really really use nfs mounts, not samba mounts. This is because an nfs mount can be used by many different users at a time, while samba connections are always on a per-user basis. So basically nfs preserves user ids, while samba doesn't. (This isn't the complete truth when we're looking at samba servers with cifs extensions to the smb protocol - but the user mapping done here is a mess that you don't want to mess with. Sorry for the pun.

2. In an nfs environment, user ids have to be synchronized across all machines using the same nfs mounts, because nfs doesn't map user ids. (Again, not the complete truth with nfs v4, and with linux extensions that allow mapping, but again, that's a mess). Professional installations use yp/nis for that. In a home environment, you don't really have to set up nis, but you still want to have the same numeric user IDs for the same users on different machines.

So if you set up several linux machines, make sure you always create your users in the same order, or specify identical user IDs.

If your users already use different ids on different machines, and you want to start using nfs, you will have to clean up user IDs first. Fortunately, that's not as hard as one might think, because the ONLY mapping between user names and numeric IDs is in the /etc/passwd file.

Assume you have two linux machines, donald and daisy. And you have four users, dad, mom, bob, and jill. First, you created dad and mom on donald, and bob and jill on daisy. Later, you added dad to daisy. And now, you decide to move their home directories to the qnap nfs share, and mount the same share on both machines.

On donald, your passwd file probably contains, among others, the lines

dad:x:1000:100:Dad:/home/dad:/bin/bash
mom:x:1001:100:Mom:/home/mom:/bin/bash

and on daisy your users are

bob:x:1000:100:Bob:/home/bob:/bin/bash
jill:x:1001:100:Jill:/home/jill:/bin/bash
dad:x:1002:100:Dad:/home/dad:/bin/bash

Now, donald's dad will always share his files with daisy's bob (same user id), and his files will never belong to dad on daisy (different user id). So you need to get your file owners right on daisy. The plan is: dad's user id should be 1000 everywhere, mom's 1001, bob's 1002, and jill's 1003.

So, log in to daisy as root. (No, we don't want to be logged in as anybody else and use sudo while we're changing their user IDs, so we really want to log in as root.)

Use vi, or another editor, to edit /etc/passwd. Change the 3rd column ("column" being the stuff between colons) to what the user IDs should be. And be sure to not touch anything else, you can easily damage your system to the point of not being able to log on anymore if you mess up the password file.

Now, the password file on daisy should look like:

bob:x:1002:100:Bob:/home/bob:/bin/bash
jill:x:1003:100:Jill:/home/jill:/bin/bash
dad:x:1000:100:Dad:/home/dad:/bin/bash

(We forgot mom right now. We'll come back to her later).

If you save the file, then list the /home/bob directory, you'll see all bob's files belong to dad now. Why that? Because unix stores the numerical user id, not the name, as each file's owner. Bob created all his files using his old user id, 1000. But user 1000 is dad now. What ls does when it displays file owners is retrieve the numerical id from the kernel, then look at /etc/passwd to map the 1000 (that the kernel knows) to the owner name (dad).

To get this right, use the chown -R program:

chown -R 1000 /home/dad
chown -R 1002 /home/bob
chown -R 1003 /home/jill

Now, when you have your user IDs right on all machines, you can start setting up an nfs share.

3. Using your qnap as an NFS server

In an environment that has ONLY linux machines, and where nobody uses samba, or ftp, to access the qnap, you don't even have to set up any users. When a user logged into donald, or daisy, creates a file, the client will send its local user id to the nfs server, who stores the file as "this belongs to uid 1000". The server has no idea that user 1000 is dad. So to move your home directories to the qnap, you just have to set up a share on it - let's call it home - and make it nfs-accessible under the name /home. Then, on your linux clients, still logged in as root, do:

mv /home /home.old
mkdir /home
mount qnap:/home /home

(now you have your old home directories accessible under /home.old, and /home pointing to the qnap)

and copy the files:

tar -C /home.old -cf - . | tar -C /home -xf -
or
rsync -rav /home.old/ /home/

(Don't let people tell you to use cp -r. tar is the tool that preserves all your file permissions, owner, group, modification times, and symlinks, and that every unix system has. cp -r -p -P gets most things right on linux, but that's gnu cp, not unix cp). If you have rsync, the advantage of it is that you can interrupt and restart.

Then, change your /etc/fstab to make the mount permanent. (Or, use your gui. But you already started doing things the guru way when you edited /etc/passwd manually, so you don't want to go back now, do you?)

4. Creating users on your qnap server, and using samba/windows shares

NFS Servers don't want/have to know about user names. But if you want to access your share by another means, like ftp or samba, your qnap has to know your users. So you use the qnap GUI to create dad, mom, bob and jill. Of course, if you have understood everything above, you want them to have user IDs 1000 through 1003. This is what the qnap doesn't allow you to do (at least not officially), and what the thread is about.

So you create your users, and the qnap auto-assigns user ID 500 - 503 to them. Which means, if they ftp to the server, or use samba, their user IDs won't match their file IDs, so they have no access rights (unless you set all files to read/write everybody, but that means dad can't hide his picture collection from mom anymore).

But fortunately, you can edit the /etc/passwd file on your qnap just like you did on daisy, and possibly use chown, so you can get stuff right again. Almost.

The problem is: Samba does its own username/user ID mapping, in the /etc/config/smbpasswd file (on the qnap). So dad, while logged into a linux machine, has user ID 1000, while samba still uses user ID 500 when he's logged into windows and accesses his home share. Which means permissions are still a mess. So, you have to edit /etc/config/smbpasswd on the qnap as well, restart the samba server, and possibly chown -R your home drives once more. Oh, and if any of your linux machines runs a samba server, there's another smbpasswd file to edit.

5. More hints and what else can go wrong?
- If you manually adjust user IDs editing /etc/passwd, the system might "not notice". This is because some linuxes run nscd, the "name server cache daemon", that reads /etc/passwd upon start, then serves out the user ids it has, not rereading the passwd file unless told to. So, before editing anything, check if you have an nscd process and kill it. It's not neccesary for your system to run, it just speeds up things a bit - but that speedup won't be noticable unless you have 1000s of users.
- You can create users as well as editing them by making new /etc/passwd entries. But if you do this, and your system uses shadow passwords, your new users won't be able to log in, and you won't be able to change their passwords, until you create them in /etc/shadow as well. And don't forget to create and chown their home directories.
john_duff
Getting the hang of things
Posts: 89
Joined: Mon Jun 02, 2008 2:34 am
Location: Ireland

Re: Possibility to set UID and GID in User management

Post by john_duff »

Thanks gbl, that's a great explanation.
I will read it a few more times before attempting any changes.
Can I run sudo -s to get a suitable shell to work with in with Ubuntu, or is that not really root?
I have the shares mounted via NFS, but presently still find some folders locked (shares are set to no limit).
control.jpg
locked.jpg
What are the best file permissions for private files? Is 640 getting there?
You do not have the required permissions to view the files attached to this post.
TS209 Pro II Version 3.3.0 Build 0924T
2 x 1TB Raid 1 (WDC WD1002FBYS-05A6B00.0 and SAMSUNG HD103UJ 1AA0)
Ubuntu WIN7&8
NFS, Samba, Twonky
gbl
Starting out
Posts: 11
Joined: Sat Mar 20, 2010 11:45 pm

Re: Possibility to set UID and GID in User management

Post by gbl »

Yes, sudo -s should work well. When you're in the shell that sudo gives you, try the following:

Code: Select all

bash-3.2# id -u
0
bash-3.2#
The single zero is your numerical user id. As the linux kernel only uses user ids, not names, this means you have root rights, no matter what you named root in the /etc/passwd file.

There's still two things that can go wrong, though:
- in a selinux (security enhanced linux) environment, there may be some restrictions that apply even after su, depending how selinux was set up
- nfs servers may, for security reasons, map the "can do everything" root user to a harmless "can do nothing" user id, for security reasons. This is controlled by the no_root_squash option in the /etc/exports file on the nfs server, i.e. the qnap.
Both of this shouldn't be a problem for you. But as i often work with nfs servers in companies, and because of the above limitations, i've gotten myself used to logging in as root on the nfs server and doing stuff from there. This has the additional advantage that, for large directory trees, working directly on the nfs server is often much faster than working from the client. Still, things should work as well when you're on your ubuntu client.

Yes, 640 permissions are ok for private files, unless you want others to be able to edit them (which means 660), or you have a real large installations with several user groups and want to give some access rights to people from other groups (which could mean 664 or 666), or if you're a programmer and want to write scripts/compiled programs you can execute (which means setting the x bit, resulting in 750 or similar). Oh, and you need to set the x bit on directories, or you won't be able to access them, even if you have the read bit set.

So after getting your root shell, do the following (substitute your login name for john):

Code: Select all

grep john /etc/passwd
ls -ldn ~john
This will show you the numerical user and group ids the john user has - first in the password file, second the permissions your home directory (which i assume is not on the qnap) has. Both should be identical. Let's assume they're 1000 (user id) and 100 (group id).

Then, try

Code: Select all

ls -ln /nas/photos
- from your desktop icon view, this seems to be your nas share. Check whether the ids are the same. If they aren't, correct them with

Code: Select all

chown -R 1000:100 /nas/photos
(substitute the ids from step 1). Last, do a

Code: Select all

chmod -R u+rwX /nas/photos
. This will give read/write to the owner, and the x bit where neccessary (this is why the X is upper case; a lower case x would give it to all files).

That should unlock all your folders for you. Remember, if you want to create any new files, whoever creates the new file must "believe" in the same user id - be it a different ubuntu system (local /etc/passwd file there), the qnap ftp server (/etc/passwd on the qnap), or the qnap samba server (/etc/config/smbpasswd on the qnap).
john_duff
Getting the hang of things
Posts: 89
Joined: Mon Jun 02, 2008 2:34 am
Location: Ireland

Re: Possibility to set UID and GID in User management

Post by john_duff »

Great stuff gbl
I seem to have the same id for both user and group:
john:x:1000:1000:John,,,:/home/john:/bin/bash
drwxr-xr-x 54 1000 1000 4096 2010-03-30 18:46 /home/john.
Should this be? Thought I had better ask as i don't want to hose the system immediately! I have it all backed up anyway.
All the folders in Photos seem to have been created as root, (I copied them back in off a usb backup after increasing raid size with 2 TB disks), except one folder(uid 500) which i added presumably through samba:
ls -ln /home/john/nas/photos
total 68
drwxrwxr-x 5 0 0 4096 2010-01-06 19:18 Astronomy
drwxrwxr-x 4 0 0 4096 2009-07-29 17:02 Bright Skies
drwxrwxr-x 2 0 0 4096 2009-07-29 17:02 Computers
drwxrwxr-x 14 0 0 4096 2009-07-29 17:08 Family
drwxrwxr-x 2 0 0 4096 2009-07-29 17:08 Flora
drwxrwxr-x 30 0 0 4096 2010-01-06 19:10 House&Grounds
drwxrwxr-x 2 0 0 4096 2009-07-29 17:09 Jpeged Photos
drwxrwxr-x 16 0 0 4096 2009-07-29 17:09 Kriters
drwxrwxr-x 2 500 100 12288 2009-08-05 16:41 Nebbie
Thanks again, you've given me great help to grok it out, and i have read quite a bit from different books & places, none as good as your explanations.
TS209 Pro II Version 3.3.0 Build 0924T
2 x 1TB Raid 1 (WDC WD1002FBYS-05A6B00.0 and SAMSUNG HD103UJ 1AA0)
Ubuntu WIN7&8
NFS, Samba, Twonky
gbl
Starting out
Posts: 11
Joined: Sat Mar 20, 2010 11:45 pm

Re: Possibility to set UID and GID in User management

Post by gbl »

Yes, having the same id for user and group is well possible. (You could always check /etc/group for the group name that the id belongs to; i bet you have a line similar to users:x:1000: in there). Historically, unix/linux systems left user and group ids from 1 to 100 for system users, but with so many systems that might require separate user ids these days, linux distributors started reserving larger ranges for system-internal use. But there's not really a standard for user ids (except zero being the root user id that has all rights), and the distinction "1-999 for application users, 1000- for real users" is a convention only, has no meaning to the kernel, and there wouldn't be anything wrong with you assigning user IDs 1-10 to your real users and using 9876-9999 for application users.

As long as you play with chown, chgrp and chmod on DATA file systems, you can't really break your system anyway, because everything you do is reversible, at least for root. Of course that's different with system files; removing the x bit from your shell isn't something you want to try. But within your /home/john directory, the worst thing that can happen is removing the read/execute access bits from your home directory, which means subsequent logins will fail, which you can easily fix by logging in as root and correcting access rights from there.

So go ahead and do a chown -R 1000:1000 /home/john/nas/photos, it won't harm you. Just remember that you probably won't be able to use samba with your files until you adjust your user id from 500 to 1000 in the smbpasswd file, and restart the samba server.
john_duff
Getting the hang of things
Posts: 89
Joined: Mon Jun 02, 2008 2:34 am
Location: Ireland

Re: Possibility to set UID and GID in User management

Post by john_duff »

10-4 gbl, I'll try it tomorrow.
TS209 Pro II Version 3.3.0 Build 0924T
2 x 1TB Raid 1 (WDC WD1002FBYS-05A6B00.0 and SAMSUNG HD103UJ 1AA0)
Ubuntu WIN7&8
NFS, Samba, Twonky
john_duff
Getting the hang of things
Posts: 89
Joined: Mon Jun 02, 2008 2:34 am
Location: Ireland

Re: Possibility to set UID and GID in User management

Post by john_duff »

I did chown and chmod as suggested and everything works well. I also changed smbpasswd to reflect the same UID and that works too.
gbl has given me the tools to reclaim my filesystem and understand it better. Thanks again.
And Qnap, please add these functions to the gui to make it easier make these changes.
TS209 Pro II Version 3.3.0 Build 0924T
2 x 1TB Raid 1 (WDC WD1002FBYS-05A6B00.0 and SAMSUNG HD103UJ 1AA0)
Ubuntu WIN7&8
NFS, Samba, Twonky
User avatar
qnutbert
Getting the hang of things
Posts: 55
Joined: Wed Jul 14, 2010 3:43 am

Re: Possibility to set UID and GID in User management

Post by qnutbert »

Yes, please! :mrgreen: :mrgreen: :mrgreen:
Being able to set/change UID and GID via web interface is vital for heterogeneous NFS/SAMBA environments.
Of course IDs can be adjusted by manually editing /etc/passwd, /etc/group and /etc/config/smbpasswd, but I should really be an option in the web interface!

Cheers,
qnutbert
Qnap TS-251+ / QTS 5.0.1.2248 / 8 GB RAM (2x 4GB SK Hynix HMT451S6BFR8A-PB) / 2x Seagate IronWolf ST6000VN001-2BB186 as RAID1 / external ASM1153E USB3 disk for backups
Uaebuntu
New here
Posts: 8
Joined: Sat Feb 26, 2011 6:46 pm

Re: Possibility to set UID and GID in User management

Post by Uaebuntu »

Me too!

QNAP is too windows orientated, some gui tools for linux users, but especially UID GID management would be great. Also better instructions and options for mounting QNAP, I made a few mistakes on my original config and had to rework once, and am about to have to do the same to get access control sorted to my satisfaction.

Good posts here, which I hope will help me sort out access from a 100% Ubuntu home network.
jayadam
New here
Posts: 5
Joined: Wed May 30, 2012 2:38 am

Re: Possibility to set UID and GID in User management

Post by jayadam »

There is a lot of good information in this post. I know its old, but perhaps it is still monitored. The post describes how the NAS assigns UIDs to local users and samba users. Can you tell me how the NAS assigns UIDs if the users are authenticated through active directory? I have verified that a UID is assigned to windows users who create files on the NAS who are authenticated via AD; I just don't know where that UID comes from.
VirtualNobody
New here
Posts: 7
Joined: Mon Dec 02, 2013 11:12 pm

Re: Possibility to set UID and GID in User management

Post by VirtualNobody »

I am disappointed that 3.5 years later, this simple feature request that would save me tonnes of hassle is still not implemented :shock:
Locked

Return to “Features Wanted”