Friday, March 09, 2007

User 99, Unknown

Most of us are familiar with typical user accounts associated with Unix systems, such as root, nobody, and daemon. Mac OS X has an additional interesting account for a user named "unknown". Unknown has the UID number 99, which is treated specially within the kernel as well as some user-level libraries. The special properties afforded to unknown are needed to make device sharing between computers as painless as possible. Let us look at what makes unknown so special.

User unknown, or more precisely, the user with a UID of 99 (we will use "user unknown" or "user 99" interchangeably throughout this document), is treated specially in the following ways:


  1. A file owned by UID 99, appears to be owned by whoever is viewing it (see the caveat immediately following)

  2. Volumes mounted with the MNT_IGNORE_OWNERSHIP flag treat all files as if they were owned by UID 99




An important caveat to the first bullet above is that this special treatment does not apply to root. If root views a file owned by unknown, the file appears as it actually is—owned by user 99. Let us look at an example.


$ touch file.txt
$ ls -l file.txt
-rw-r--r-- 1 jgm jgm 0 Mar 9 22:07 file.txt

$ sudo chown 99:99 file.txt
$ ls -l file.txt
-rw-r--r-- 1 jgm jgm 0 Mar 9 22:07 file.txt

$ sudo ls -l file.txt
-rw-r--r-- 1 unknown unknown 0 Mar 9 22:07 file.txt


We can see here that I created the file file.txt, changed its owner and group to 99, but the file continues to show that I own it. However, if I use sudo to list the file as root, then we can see that the real owner of the file is indeed unknown. Further, we can verify the behavior when we list the file as another, non-root user.


$ sudo -u test ls -l file.txt
-rw-r--r-- 1 test test 0 Mar 9 22:07 file.txt


This special treatment is handled in the VFS layer of the kernel, specifically, in the file xnu/bsd/vfs/kpi_vfs.c. In that file, the vnode_getattr() function has logic that looks like this:


int
vnode_getattr(...) {
...

if ((nuid == 99) && !vfs_context_issuser(ctx))
nuid = kauth_cred_getuid(vfs_context_ucred(ctx));
...
}


This shows the logic used when retrieving the attributes of a vnode (basically, a vnode is an in-kernel structure that representats a file). We see that if the vnode is owned by UID 99, and the current calling process is not root, then change the vnode's UID to that of the calling process. The equivalent logic for handling a GID of 99 is not shown here. This is exactly the behavior that was demonstrated above.



The second special property of user unknown mentioned above was that volumes mounted with the MNT_IGNORE_OWNERSHIP flag cause all files to appear as if they were owned by user unknown. Additionally, new files will be created with an owner and group of unknown. In many cases, the MNT_IGNORE_OWNERSHIP flag can be controlled on a per-volume basis by checking the "Ignore ownership on this volume" checkbox in the volume's "Get Info" Finder window. However, it can also be set by specifying MNT_IGNORE_OWNERSHIP when calling mount(2).

We can determine whether or not a volume has this flag set by using the following C program.


$ cat mnt_ownership.c
#include <stdio.h>
#include <sys/param.h>
#include <sys/mount.h>

int
main(int argc, char **argv) {
struct statfs sb;
int ignore_flag;

/* argv[1] is path to the volume or a file/folder within */
statfs(argv[1], &sb);
ignore_flag = (sb.f_flags & MNT_IGNORE_OWNERSHIP);
printf("ownership %s\n", ignore_flag ? "ignored" : "enabled");
return 0;
}

$ gcc -o mnt_ownership mnt_ownership.c -Wall
$ ./mnt_ownership /Volumes/TINY
ownership ignored


We can see here that the mounted volume for my iPod shuffle is ignoring ownership. This means that all files on the iPod should appear to be owned by me (or whomever, depending on the rules discussed above), and files created on the iPod should be created as user 99. Let us look at an example.


$ cd /Volumes/TINY
$ ls -l
total 16
drwxrwxrwx 1 jgm jgm 8192 Jan 27 14:12 iPod_Control/

$ sudo ls -l
total 16
drwxrwxrwx 1 unknown unknown 8192 Jan 27 14:12 iPod_Control

$ touch file.txt
$ ls -l file.txt
-rwxrwxrwx 1 jgm jgm 0 Mar 10 16:27 file.txt

$ sudo ls -l file.txt
-rwxrwxrwx 1 unknown unknown 0 Mar 10 16:27 file.txt


This special behavior is also handled in the VFS layer of the kernel—it's actually handled about 5 lines above the vnode_getattr() snippet discussed above. The relevant code from the function is highlighted here.


int
vnode_getattr(...) {
...
/*
* Handle uid/gid == 99 and MNT_IGNORE_OWNERSHIP here.
*/
...
if (vp->v_mount->mnt_flag & MNT_IGNORE_OWNERSHIP) {
nuid = vp->v_mount->mnt_fsowner;
if (nuid == KAUTH_UID_NONE)
nuid = 99;
...
}


if ((nuid == 99) && !vfs_context_issuser(ctx))
nuid = kauth_cred_getuid(vfs_context_ucred(ctx));
...
}


We see that if the MNT_IGNORE_OWNERSHIP flag is specified, the mnt_fsowner value of the mounted file system is consulted. If that value is KAUTH_UID_NONE, then the kernel hardcodes a value of 99—user unknown. Following that, we go through the same logic as before for handling files owned by 99.

One question this brings up is, what if the mnt_fsowner value is not KAUTH_UID_NONE? In that case, the files on the volume will appear to be owned by the user specified in mnt_fsowner. In the kernel, HFS+ is the only file system that actually makes use of this feature. This fact is actually commented in several places with /* XXX 3762912 hack to support HFS filesystem 'owner' */.


Common Questions and Answers



Does this mean that all users can see files owned by user 99?

No. There is more than simply ownership involved in deciding whether or not you can view a file. For example, if the mode of a file that you own is 000, then you will not be able to read that file. Furthermore, if you are denied access to any directory in a file's path, you will be unable to read it. These are just a few of the reasons why this answer is "no".


Is user 99 only given this special treatment on volumes mounted with MNT_IGNORE_OWNERSHIP?

No. User 99 is treated the same on all volumes mounted under Mac OS X.


Why was this stuff done?

The folks at Apple would know for sure, however, I assume it was added to simplify the sharing of devices (e.g., thumb drives and iPods) among computers. If this were not done, then your real UID would be consulted when determining your access to a file. And the fact that your UID may differ on different computers could make this whole process troublesome.


Should I uncheck "Ignore ownership on this volume" on my devices?

Maybe. If the device is shared among several computers, like an iPod or a thumb drive, then you probably want to leave that box checked (see the answer to the previous question). However, if you have a 500GB external drive that you always leave attached to one machine, then unchecking that box is probably a good idea.


In the first paragraph, you mention that some user-level libraries treat user 99 specially. What are you referring to?

UPDATE: Some Carbon APIs do return incorrect information when displaying metadata about files owned by "unknown" to a root process—they show root as owning the file, when they should report it as user 99. This issue may be in the Carbon framework itself, or in the system calls used to retrieve the information (I haven't looked into it).

11 comments:

leeg said...

For some reason, when I was reading your post in NNW I thought this was a DailyWTF article. It was only the lack of punchline which made me re-evaluate it...and I'm still not sure ;-)

Bill said...

Hi Greg, I'm a bit of Unix Noob and was wondering what types of book or resource you would recommend to someone looking to improve their Unix skills. Thanks!

Drew Thaler said...

UID/GID 99 are such a mess... You're right that it came about because of the requirement that removable media has to work across different systems. But it gets even worse than what's above: if I remember correctly, each major revision of Mac OS X through 10.4 actually had slightly different behaviors because nobody ever really documented how it should behave, and as things were overhauled the behavior drifted slightly.

On either 10.2 or 10.3 (I forget) the kernel would actually lie to root -- as root, you wouldn't see UID 99, you saw UID 0. So the only way to tell from userspace that a file had UID 99 was to check it with two different EUIDs (e.g. your own UID and root) and see if the UIDs the kernel returned were different. If they were, you could infer the presence of UID 99 on the file. Grody to the max.

While I was at Apple I actually campaigned to get UID/GID 99 renamed to "anybody" instead of "unknown", and then have the filesystem publish the correct UID/GID (99) but still leave the same authorization behavior. If that were implemented, then in Terminal you'd see this:

drew% ls -la ~/Documents
-rw-r--r-- 1 drew drew 12 Mar 12 2007 this_file_is_mine
-rw-r--r-- 1 anybody anybody 12 Mar 12 2007 this_file_is_owned_by_99

That seems much more truthful, and makes it way more obvious that the file or directory is accessible by anybody. I don't know the final result of that whole issue, since I wound up leaving Apple, but maybe Leopard will finally change the behavior yet again. :-)

Drew Thaler said...

One more thing, regarding Carbon: "I assume this is because they interact directly with the HFS catalog, so they don't go through the kernel hooks."

Actually, Carbon goes through the low-level BSD calls into the kernel like everyone else. But it also uses some uncommon calls in unistd.h that were specifically designed to support Carbon's APIs -- getattrlist (FSGetCatalogInfo), getdirentriesattr (FSGetCatalogInfoBulk), and searchfs (FSCatalogSearch). IIRC, the main reason you get different behavior from those calls is that they trigger different codepaths in the HFS+ code, some of which handle UID 99 differently and inconsistently.

With that said, I'm actually not positive if there is some residual special-casing inside CarbonCore for UID 99. There might be. Like I said ... a mess.

nigel said...

You're right Drew, we've seen quite different behaviour of 99/99 in different version of OS X, and you've always had to go poking around whatever released source was out there to work out what the hell was going on...

From memory it was 10.2 that lied to root and it was "fixed" in 10.3

"anybody" really does make a lot more sense, and would be such a trivial change to make.

Anonymous said...

Learn about the history of this from the Apple perspective

Sami said...

To nigel: No, 10.3.9 still lies to root. I just checked. (Am I the only person in the known universe who did not upgrade to Tiger?)

Rosyna said...

Anonymous, I would like to point out that article is around seven years old and some of the limitations mentioned in HFS+ and Mac OS X no longer exist. Likewise, some of the implementation has changed dramatically and will continue to change in the future.

One odd thing I did note, is that it mentioned HFS+ uses colons as directory separators. This is not strictly true as HFS+ has no directory separator. The various APIs that developers could call would consider : to be a separator when given a full/relative path but it wasn't a feature of HFS+ itself. Paths are evil and should never be used. Especially by developers.

I just mention this because I know that link, although old, is going to show up on a lot more anti-Apple/Anti-Carbon/Anti-PreNeXT blog posts/forum posts now. But that's how it always is on the internets.

http://xeoss.com/ said...
This comment has been removed by a blog administrator.
Matt Stewart said...
This comment has been removed by a blog administrator.
Roger said...

Thanks for the hint about Unknown user. I've search for a long time the "any" user, countrary to wheel or admin, but never found anything until now.

If you mark user and group as unknown, anyone will have access to it independent of its file permissions. You don't need to alter the traditional user, group or other permissions. Right?

I even started to think Unix groups and permissions would not allow "any" user, but that Access Control Lists (ACL) would do it.