nfsv4 sec=krb5p and user impersonation

[Matt Garman]

I'm trying to understand the nuances of how user authentication works
with NFSv4 using the sec=krb5p (or presumably any "krb5" sec option).
In particular, I am concerned about user impersonation.

Here's a situation which hopefully better explains the scenario:

Say there are a bunch of NFSv4 sec=krb5p client Linux servers. These
all mount a single share from an NFS server. That share contains user
home directories. All non-root user accounts authenticate via
Kerberos. Root authentication is local (/etc/passwd, /etc/shadow).

Case 1: I login as root directly to one of the nfs client servers. If
I "su -l" to a user, I still get "permission denied" when I try to see
his home directory. (Unless, of course, I then run kinit and type in
that user's password.)

Case 2: I login first as a user, then "su -l" to root. At this point,
I still get "permission denied" when trying to look at any user's home
directory. But I can then "su -l <user>", where <user> is *anyone*,
and I can see their home directory (without knowing their password).

In short, the only difference between Case 1 and Case 2 is that Case 2
starts off as being logged in as a user, then does su to root; whereas
Case 1 starts off as root directly.

The only thing I can figure is that in Case 2 a Kerberos ticket is
created, since I'm logging in as the user. Since in Case 1, I login
as root, the authentication is local to that machine, and no Kerberos
ticket is created. But in Case 2, it appears that the original user
ticket somehow becomes "universal", in that, after su'ing to root, I
can then su to anyone and see his files.

All Kerberos implementations are MIT, native CentOS (RHEL) packages.
In my case, client systems are CentOS 5.7, using krb5 1.6.1-62.
Server is CentOS 6.4, using krb5 1.10.3-10.

Thanks!
Matt

[Cedric Blancher]

I think part of the problem is that you are using Linux (SLES, RHEL or
Centos doesn't matter), and the Linux NFS client implementation does
not support negotiation for the authentication in violation of RFC
3530.

So if you want to use Kerberos5 with NFS on SLES or RHEL/Centos you a)
must have proper tickets in your cache and use kswitch before calling
mount and b) you must always specify auth=krb5p or krb5i if you want
Kerberos authentication.

Other NFS implementations just negotiate the authentication required
and try from strongest to weakest authentication method as provided by
the server, and even all realms in a DIR: cache, as RFC 3530 requires

Ced
--
Cedric Blancher <cedric....@gmail.com>
Institute Pasteur

[Matt Garman]

On Tue, Sep 9, 2014 at 5:22 PM, Cedric Blancher
<cedric....@gmail.com> wrote:
> I think part of the problem is that you are using Linux (SLES, RHEL or
> Centos doesn't matter), and the Linux NFS client implementation does
> not support negotiation for the authentication in violation of RFC
> 3530.
>
> So if you want to use Kerberos5 with NFS on SLES or RHEL/Centos you a)
> must have proper tickets in your cache and use kswitch before calling
> mount and b) you must always specify auth=krb5p or krb5i if you want
> Kerberos authentication.

I think there are two (at least?) "authentications" taking place, and
you might be addressing the wrong one. It's also possible that I'm
hopelessly lost, but bear with me. :)

I would call "authentication 1" the *machine's* authentication that
allows the mount to take place. All the client servers do the mount
at boot-time. I have created machine principles in my kerberos
database. Each machine has its specific key exported to
/etc/krb5.keytab so that it can do a krb5p mount.

Though Linux may not support authentication negotiation, I don't
believe I care, as I have the exported shares set up for krb5p only.
Likewise, on each client machine's /etc/fstab file, I explicitly pass
the sec=krb5p option to mount.nfs4.

So I *think* I have the mounting setup correct.

What I call "authentication 2" is the actual user- and file-level
permissions, i.e. who can see what file. The share is mounted
regardless. But at this point, under what circumstances is root
allowed to see various users' files? How is it that root can
"authenticate" as user XYZ without knowing XYZ's password, under the
case I outlined in the original email?

Thanks!
Matt

[Nordgren, Bryce L -FS]

> What I call "authentication 2" is the actual user- and file-level permissions, i.e.
> who can see what file. The share is mounted regardless. But at this point,
> under what circumstances is root allowed to see various users' files? How is
> it that root can "authenticate" as user XYZ without knowing XYZ's password,
> under the case I outlined in the original email?

Consider this scenario (which I just verified):

1] I sftp'ed to my fileserver (CentOS 7 + sssd + kerberos5 to active directory). This involved an AS exchange and the creation of a ticket cache.

2] I ssh'ed to my fileserver as root.

3] As root on fileserver: export KRB5CCNAME=KEYRING:persistent:10001 (my uid number)

4] klist now shows my credentials.

5] ldapsearch -Y GSSAPI -H ... works as me.

6] klist shows an additional cross-realm TGT and an ldap service ticket in my cache.

Note that root never had to know my password. So to summarize, All I have to do is look at who is actively using my fileserver, "getent passwd <them>", and set my KRB5CCNAME to their uid. Then I can ssh to whatever other machine I'd like, as them. Or visit kerberized websites, or mount kerberized NFS shares, etc.

This will work for any situation where a user causes a ticket cache to be present on a machine you control, whether they type the password into your machine or they just forward their credentials to you. Note this would not work for an NFS server, as the user would typically only transfer a service ticket for nfs/myserver.example.com.

Kerberos is one step better than host-based authentication, but only one. Essentially, I can't impersonate just anyone, but I can impersonate anyone foolish enough to transfer credentials to my machine. :)

This "feature" implies that some sort of host based access control is necessary. Machines should be reachable from where you are only if: a] the target machine has the same administrator; b] the target machine's administrator trusts your administrator; or c] your machine is an end-user workstation (console) that only one person at a time should be logging into. This partitions a domain into little trust bubbles, anchored in end-user workstations.

Bryce




This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately.

[Matt Garman]

Hi Bryce, thanks for your help. I am unable to duplicate your scenario...

On Tue, Sep 9, 2014 at 6:44 PM, Nordgren, Bryce L -FS
<bnor...@fs.fed.us> wrote:
> 1] I sftp'ed to my fileserver (CentOS 7 + sssd + kerberos5 to active directory). This involved an AS exchange and the creation of a ticket cache.

Instead of this, I ssh in to my server as myself. This results in me
having a credentials cache that I can see with klist, and also I have
a /tmp/krb5cc_myuid_random file.

> 2] I ssh'ed to my fileserver as root.
>
> 3] As root on fileserver: export KRB5CCNAME=KEYRING:persistent:10001 (my uid number)
>
> 4] klist now shows my credentials.

I am unable to reproduce this. I tried both KEYRING:persistent:myuid,
and KEYRING:user:myusername. In both cases, when I run klist after
setting this variable, it says:

klist: No credentials cache found while retrieving principal name

However, if I export KRB5CCNAME=FILE:/tmp/krb5cc_myuid_random, then
run klist, I get the same result as when I run klist natively (as me,
i.e. step [1] above).

So even though I can get klist to show my user's tickets, I still get
"permission denied" if I try to "ls" my nfs4 sec=krb5p mounted home
directory. And, if I try to "kinit myusername" it prompts for my
password.

> Note that root never had to know my password. So to summarize, All I have to do is look at who is actively using my fileserver, "getent passwd <them>", and set my KRB5CCNAME to their uid. Then I can ssh to whatever other machine I'd like, as them. Or visit kerberized websites, or mount kerberized NFS shares, etc.

> ...


Your explanation is extremely helpful. The takeaway here is that root
user can impersonate any Kerberos user on a machine if that user has
an active credentials cache.

However, I'd still like to understand the underlying mechanics to
explain my original scenarios and why I can't reproduce your example
above.

Thanks again,
Matt

[Nordgren, Bryce L -FS]

> I am unable to reproduce this. I tried both KEYRING:persistent:myuid, and
> KEYRING:user:myusername. In both cases, when I run klist after setting this
> variable, it says:
>
> klist: No credentials cache found while retrieving principal name
>
> However, if I export KRB5CCNAME=FILE:/tmp/krb5cc_myuid_random, then
> run klist, I get the same result as when I run klist natively (as me, i.e. step [1]
> above).

Ah. KEYRING:persistent:myuid is just where sssd on my machine puts the ccache. You're quite right to substitute the actual path to wherever yours are stored. Sorry for the confusion.

> So even though I can get klist to show my user's tickets, I still get "permission
> denied" if I try to "ls" my nfs4 sec=krb5p mounted home directory. And, if I
> try to "kinit myusername" it prompts for my password.

Kinit will always prompt for a password. The key to impersonation is to steal the TGT that the person got after they did a kinit. :)

NFS is extra complicated because of the ID mapper. You'd also need to convince your local system to idmap "root" to the user you're trying to impersonate. That way, the NFS server sees you "mapped" as the other user and presenting the other user's credentials. It may just be mapping you as "root@localhost" which would give you permission denied.

In fact, if you "su - username", then export KRB5CCNAME={the user's Kerberos cache}, that may allow NFS to start working. In fact, this may be why it worked in your OP. Don't write this off yet. Getting this to work may just take some tinkering. NFS just has more planets which need to align, that's all.

If you have SSO set up in your domain, however, you can just ssh to some other machine as the user. The other machine should have no idea that you used to be root and should let you do "whatever" to the person's NFS-mounted files. There are many ways to skin this cat.

> Your explanation is extremely helpful. The takeaway here is that root user
> can impersonate any Kerberos user on a machine if that user has an active
> credentials cache.

Yes, so long as it is understood that the local machine is just patient zero. Once impersonated, the scope is not limited to the machine on which the user's cache was deposited. Any machine reachable via SSO. Any kerberized website in the local domain. Any foreign domains reachable by Kerberos trust. The impersonator could be anyone, on any domain-managed-or-reachable resource, for as long as the TGT is renewable.

This was the reason to suggest host based access control rules which limit "potential destinations" reachable from systems where you don't trust root. It may also be a reason to ensure that NFS mountpoints are only shared between clients which have the same administrators, or administrators who trust one another. A little compartmentalization setup to quarantine potentially sick machines, if you will. "Project resources" make good clumps of machines run by people who should trust each other.

> However, I'd still like to understand the underlying mechanics to explain my
> original scenarios and why I can't reproduce your example above.

I would suggest repeating your original experiment, keeping in mind that the two main things which are needed are: access to a Kerberos credentials cache; and correctly mapping to an NFS ID. In your case two, in particular, do a klist before trying to list their home directory. I suspect you will find a correlation between NFS working and the user you're "su'ed" to having access to an active ccache. I'm not going to be much help past this point, I fear, as I lack detailed knowledge of NFS's inner workings.

[Nordgren, Bryce L -FS]

> However, I'd still like to understand the underlying mechanics to explain my
> original scenarios and why I can't reproduce your example above.

The following suggests that spoofing a user as root may require running rpc.gssd with -n...I think I'd suggest su-ing to the user account because getting it to work from root may be complicated.

from "http://linux.die.net/man/8/rpc.gssd":

By default, rpc.gssd treats accesses by the user with UID 0 specially, and uses "machine credentials" for all accesses by that user which require Kerberos authentication. With the -n option, "machine credentials" will not be used for accesses by UID 0. Instead, credentials must be obtained manually like all other users. Use of this option means that "root" must manually obtain Kerberos credentials before attempting to mount an nfs filesystem requiring Kerberos authentication.