No Support for Kerberized NFS?

[Gabriel Cavalcante]

Hello Folks,

I'm trying to setup a Kerberized NFS Server and clients using the coreOS. Initially I was having the "permission denied" on all mount commands ... Then I Started to debug with rpcdebug. At the end, looks like there is no rpc.gssd binary on /usr/sbin, which I think is need for Krb Connections.

With this in mind, I need to ask: There is no support for NFSv4 with krb5?

Thanks in advance.

[Brandon Philips]

Can you give some more details on your setup?

What NFS server are you using?

What is giving you permission denied?

What did you debug with rpc debug? And where did you expect there to to a rpc.gssd? Inside of the kubelet container?

Thank You,

Brandon

--
You received this message because you are subscribed to the Google Groups "CoreOS User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to coreos-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Gabriel SERPRO]

Hey,

After some headaches my setup is now working gracefully.

I'm using the coreOS itself for server and clients. Actually I had to do the following steps:

Server:

1) Copy the rpc.gssd (rpc krb client) and gssproxy (nfs_sec_krb5)  from the fedora 25 container with some libs to the server;

2) Configure and create service for gssproxy;

3) Configure nfs-server normally and start and enable rpc-statd, rpc-imapd;

4) Create principal and generate the keytab for the server.

5) Optional: copy rpg.gssd and create a service for it (for allow the server itself mount its folder).

Clients
1) Copy the rpc.gssd to the clients;

2) Create principal, generate keytab and create a service for rpc.gssd;

3) Create a systemd mount for the target.

The relevant part of cloud-config.yml in the end is:

write_files:
  - path: "/home/core/gss_prestart.sh"
    permissions: "0555"
    owner: "root"
    content: |
        #!/bin/bash

        if [ ! -f "/home/core/.configured" ]; then
            echo "Creating Stub Container to copy the necessary Binaries"
            docker run -d --name stub fedora:25 sh -c "while true; do sleep 1000; done"
            docker exec stub yum install -y nfs-utils gssproxy -y
            mkdir -p /opt/{bin,etc,lib}
            touch /home/core/.configured
           
            echo "Installing/Updating GssRPC Client Binary."
            docker cp stub:/usr/sbin/rpc.gssd /opt/bin/
            docker cp stub:/usr/lib64/libtirpc.so.3.0.0 /opt/lib/libtirpc.so.3

            echo "Installing/Updating Gssproxy Binary."
            mkdir -p /var/lib/gssproxy/
            mkdir -p /var/lib/fns/rpc_pipefs/gssd
            docker cp stub:/usr/sbin/gssproxy /opt/bin/
            docker cp stub:/usr/lib64/libcollection.so.4.1.0 /opt/lib/libcollection.so.4
            docker cp stub:/usr/lib64/libini_config.so.5.2.0 /opt/lib/libini_config.so.5
               
            cat <<EOT > /opt/etc/gssproxy.conf
        [gssproxy]

        [service/nfs-server]
         mechs = krb5
         socket = /run/gssproxy.sock
         cred_store = keytab:/etc/krb5.keytab
         trusted = yes
         kernel_nfsd = yes
         euid = 0
        EOT

            docker rm -f stub
        fi

    - name: "gss-bootstrap.service"
      enable: true
      command: "start"
      content: |
        [Unit]
        Description=RPC GSS BootStrap
        Before=docker.service rpc-gssd.service
       
        [Service]
        Type=oneshot
        ExecStart=/home/core/gss_prestart.sh
       
    - name: "rpc-gssd.service"
      enable: true
      command: "start"
      content: |
        [Unit]
        Description=RPC GSSAPI Client
        After=network.target rpc-idmapd.service
       
        [Service]
        Type=simple
        Restart=on-failure
        Environment="LD_LIBRARY_PATH=/opt/lib"
        ExecStartPre=/home/core/gss_prestart.sh
        ExecStart=/opt/bin/rpc.gssd -f -k /etc/krb5.keytab
       
        [Install]
        WantedBy=multi-user.target
 
  - name: media-docker.mount
      command: start
      content: |
        [Unit]
        After=gssproxy.service
        [Mount]
        What=10.136.35.71:/media/data
        Where=/media/docker
        Type=nfs

2016-12-29 17:36 GMT-02:00 Brandon Philips <brandon...@coreos.com>:

Can you give some more details on your setup?

What NFS server are you using?

What is giving you permission denied?

What did you debug with rpc debug? And where did you expect there to to a rpc.gssd? Inside of the kubelet container?

Thank You,

Brandon

On Thu, Dec 29, 2016 at 2:50 AM Gabriel Cavalcante <gabriel.cavalcante88@gmail.com> wrote:

Hello Folks,

I'm trying to setup a Kerberized NFS Server and clients using the coreOS. Initially I was having the "permission denied" on all mount commands ... Then I Started to debug with rpcdebug. At the end, looks like there is no rpc.gssd binary on /usr/sbin, which I think is need for Krb Connections.

With this in mind, I need to ask: There is no support for NFSv4 with krb5?

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups "CoreOS User" group.

To unsubscribe from this group and stop receiving emails from it, send an email to coreos-user+unsubscribe@googlegroups.com.

[Gabriel Cavalcante]

Lets go ...

Q: Can you give some more details on your setup?
A: 75 Vm's using coreOS installed using alpha channel.

Q: What is giving you permission denied?
A: My NFS Server was returning permission denied ... but the realm problem is that I cannot start a rpc client with krb5 support because my rpc-statd was stopped and I dont had the rpc.gssd binary running.

Q: What did you debug with rpc debug? And where did you expect there to to a rpc.gssd? Inside of the kubelet container?
A: I tried to debug my nfs and rpc connections.
    Well, I need to mount the volume to use with convoy, so my coreOS should have the binaries needed.

[Brandon Philips]

Hello Gabriel-

Are you using Kubernetes to mount this NFS mount points using krb5 auth?

If we included rpc.gssd and gssproxy inside of Container Linux to auth the NFS server/client would that fix your issue completely?

Brandon

On Thu, Dec 29, 2016 at 2:50 AM Gabriel Cavalcante <gabriel.ca...@gmail.com> wrote:

Hello Folks,

I'm trying to setup a Kerberized NFS Server and clients using the coreOS. Initially I was having the "permission denied" on all mount commands ... Then I Started to debug with rpcdebug. At the end, looks like there is no rpc.gssd binary on /usr/sbin, which I think is need for Krb Connections.

With this in mind, I need to ask: There is no support for NFSv4 with krb5?

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups "CoreOS User" group.

To unsubscribe from this group and stop receiving emails from it, send an email to coreos-user...@googlegroups.com.

[Gabriel Cavalcante]

Hello Brandon,

I'm not using Kubernets at all, just some resources like docker, flannel, calico, etc.

I think if the rpc.gssd and gssproxy were included as well (we already have the rpc.idmapd, rpc.statd in the standard image) it would be awesome to this type of setup.

[Brandon Philips]

Got it. And the entire reason you need these is because your NFS server and clients are authd via kerberos 5, right?

[Gabriel SERPRO]

Just updating the last interactions on the list, unfortunatelly I just repplied for specific ppl, not the list.

---------- Forwarded message ----------
From: Gabriel SERPRO <gabriel.ca...@gmail.com>
Date: 2016-12-29 20:57 GMT-02:00
Subject: Re: No Support for Kerberized NFS?
To: Brandon Philips <brandon...@coreos.com>

Normally only the gssproxy does, but I got what you Said, I'll test the rpc.gssd on monday creating a container and an alias to it.

Em 29 de dez de 2016 8:54 PM, "Brandon Philips" <brandon...@coreos.com> escreveu:

No, I think what you did is necessary. The only thing I would try is to see if running rpc.gssd from inside of a container would work. I don't know how that binary communicates with the kernel though.

On Thu, Dec 29, 2016 at 2:52 PM Gabriel SERPRO <gabriel.ca...@gmail.com> wrote:

Yeah and they re all coreOS as well.

Any other ideias on how can achieve the same setup without the binaries?

To unsubscribe from this group and stop receiving emails from it, send an email to coreos-user+unsubscribe@googlegroups.com.

[Gabriel Cavalcante]

Well, for now, after all I cannot made the nfs + convoy run smoothly because the root user. I tried to use IDMAPD to map the machine into the root account, but I think this is maybe another question.

Em sexta-feira, 30 de dezembro de 2016 07:04:16 UTC-2, Gabriel Cavalcante escreveu:

Just updating the last interactions on the list, unfortunatelly I just repplied for specific ppl, not the list.

---------- Forwarded message ----------
From: Gabriel SERPRO <gabriel.cavalcante88@gmail.com>
Date: 2016-12-29 20:57 GMT-02:00
Subject: Re: No Support for Kerberized NFS?
To: Brandon Philips <brandon...@coreos.com>

Normally only the gssproxy does, but I got what you Said, I'll test the rpc.gssd on monday creating a container and an alias to it.

Em 29 de dez de 2016 8:54 PM, "Brandon Philips" <brandon...@coreos.com> escreveu:

No, I think what you did is necessary. The only thing I would try is to see if running rpc.gssd from inside of a container would work. I don't know how that binary communicates with the kernel though.

[Brandon Philips]

cc Barak and Luis

On Tue, Jan 3, 2017 at 2:25 PM Gabriel Cavalcante <gabriel.ca...@gmail.com> wrote:

Well, for now, after all I cannot made the nfs + convoy run smoothly because the root user. I tried to use IDMAPD to map the machine into the root account, but I think this is maybe another question.

Em sexta-feira, 30 de dezembro de 2016 07:04:16 UTC-2, Gabriel Cavalcante escreveu:

Just updating the last interactions on the list, unfortunatelly I just repplied for specific ppl, not the list.

---------- Forwarded message ----------
From: Gabriel SERPRO <gabriel.ca...@gmail.com>
Date: 2016-12-29 20:57 GMT-02:00
Subject: Re: No Support for Kerberized NFS?
To: Brandon Philips <brandon...@coreos.com>

Normally only the gssproxy does, but I got what you Said, I'll test the rpc.gssd on monday creating a container and an alias to it.

Em 29 de dez de 2016 8:54 PM, "Brandon Philips" <brandon...@coreos.com> escreveu:

No, I think what you did is necessary. The only thing I would try is to see if running rpc.gssd from inside of a container would work. I don't know how that binary communicates with the kernel though.