Solaris 11, zfs share problem no_root_squash

[Juergen Schroeder]

Hello,

I want to share a separate zfs fs /datapool3 on a sol11
server (s5) to clients, which uses solaris 10 or linux.

The problem is on the client I can't change ownership of files, which
is used by e.g. rsync (Yes, I can use rsync via ssh)

On the NFS server

zpool list
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
datapool3 1,80T 736G 1,08T 39% 1.00x ONLINE -
rpool 278G 95,1G 183G 34% 1.00x ONLINE -


I share it with

zfs set share=name=data3,path=/data3,prot=nfs,sec=sys,\
rw=@192.168.100:@192.168.101:host1.domain.de,\
root=@192.168.100:@192.168.101 datapool3

I got

zfs get share datapool3
NAME PROPERTY VALUE SOURCE
name=data3,path=/data3,prot=nfs,sec=sys,root=@192.168.100:@192.168.101,rw=@192.168.100:@192.168.101:host1.domain.de local


zfs get share.nfs.all datapool3%data3
NAME PROPERTY VALUE SOURCE
datapool3%data3 share.nfs.aclok off default
datapool3%data3 share.nfs.anon 0 local
datapool3%data3 share.nfs.charset.* ... default
datapool3%data3 share.nfs.cksum default
datapool3%data3 share.nfs.index default
datapool3%data3 share.nfs.log default
datapool3%data3 share.nfs.noaclfab off default
datapool3%data3 share.nfs.nosub off default
datapool3%data3 share.nfs.nosuid off default
datapool3%data3 share.nfs.public off default
datapool3%data3 share.nfs.sec sys local
datapool3%data3 share.nfs.sec.* ... local

zfs get share.nfs.sec datapool3%data3
NAME PROPERTY VALUE SOURCE
datapool3%data3 share.nfs.sec.sys.none default
datapool3%data3 share.nfs.sec.sys.ro local
datapool3%data3 share.nfs.sec.sys.root @192.168.100:@192.168.101 local
datapool3%data3 share.nfs.sec.sys.root_mapping default
datapool3%data3 share.nfs.sec.sys.rw @192.168.100:@192.168.101:host1.domain.de local




On the (solaris10) client I had

root@client1:/root> showmount -e s5
export list for s5:
/data3 @192.168.100,@192.168.101:host1.domain.de


In the /etc/auto_master

/net -hosts -browse
/nfs auto_nfs -browse

/etc/auto_nfs
data3 -fstype=nfs s5:/data3

The share ist found under /net/ and /nfs. I can

lbs@host2:/prod>cp lptest /net/s5/data3/
lbs@host2:/prod>ls -l /net/s5/data3/lptest
-rwxrwxr-x 1 lbs lbs 64 Sep 8 11:11 /net/s5/data3/lptest*

root@host4:/root> cp testfile /nfs/data3/
root@host4:/root> ls -l /nfs/data3/testfile
-rw-rw-r-- 1 root root 1401 Sep 8 11:25 /nfs/data3/testfile

BUT

root@host4:/root> ll -n /nfs/data3/lptest
-rwxrwxr-x 1 823 811 64 Sep 8 11:31 /nfs/data3/lptest*
root@host4:/root> chown 812 /nfs/data3/lptest
chown: /nfs/data3/lptest: Permission denied <---

root@host4:/root> chown 0 /nfs/data3/lptest
root@host4:/root> ll -n /nfs/data3/lptest
-rwxrwxr-x 1 0 811 64 Sep 8 11:31 /nfs/data3/lptest*

and not back to old uid

root@host4:/root> chown 823 /nfs/data3/lptest
chown: /nfs/data3/lptest: Permission denied <---

Any hints?

Juergen

[John D Groenveld]

In article <uin2cc-...@pcub802.ub-mitarb.Uni-Marburg.DE>,

Juergen Schroeder <js...@gmx.de> wrote:
>root@host4:/root> chown 812 /nfs/data3/lptest
>chown: /nfs/data3/lptest: Permission denied <---

# snoop host host4 port 2049

John
groe...@acm.org

[cindy.sw...@gmail.com]

Hi Juergen,

Your syntax looks correct to me, including the FQDN.

Can you confirm that this NFS server is running Solaris 11 and not a Solaris 11 update release?
If a Solaris 11 update, we can simplify the NFS share syntax.

I wonder if this is the mirror mount problem, where the Solaris 10 or Linux NFS client doesn't traverse the mounted file system descendants, if I'm understanding correctly.

I can try to reproduce this tomorrow between my S11+ system and my S10 system.

Can you run a test with similar syntax between 2 Solaris 11 systems (server and client) to see if it fails the same way?

Thanks, Cindy

[John D Groenveld]

In article <ba3e9d11-2647-49fc...@googlegroups.com>,

<cindy.sw...@gmail.com> wrote:
>Can you confirm that this NFS server is running Solaris 11 and not a
>Solaris 11 update release?

s11server# pkg info entire

s10client# uname -srv

John
groe...@acm.org

[Juergen Schroeder]

Hello

I think its a nfs version problem on Sol 10.

cindy.sw...@gmail.com wrote:
> On Tuesday, September 8, 2015 at 4:08:08 AM UTC-6, Juergen Schroeder wrote:
> >
> > The problem is on the client I can't change ownership of files, which
> > is used by e.g. rsync (Yes, I can use rsync via ssh)

> Your syntax looks correct to me, including the FQDN.

> Can you confirm that this NFS server is running Solaris 11 and not a Solaris 11 update release?
> If a Solaris 11 update, we can simplify the NFS share syntax.

Sol 11 server:

root@suub06:/net/s5/data3>pkg info entire
Name: entire
Zusammenfassung: entire incorporation including Support Repository Update
(Oracle Solaris 11.2.11.5.0).
Herausgeber: solaris
Version: 0.5.11 (Oracle Solaris 11.2.11.5.0)
Build-Version: 5.11
Zweig: 0.175.2.11.0.5.0
Packaging-Datum: 10. Juni 2015 18:06:17 Uhr
Größe: 5.46 KB
FMRI:
pkg://solaris/ent...@0.5.11,5.11-0.175.2.11.0.5.0:20150610T180617Z
root@suub06:/net/s5/data3> cat /etc/release
Oracle Solaris 11.2 SPARC
Copyright (c) 1983, 2015, Oracle and/or its affiliates. All rights
reserved.
Assembled 17 March 2015

Sol 10 client:

uname -srv
SunOS 5.10 Generic_147147-26
cat /etc/release
Oracle Solaris 10 1/13 s10s_u11wos_24a SPARC
Copyright (c) 1983, 2013, Oracle and/or its affiliates. All rights
reserved.
Assembled 17 January 2013

> I can try to reproduce this tomorrow between my S11+ system and my S10 system.

Nice.

> Can you run a test with similar syntax between 2 Solaris 11 systems
> (server and client) to see if it fails the same way?

yes

Sol 11 client:

root@suub06:/net/s5/data3>ll testfile
-rw-rw-r-- 1 root root 1401 Sep 8 11:25 testfile
root@suub06:/net/s5/data3>chown 823:811 testfile
root@suub06:/net/s5/data3>ll testfile
-rw-rw-r-- 1 823 811 1401 Sep 8 11:25 testfile

Linux client:

root@meh:/nfs/data3>ll testfile
-rw-rw-r--+ 1 823 811 1401 Sep 8 11:25 testfile
root@meh:/nfs/data3>chown 800:700 testfile
root@meh:/nfs/data3>ll testfile
-rw-rw-r--+ 1 800 700 1401 Sep 8 11:25 testfile


On sol 11<->11 and linux no problems. Linux has in the auto.nfs

data3 -fstype=nfs,vers=3 192.168.100.15:/data3

This suggest me that I have to told the sol 10 systems to use nfs v3.


> Thanks, Cindy

Thanks also

Juergen

[John D Groenveld]

In article <fbs5cc-...@pcub802.ub-mitarb.Uni-Marburg.DE>,

Juergen Schroeder <js...@gmx.de> wrote:
>This suggest me that I have to told the sol 10 systems to use nfs v3.

Lets see your snoop traffic for the chown error with the S10
NFSv4 client.

John
groe...@acm.org

[Juergen Schroeder]

root@host4:/root> snoop -d vnet1 host host4 port 2049
Using device vnet1 (promiscuous mode)
host4 -> nfsserv NFS C 4 (getattr ) PUTFH FH=6545 GETATTR 10011a b0a23a
nfsserv -> host4 NFS R 4 (getattr ) NFS4_OK PUTFH NFS4_OK GETATTR NFS4_OK
host4 -> nfsserv NFS C 4 (getattr ) PUTFH FH=9590 GETATTR 10011a b0a23a
nfsserv -> host4 NFS R 4 (getattr ) NFS4_OK PUTFH NFS4_OK GETATTR NFS4_OK
host4 -> nfsserv NFS C 4 (setattr ) PUTFH FH=9590 SETATTR ST=SPC0 GETATTR 10011a b0a23a
nfsserv -> host4 NFS R 4 (setattr ) NFS4ERR_BADOWNER PUTFH NFS4_OK SETATTR 0 0
host4 -> nfsserv TCP D=2049 S=622 Ack=4220248314 Seq=2832449379 Len=0 Win=32806 Options=<nop,nop,tstamp 124822954 591931231>

BTW s10 host4 is a LDOM

Juergen

[Juergen Schroeder]

John D Groenveld <groe...@cse.psu.edu> wrote:

root@host4:/root> snoop -d vnet1 host nfsservport 2049

Using device vnet1 (promiscuous mode)

host4-> nfsserv NFS C 4 (getattr ) PUTFH FH=6545 GETATTR 10011a b0a23a
nfsserv-> host4 NFS R 4 (getattr ) NFS4_OK PUTFH NFS4_OK GETATTR NFS4_OK
host4-> nfsserv NFS C 4 (getattr ) PUTFH FH=9590 GETATTR 10011a b0a23a
nfsserv-> host4 NFS R 4 (getattr ) NFS4_OK PUTFH NFS4_OK GETATTR NFS4_OK
host4-> nfsserv NFS C 4 (setattr ) PUTFH FH=9590 SETATTR ST=SPC0 GETATTR 10011a b0a23a
nfsserv-> host4 NFS R 4 (setattr ) NFS4ERR_BADOWNER PUTFH NFS4_OK SETATTR 0 0
host4-> nfsserv TCP D=2049 S=622 Ack=4220249442 Seq=2832450467 Len=0 Win=32806 Options=<nop,nop,tstamp 124829830 591938107>
host4-> nfsserv TCP D=2049 S=622 Fin Ack=4220249442 Seq=2832450467 Len=0 Win=32806 Options=<nop,nop,tstamp 124859824 591938107>
nfsserv-> host4 TCP D=622 S=2049 Ack=2832450468 Seq=4220249442 Len=0 Win=32806 Options=<nop,nop,tstamp 591968107 124859824>
nfsserv-> host4 TCP D=622 S=2049 Fin Ack=2832450468 Seq=4220249442 Len=0 Win=32806 Options=<nop,nop,tstamp 591968107 124859824>
host4-> nfsserv TCP D=2049 S=622 Ack=4220249443 Seq=2832450468 Len=0 Win=32806 Options=<nop,nop,tstamp 124859824 591968107>

BTW sol10 host4 is a LDOM

Juergen

[John D Groenveld]

In article <8dq6cc-...@pcub802.ub-mitarb.Uni-Marburg.DE>,

Juergen Schroeder <js...@gmx.de> wrote:
> host4-> nfsserv NFS C 4 (setattr ) PUTFH FH=9590
>SETATTR ST=SPC0 GETATTR 10011a b0a23a
> nfsserv-> host4 NFS R 4 (setattr )
>NFS4ERR_BADOWNER PUTFH NFS4_OK SETATTR 0 0

NFSv4 expects server and client to agree on user/group id mapping.
See nfsmapid(1M)

John
groe...@acm.org

[John D Groenveld]

In article <fbs5cc-...@pcub802.ub-mitarb.Uni-Marburg.DE>,

Juergen Schroeder <js...@gmx.de> wrote:
>This suggest me that I have to told the sol 10 systems to use nfs v3.

Specify mount_nfs(1M) option vers=3 in your automount(1M) map.

John
groe...@acm.org

[Juergen Schroeder]

John D Groenveld <groe...@cse.psu.edu> wrote:

> Juergen Schroeder <js...@gmx.de> wrote:
> >This suggest me that I have to told the sol 10 systems to use nfs v3.

> Specify mount_nfs(1M) option vers=3 in your automount(1M) map.

Thanks John

/net -hosts -nosuid,browse,vers=3
/nfs auto_nfs -browse,vers=3

this did the trick.

Juergen

[cindy.sw...@gmail.com]

I'm glad this works but I don't understand why you need to do this.

I tried this syntax on system running later bits than S11.3 (NFS server) and a S10u11 (NFS client) and it works like a champ.

Then, I tried identical syntax on S11.3 (NFS server) and the S10u11 (NFS client) and it doesn't work.
I can't even touch a new file with root access in this config.

Very frustrating and I don't know why yet.

Thanks, Cindy

[John D Groenveld]

In article <c58e145e-649f-4c89...@googlegroups.com>,

<cindy.sw...@gmail.com> wrote:
>I'm glad this works but I don't understand why you need to do this.
>
>I tried this syntax on system running later bits than S11.3 (NFS server)
>and a S10u11 (NFS client) and it works like a champ.

Confirm that you're testing with a userid that exists on the NFSv4
client but not on on the server:
s11nfsv4server# getent passwd 666 && getent group 666
s10nfsv4client# groupadd -g 666 foo && useradd -u 666 -g 666 foo
s10nfsv4client# # getent passwd 666 && getent group 666
foo:x:666:666::/home/foo:/bin/sh
foo::666:

Create your share:
s11nfsv4server# zfs create -o sharenfs=on tank/foo
s11nfsv4server# share -F nfs -o rw,root /tank/foo

Mount and test:
s10nfsv4client# mount -o vers=4 s11nfsv4server:/tank/foo /mnt
s10nfsv4client# touch /mnt/bar
s10nfsv4client# getent passwd 666
foo:x:666:666::/home/foo:/bin/sh
s10nfsv4client# chown 666 /mnt/bar
chown: /mnt/foo: Permission denied
s10nfsv4client# getent passwd 667
s10nfsv4client# chown 667 /mnt/bar

Snoop and confirm s11nfsv4server returns bad owner for 666
# snoop port 2049

NFS4ERR_BADOWNER PUTFH NFS4_OK SETATTR 0 0

John
groe...@acm.org

[Juergen Schroeder]

John D Groenveld <groe...@cse.psu.edu> wrote:

> In article <c58e145e-649f-4c89...@googlegroups.com>,
> <cindy.sw...@gmail.com> wrote:
> >I'm glad this works but I don't understand why you need to do this.
> >
> >I tried this syntax on system running later bits than S11.3 (NFS server)
> >and a S10u11 (NFS client) and it works like a champ.

> Confirm that you're testing with a userid that exists on the NFSv4
> client but not on on the server:

yes indeed, a userid known/used on both sides is necessary.

I have always synced my passwd, shadow and group files on the sol 10 boxes.
Between sol 11 and 10 I haven't dared to do this, so some users are missed
on the sol 11 box.

I think the world becomes a bit more complexer with this.

Juergen