Folder access issue on Isilon NFS Export is the user is part of more than 16 local Groups.

[GK]

All,

Posting the folder access issue that was seen on Isilon NAS ... if you have encountered this issue and there is a solution please let me know what it is.

===============

Issue: Unix local Users unable to write to Isilon NFS exported local mount folder if they are a member of more than 16 local groups in local unix system. The reason is that NFSv3 clients use the AUTH_SYS authentication method to pass credentials to the system. Using AUTH_SYS, clients send the user's ID (UID), primary group ID (GID) and up to 16 supplemental GIDs. If the supplimental groups are more than 16 isilon NFS exports only vaidates the first 16 groups and then allows access based on this and causes access denied to those groups other than first 16 sent by clients.  

Issue Documented in Isilon:  Yes. It is https://support.emc.com/kb/89550 . But this KB is not updated recently and dont know if it applies to 8.0.0.4 code version. Tired the " --map-lookup-uid=yes" setting on the Isilon (isi nfs exports modify --id=xxx --map-lookup-uid=true) and did a unmount/mount of Export and it did not resolve it. The user still gets access denied even though the group he is member of is already present on the folder permission with rwx value.

NFS Export Name = nfs.MAXDCISILON01.corp.pep.pvt:/ifs/MAXDCISILON01/NFS/NAB-Informatica-QA

NFS Export Name user use to mount including sub folder = nfs.MAXDCISILON01.corp.pep.pvt:/ifs/MAXDCISILON01/NFS/NAB-Informatica-QA/phap1525_nas/var/opt/maxlab_nas/etlndw ( sub-folder mounting is enabled)

Local mount point = /var/opt/maxlab/etlndw

Problematic folder = /var/opt/maxlab/etlndw/FNDW/logs

Folder Permissions for "logs" folder = 775. User = peletldw ,Group = "etlndw" are the owners (drwxrwxr-x 2 peletldw etlndw 8823 Sep 11 16:15 logs). On the isilon side the folder has only standard POSIX permissions with 3 standard ACLs for user/Groups/others.

What works = User "rkulish" is a member of Group name "etlndw" and he can write to the folder /var/opt/maxlab/etlndw/FNDW/logs. The user is part of less than 16 Unix member groups locally.

What does not work = User "skesava1" is also a member of Group name "etlndw" and he cannot write to the folder /var/opt/maxlab/etlndw/FNDW/logs. The user is part of more than 16 Unix member groups.

******** the user "rkulish" is able to write to the folder since the use is part of less than 16 local groups *****

root@Linxap00694:~# id rkulish | grep -i etlndw

uid=25468(rkulish) gid=22859(cps) groups=22859(cps),2016(etlbdw),343(informat),22895(pbdwetl),24805(etlndw),479(infgrp),27390(pmx),22860(bcps),28376(canpb)

root@Linxap00694:~#

root@Linxap00694:~#  mount | grep -i /var/opt/maxlab/etlndw

nfs.MAXDCISILON01.corp.pep.pvt:/ifs/MAXDCISILON01/NFS/NAB-Informatica-QA/phap1525_nas/var/opt/maxlab_nas/etlndw on /var/opt/maxlab/etlndw type nfs (rw,relatime,vers=3,rsize=131072,wsize=524288,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.2.64,mountvers=3,mountport=300,mountproto=udp,local_lock=none,addr=192.168.2.64)

root@Linxap00694:~# su - rkulish

rkulish@Linxap00694:/home/rkulish> cd /var/opt/maxlab/etlndw/

rkulish@Linxap00694:/var/opt/maxlab/etlndw> pwd

/var/opt/maxlab/etlndw

rkulish@Linxap00694:/var/opt/maxlab/etlndw>

rkulish@Linxap00694:/var/opt/maxlab/etlndw> cd FNDW

rkulish@Linxap00694:/var/opt/maxlab/etlndw/FNDW> ls -la

total 110

drwxrwxr-x 6 peletldw etlndw  114 Sep 10 16:32 .

drwxrwxrwx 4 peletldw etlndw  152 Sep 11 02:33 ..

-rwxrwxr-x 1 peletldw etlndw    0 Sep 10 16:32 abc

drwxrwxr-x 4 peletldw etlndw   67 Sep 11 02:02 data

drwxrwxr-x 2 peletldw etlndw 8823 Sep 11 16:15 logs

drwxrwxr-x 2 peletldw etlndw    0 Jul 15  2010 lost+found

drwxrwxr-x 2 peletldw etlndw    0 Jul 15  2010 tmp

rkulish@Linxap00694:/var/opt/maxlab/etlndw/FNDW> cd logs

rkulish@Linxap00694:/var/opt/maxlab/etlndw/FNDW/logs> whoami

rkulish

rkulish@Linxap00694:/var/opt/maxlab/etlndw/FNDW/logs> id

uid=25468(rkulish) gid=22859(cps) groups=22859(cps),343(informat),479(infgrp),2016(etlbdw),22860(bcps),22895(pbdwetl),24805(etlndw),27390(pmx),28376(canpb)

rkulish@Linxap00694:/var/opt/maxlab/etlndw/FNDW/logs> touch rkulish1.log

rkulish@Linxap00694:/var/opt/maxlab/etlndw/FNDW/logs> ls -la | grep -i rkulish1.log

-rw-r----- 1 rkulish  etlndw      0 Sep 11 17:14 rkulish1.log

rkulish@Linxap00694:/var/opt/maxlab/etlndw/FNDW/logs> exit

root@Linxap00694:~#

root@Linxap00694:~#

****** On the same NFS Exported folder, a different user "skesava1" is unable to write to it ******

root@Linxap00694:~# su - skesava1

skesava1@Linxap00694:/home/skesava1> whoami

skesava1

skesava1@Linxap00694:/home/skesava1>

skesava1@Linxap00694:/home/skesava1> id | grep -i etlndw

uid=26036(skesava1) gid=22895(pbdwetl) groups=22895(pbdwetl),218(ofagrp),301(orastaff),343(informat),479(infgrp),2008(wms),2016(etlbdw),2070(dwetl),3055(pqtgsyb),3135(pqtg),12100(tibgrp),22765(ar),22770(cc),22778(extract),22804(hsa),22806(gli),22864(sfeo),22940(horizops),23104(nis),24470(sdms),24805(etlndw)

skesava1@Linxap00694:/home/skesava1> cd /var/opt/maxlab/etlndw/

skesava1@Linxap00694:/var/opt/maxlab/etlndw> pwd

/var/opt/maxlab/etlndw

skesava1@Linxap00694:/var/opt/maxlab/etlndw>

skesava1@Linxap00694:/var/opt/maxlab/etlndw> cd FNDW

skesava1@Linxap00694:/var/opt/maxlab/etlndw/FNDW> ls -la

total 110

drwxrwxr-x 6 peletldw etlndw  114 Sep 10 16:32 .

drwxrwxrwx 4 peletldw etlndw   71 Sep 11 17:19 ..

-rwxrwxr-x 1 peletldw etlndw    0 Sep 10 16:32 abc

drwxrwxr-x 4 peletldw etlndw   67 Sep 11 02:02 data

drwxrwxr-x 2 peletldw etlndw 8853 Sep 11 17:14 logs

drwxrwxr-x 2 peletldw etlndw    0 Jul 15  2010 lost+found

drwxrwxr-x 2 peletldw etlndw    0 Jul 15  2010 tmp

skesava1@Linxap00694:/var/opt/maxlab/etlndw/FNDW> cd logs

skesava1@Linxap00694:/var/opt/maxlab/etlndw/FNDW/logs> whoami

skesava1

skesava1@Linxap00694:/var/opt/maxlab/etlndw/FNDW/logs> id

uid=26036(skesava1) gid=22895(pbdwetl) groups=22895(pbdwetl),218(ofagrp),301(orastaff),343(informat),479(infgrp),2008(wms),2016(etlbdw),2070(dwetl),3055(pqtgsyb),3135(pqtg),12100(tibgrp),22765(ar),22770(cc),22778(extract),22804(hsa),22806(gli),22864(sfeo),22940(horizops),23104(nis),24470(sdms),24805(etlndw)

skesava1@Linxap00694:/var/opt/maxlab/etlndw/FNDW/logs> touch skesava2.log

touch: cannot touch ‘skesava2.log’: Permission denied ---------------------------------------> Getting permissions denied....

skesava1@Linxap00694:/var/opt/maxlab/etlndw/FNDW/logs>

[Erik Weiman]

It’s a limit that isn’t going away. This is what the NFSv3 RFC calls for and that is what Isilon is matching. That’s accurate for all versions of OneFS that support NFSv3. 

Map lookup UID is very helpful but it requires that these users exist in an auth provider that the cluster can talk to (AD / LDAP / NIS). 

Do you have any user mapping rules in the zone they are connecting from? (System, for example). 

Identity management can be difficult to get the initial setup correct but once setup it works really well in my opinion. 

There are some useful commands that you may want to be aware of regarding testing access without needing an affected user to be present or needing more than their username/UID. 

Should be something like this to check Filesystem access based on effective permissions without taking export settings into account (there should be UID options as well):

# isi auth access -v --user=blah --path=/ifs/data/test

If they don’t have access there that needs to be fixed. 

Here you can see all of the identities the cluster is aware of for this user. 

# isi auth access token --user=blah

Here is how you can effectively “su” to that user from the CLI on the cluster taking advantage of the user mapping rules / full token directly in your current shell session. 

# isi_run -l user 

You may need to look into user mapping rules like merging users identities for identical names, or leveraging rfc2307 AD services for Unix. I would also think that Kerberized NFSv3 would work to handle their identity. It really is dependent upon your exact config and what you expect or need done. And don’t forget in most cases you’ll need traversal (execute) permission from /ifs down the tree to where they mount for that to succeed if there aren’t windows ACLs in the mix. 

There is a white paper you probably want to find that talks about AIMA in more detail than I can type on my phone about.  

--

Erik Weiman 

Sent from my iPhone 7

--
You received this message because you are subscribed to the Google Groups "Isilon Technical User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isilon-user-gr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[TheTomO]

This ended up being an issue with too many *local linux client* groups (/etc/group).  map-lookup-uid allows Isilon to query itself & other providers for more groups ... but since there weren't any on the Isilon & no AD/LDAP/NIS to query, the 16 group limit was in play & caused the denial.  

Testing by creating a local Isilon user + GID to match the one having an issue, then creating a local Isilon group with matching GID (the one beyond #16), flipping map-lookup-uid to true, allowed that 17+th group (just created) to be seen via a local provider & grant access.

Really, the best route, like Erik said, is to go with an external Auth provider for scalability because adding 100+ users and 100+ groups and their corresponding memberships, and making changes will be an administrative nightmare. 

[Gumar K]

Thank you Erick for the quick response. I also got the same reply from Isilon support on this and they said that it is a NFSv3 limitation and advise to have a external authentication providers like - LDAP/NIS/AD and enable map-lookup to make it work. The work around given was to create local users/groups in NAS same as Unix systems and enable map-look-up-uid=true, but that is a cumbersome process and we don't really want to do that. So instructed our Unix team to reduce the member of groups to 16 or less. We dont have any user mapping rules in our environment. Thanks for the commands to check access.