Looking for shared folder access as if setuid would work on directories

[K Venken]

I have a number of students which need to cooperate in (different)
groups. The obvious solution would be to setup a group for each as
shared folder. Each shared group has a dedicated user with the same name
(for convenience). For instance, lets call this group/user 'carbon':

addgroup -g 1506 carbon
adduser -b /export/shared/groups -g 1506 -m -s /bin/false -u 1506 carbon

Then the idea was to assign and protect all files in this folder to this
shared group with

chmod 770 /export/shared/groups/carbon
chmod ug+s /export/shared/groups/carbon

And finally add the different members to this group:

usermod -aG student1 carbon
usermod -aG student2 carbon
...

Unfortunately, this does not work. If student1 creates a file in
/export/shared/groups/carbon, it will be owned by group carbon but not
by user carbon. After some digging through some documentation this seems
to be a choice -for Linux at least- see
https://en.wikipedia.org/wiki/Setuid. *Setting the setuid bit on a
directory has no effect*. This is unfortunate for following reasons:

The file is still owned by student1 and can not be edited by student2.
Of course, each student can use umask 0007 but this has two problems:
- this would apply to the other groups each student/mentor is member of
which is an unwanted sideeffect
- this can not be used with files uploaded with WinSCP for instance, it
seems not to be possible to indicate umask when uploading files, or
change protections in this case.

The intention is to have all files in this directory also to be owned by
this dedicated (carbon) user. As I understand it, this will never work
in Linux, or is there a kernel option which allows this?

There are some workaround's

* Setup an NFS share with all_squash,anonuid=1506,anongid=1506
* Create a (number of) account agnostic filesystem (vfat for instance)
in a loopbackfile and let it be mounted by carbon

Another complication is that the home folders and thus also the shared
folders are exported through NFS to a set of different hosts as well
(around 30). Creating new shares or filesystems would require to modify
the NFS /etc/exports in the server and the NFS imports (/etc/fstab) in
all the clients as well. This is not needed if the shared folders are
part of the exported filesystem. (To be complete, NIS is used to
distribute the accounts)

This brings me to the question if there is a better way to emulate this
effect?

Apart from my question, it peeks my curiosity. Why was this choice made?
Is there any rationale behind it. For BSD it would work, see
https://forums.freebsd.org/threads/suid-bit-on-a-directory.41960/, but
then not by default on Solaris.

[Chris Elvidge]

You need to set access control lists too. See:
https://superuser.com/questions/381416/how-do-i-force-group-and-permissions-for-created-files-inside-a-specific-directo
and/or
https://unix.stackexchange.com/questions/12842/make-all-new-files-in-a-directory-accessible-to-a-group

--
Chris Elvidge
England

[K Venken]

That works great on a single host, but not on an NFS exported directory.
I created 2 directories owned by carbon:carbon with permissions
rwxrwx---, so a regular user cannot access it unless he is added to the
group.

For a standard directory, after logging in on a remote host (and NIS
having the carbon user group made available) I can go into it.

For a directory with the acl set I get permission denied. Even when I
use both commands setfacl on the NFS client as well I still get
permission denied. It seems that acl is not working accros NFS? In this
case it is not an option.

[Richard Kettlewell]

K Venken <karel....@domain.invalid> writes:
> I have a number of students which need to cooperate in (different)
> groups. The obvious solution would be to setup a group for each as
> shared folder. Each shared group has a dedicated user with the same
> name (for convenience). For instance, lets call this group/user
> 'carbon':

The solution everyone else uses is to just use groups (with the setgid
bit on directories to ensure all files get the right group), and not
worry about the owning user on the shared files.

Your users will need to use a umask of 002. You will need to give each
user their own private group so that their private files don’t end up
writable by a common group.

[...]

> Apart from my question, it peeks my curiosity. Why was this choice
> made?

Because the expected solution is to use groups.

--
https://www.greenend.org.uk/rjk/

[K Venken]

Richard Kettlewell wrote:
> K Venken <karel....@domain.invalid> writes:
>> I have a number of students which need to cooperate in (different)
>> groups. The obvious solution would be to setup a group for each as
>> shared folder. Each shared group has a dedicated user with the same
>> name (for convenience). For instance, lets call this group/user
>> 'carbon':
>
> The solution everyone else uses is to just use groups (with the setgid
> bit on directories to ensure all files get the right group), and not
> worry about the owning user on the shared files.

The owner matters as others cannot edit the files (see next) This is why
having setuid on a directory changing the owner would be perfect.

> Your users will need to use a umask of 002.

Which is only used when creating files from the host itself --this is
after interactive login--, not when using scp, WinSCp etc. The group
permissions are r-- when uploading files, so group members can't edit
each other files.

You will need to give each
> user their own private group so that their private files don’t end up
> writable by a common group.

Indeed, not the default in Slackware as all users end up in the users
group (when using adduser) as primary group, but it makes sense to get
the users out of this group and have them their own group.

> [...]
>> Apart from my question, it peeks my curiosity. Why was this choice
>> made?
>
> Because the expected solution is to use groups.

As intended, but it leaves the umask problem to be fixed.

So far,...

ACL won't hold in NFS exported filesystems
umask won't hold for scp, ftp,...

[Grant Taylor]

On 2/26/21 3:58 AM, K Venken wrote:
> The file is still owned by student1 and can not be edited by student2.
> Of course, each student can use umask 0007 but this has two problems:
> - this would apply to the other groups each student/mentor is member of
> which is an unwanted sideeffect
> - this can not be used with files uploaded with WinSCP for instance, it
> seems not to be possible to indicate umask when uploading files, or
> change protections in this case.

Dare I say it, it sounds like you need POSIX ACLs.

Remember, that ACLs can't /add/ permissions, they can only /remove/
permissions.

So, if you do set the umask to (0)007, you could then more precisely
remove group permissions that you don't want.

The other /hack/ is to have a cron script (et al.) that will find files
and chown them as necessary. -- I'd have to check the syntax, but I
know that find can find files owned by a specific user / UID. So I
would expect that you can invert that logic. Thus you should be able to
do a find targeted at the directory for all files /not/ owned by a
specific user, i.e. carbon.

I'd suggest spending 5-15 minutes looking at POSIX ACLs to 1) understand
them and 2) decide if such might be able to help you out. Have a cup of
your preferred morning beverage and do a bit of reading. ;-)



--
Grant. . . .
unix || die

[Rich]

Grant Taylor <gta...@tnetconsulting.net> wrote:
> On 2/26/21 3:58 AM, K Venken wrote:
>> The file is still owned by student1 and can not be edited by student2.
>> Of course, each student can use umask 0007 but this has two problems:
>> - this would apply to the other groups each student/mentor is member of
>> which is an unwanted sideeffect
>> - this can not be used with files uploaded with WinSCP for instance, it
>> seems not to be possible to indicate umask when uploading files, or
>> change protections in this case.
>

> The other /hack/ is to have a cron script (et al.) that will find files
> and chown them as necessary. -- I'd have to check the syntax, but I
> know that find can find files owned by a specific user / UID. So I
> would expect that you can invert that logic.

Yes, ! inverts:

find \! -user carbon -print0

Would find all files/directories not owned by 'carbon' in the current
dir. The OP would have to append the proper xargs invocation for the
desired outcome.

The problem here is the latency between cron invocations of the "change
owner" script during which the files will not be 'carbon' up until the
next time the script runs.

[Rich]

Grant Taylor <gta...@tnetconsulting.net> wrote:
> On 2/26/21 3:58 AM, K Venken wrote:
>> The file is still owned by student1 and can not be edited by student2.
>> Of course, each student can use umask 0007 but this has two problems:
>> - this would apply to the other groups each student/mentor is member of
>> which is an unwanted sideeffect
>> - this can not be used with files uploaded with WinSCP for instance, it
>> seems not to be possible to indicate umask when uploading files, or
>> change protections in this case.
>

> The other /hack/ is to have a cron script (et al.) that will find files
> and chown them as necessary.

Another option, that will reduce the latency of waiting for a cron job
to run, would be to use the inotify system to get event notifications
of file creation, and run the chown at that point:

Linux Filesystem Events with inotify
https://www.linuxjournal.com/content/linux-filesystem-events-inotify

[Grant Taylor]

On 2/26/21 9:20 AM, Rich wrote:
> Would find all files/directories not owned by 'carbon' in the current
> dir. The OP would have to append the proper xargs invocation for
> the desired outcome.

I would also be possible to use find's -exec option instead of xargs.

> The problem here is the latency between cron invocations of the
> "change owner" script during which the files will not be 'carbon'
> up until the next time the script runs.

Your other message aptly mentioned inotify / dnotify. There are a few
options in that category.

Depending on the OP's needs, maybe a once a minute cron job would suffice.

[Javier]

K Venken <karel....@domain.invalid> wrote:
> This brings me to the question if there is a better way to emulate this
> effect?

Set up a specific user 'team' for shared files between users.
For accessing other users files in the same computer you can use sshfs.
With sshfs you don't need to do anything with groups and permissions.

For mounting mounting user Team's $HOME and making it accesible for Alice.

mounted_team_dir=/home/alice/mounted_sshfs_team
mkdir -p ${mounted_team_dir}
sshfs team@localhost:/home/team ${mounted_team_dir}
ls -l ${mounted_team_dir}

To not being asked for password everytime you ssh into Team's account:

ssh team@localhost 'mkdir -p .ssh; cat >>.ssh/authorized_keys;'\
'chmod 700 .ssh;chmod 600 .ssh/authorized_keys' < ${HOME}/.ssh/id_rsa.pub

PS1: I wouldn't use sshfs it for users in different computers.
It's not good with network latency, and when there is a network
disconnection it behaves horribly and keeps waiting forever. For
different machines use NFS, but that is more work to set it up.

PS2: security-wise this is not perfect, since the students would have
access to 'team' account and would be able to disable access to the
other people.

[Lew Pitcher]

On Fri, 26 Feb 2021 11:58:00 +0100, K Venken wrote:

> I have a number of students which need to cooperate in (different)
> groups. The obvious solution would be to setup a group for each as
> shared folder. Each shared group has a dedicated user with the same name
> (for convenience).

[...]

> Then the idea was to assign and protect all files in this folder to this
> shared group

[...]

> And finally add the different members to this group:

[...]

> Unfortunately, this does not work. If student1 creates a file in
> /export/shared/groups/carbon, it will be owned by group carbon but not
> by user carbon.

[...]

> The file is still owned by student1 and can not be edited by student2.

[...]

> The intention is to have all files in this directory also to be owned by
> this dedicated (carbon) user.

[snip]

"Also owned"? As a file can have only /one/ UID owner, how do you see
files in this shared directory being "owned"?

ISTM that you haven't clearly explained what problem you are trying to
solve, and are looking in the wrong direction for a solution.

Here's what I gleaned from your post; correct me if I am wrong...

You want to facilitate a team effort, in which all members of the team
have the ability to alter selected files, each of which may be created by
other team members.

You have not found a simple method to enable this sort of collaboration.
You have looked at placing all the members of the team in a single unix
group, but have found that files created by individual members still
carry permission bits that may exclude other group members from altering
the file.

You looked at restricting the unix group to saving files in a specially-
crafted rendezvous directory, which is marked SETGID, and follows BSD
semantics. But, this only ensures that all files written to this
directory are owned by the GROUP owner. It does not enforce your
preferred set of group permissions to permit other members of the group
to alter the contents of the file.

You are casting about for a way to ensure that no file in the directory
has permission settings that would restrict members of the group (other
than the file owner) from altering the file.

You ask if there is some way that you can ensure that all files in this
rendezvous directory are owned by a specific user; I have reason to
believe that, should this be possible, it would only make your situation
worst. Files have only /one/ UID owner; to make it such that all files in
your rendezvous directory are /owned/ by the directory's UID owner is to
ensure that none of those files are owned by their creators. Which makes
the USER permissions moot; none in the group "owns" the file, so USER
permissions don't apply.

So, group members will have to depend on each file's GROUP permissions to
permit their access and alteration; exactly the situation you are in
without having all files "owned" by the directory's UID owner.

Of course, there is a workaround for your "directory UID owns all the
files" dilemma: each user in the group must "su" to become the "directory
UID" user. And, this solution has it's own complications and issues.

Of course, you could just "bite the bullet", leave things as they are now
(all team members belong to same unix group, directory owned by group
with BSD setgid semantics), and dictate that each team member must ensure
that any file to be "shared" has GROUP WRITE access.

FWIW, you might look at setting the "sticky bit" on your rendezvous
directory: that would ensure that "a file in that directory can be
renamed or deleted ONLY by the owner of the file, by the owner of the
directory, or by a privileged process".

HTH
--
Lew Pitcher
"In Skills, We Trust"

[K. Venken]

Thanks for your great explanation. It 's much better then mine and
exactly poses what I meant (you are doing a much better job then I did).

My (incorrect) thought was that if a file is owned by a "common" user
things would become easier.

> You want to facilitate a team effort, in which all members of the team
> have the ability to alter selected files, each of which may be created by
> other team members.

That's correct, that's the purpose. Slight addition, it would be useful
if the group is not accessible by members not of the group, but this is
handled by the world (other) permissions.

> You have not found a simple method to enable this sort of collaboration.
> You have looked at placing all the members of the team in a single unix
> group, but have found that files created by individual members still
> carry permission bits that may exclude other group members from altering
> the file.

Yes, indeed. The problem is that different users can't edit each other
files, which should be facilitated. Looking back at it, you are correct
in your assessment below that trying to make the files owned by a
predefined (different) user does not help. This was an incorrect
projection of the NFS option of all_squash. (I'll come back on that)

> You looked at restricting the unix group to saving files in a specially-
> crafted rendezvous directory, which is marked SETGID, and follows BSD
> semantics. But, this only ensures that all files written to this
> directory are owned by the GROUP owner. It does not enforce your
> preferred set of group permissions to permit other members of the group
> to alter the contents of the file.

Correct

> You are casting about for a way to ensure that no file in the directory
> has permission settings that would restrict members of the group (other
> than the file owner) from altering the file.

I am a little confused. Other group members must be able to use each
others files as if it were their own files. I might have not been clear
on this though,... For instance a mentor/supervisor might be setting up
an example or exercise which the different group members (together)
could solve.

> You ask if there is some way that you can ensure that all files in this
> rendezvous directory are owned by a specific user; I have reason to
> believe that, should this be possible, it would only make your situation
> worst. Files have only /one/ UID owner; to make it such that all files in
> your rendezvous directory are /owned/ by the directory's UID owner is to
> ensure that none of those files are owned by their creators. Which makes
> the USER permissions moot; none in the group "owns" the file, so USER
> permissions don't apply.

I didn't see it as being worse. In the end, if ownership has been
removed by making it all the same user, it would appear as being
"common" group files with no ownership. Every member of the group now
owns (as far as this still makes sense) the file. This could be achieve
with the NFS option mentioned before. But the conclusion is correct.

> So, group members will have to depend on each file's GROUP permissions to
> permit their access and alteration; exactly the situation you are in
> without having all files "owned" by the directory's UID owner.

Indeed

> Of course, there is a workaround for your "directory UID owns all the
> files" dilemma: each user in the group must "su" to become the "directory
> UID" user. And, this solution has it's own complications and issues.
>
> Of course, you could just "bite the bullet", leave things as they are now
> (all team members belong to same unix group, directory owned by group
> with BSD setgid semantics), and dictate that each team member must ensure
> that any file to be "shared" has GROUP WRITE access.

In retrospect, that would still be a solution "ensure that any file to
be shared has GROUP WRITE access" regardless of who owns he file. The
trick is to ensure this. In another post, Rich mentioned to use inotify
for this. As soon as a file is created, modify the group permissions to
be g+w. Together with the setgid that might be a solution.

> FWIW, you might look at setting the "sticky bit" on your rendezvous
> directory: that would ensure that "a file in that directory can be
> renamed or deleted ONLY by the owner of the file, by the owner of the
> directory, or by a privileged process".

I don't think it fully solves the problem. The point is to give all
members of the group full access to each other files in this group
directory.

[K. Venken]

I didn't want to go there. find might choke on the size of the
filesystem (20 TB, ~ 50 users) Of course, shared groups are much
smaller, so it could work, if limited to this.

[K. Venken]

I have written a little script which would exactly do that.

#!/usr/bin/tclsh
set fh [open "|inotifywait -r -m -e create carbon" r]
while { [gets $fh data]>=0} {
set fn [join [lrange $data 2 end]]
puts "chmod g+w \"$fn\""
exec chmod g+w [file join carbon $fn]
}
close $fh

It doesn't handle all fault conditions, it doesn't handle subdirectories
properly, but in principle it sets group permissions to w on each file
or directory created. Together with the setgid, it might be a solution.

[Grant Taylor]

On 2/26/21 2:00 PM, K. Venken wrote:
> I didn't want to go there. find might choke on the size of the
> filesystem (20 TB, ~ 50 users) Of course, shared groups are much
> smaller, so it could work, if limited to this.

Hence targeted find. ;-)

At that size I'd wonder if there might be something like FS transaction
logs (a la. ZFS send) that you might be able to leverage.

[Rich]

Grant Taylor <gta...@tnetconsulting.net> wrote:
> On 2/26/21 9:20 AM, Rich wrote:
>> Would find all files/directories not owned by 'carbon' in the current
>> dir. The OP would have to append the proper xargs invocation for
>> the desired outcome.
>
> I would also be possible to use find's -exec option instead of xargs.

Yes, although unless one is careful there, one can get 1 fork per
filename from find's option, whereas xargs always fills out maximum
size arguments lists, reducing the number of forks, and increasing the
net performance, dramatically.

>> The problem here is the latency between cron invocations of the
>> "change owner" script during which the files will not be 'carbon' up
>> until the next time the script runs.
>
> Your other message aptly mentioned inotify / dnotify. There are a few
> options in that category.

Yes, if the OP needs "instant" (or nearly so) changes, then inotify is
the way to go.

> Depending on the OP's needs, maybe a once a minute cron job would
> suffice.

Certianlty that is possible. We don't know this part of the OP's
needs, but it is possible that once a minute or once an hour or
something could be "good enough" for the OP.

[Rich]

The find command never chokes, no matter the size of the filesystem it
is asked to traverse. But the larger the filesystem (as in number of
files and number of directories, not block usage) the longer it will
take find to traverse the hierarchy.

[Rich]

You probably will want to use the --csv output option, and the Tcllib
csv module to parse the csv output, as your lrange followed by file
join will convert filenames with embedded spaces into a false directory
hierarchy.

Also, no need to exec a chmod, Tcl's file attributes is fully capable
of setting the group writable bit. Although if you simply want the
group writable bit to be on, setting an appropriate umask for these
users would have it set to on by default without any other tools
needed.

[Rich]

K. Venken <karel....@domain.invalid> wrote:

> Lew Pitcher wrote:
> My (incorrect) thought was that if a file is owned by a "common" user
> things would become easier.

As each "user" in the Unix security setup is distinct, there can be no
"common" user under Unix style permissions. The only way a "common"
user could work is if all students logged in under the same username
and a shared password.

>> You are casting about for a way to ensure that no file in the directory
>> has permission settings that would restrict members of the group (other
>> than the file owner) from altering the file.
>
> I am a little confused. Other group members must be able to use each
> others files as if it were their own files. I might have not been clear
> on this though,... For instance a mentor/supervisor might be setting up
> an example or exercise which the different group members (together)
> could solve.

The 'task' you have outlined is what the 'group' membership under Unix
style permissions was meant to solve. Allow different users to
share/collaborate/edit/etc. some subset of files within the filesystem.

>> Of course, you could just "bite the bullet", leave things as they are now
>> (all team members belong to same unix group, directory owned by group
>> with BSD setgid semantics), and dictate that each team member must ensure
>> that any file to be "shared" has GROUP WRITE access.
>
> In retrospect, that would still be a solution "ensure that any file to
> be shared has GROUP WRITE access" regardless of who owns he file. The
> trick is to ensure this.

The standard Unix way to ensure this is an appropriate umask setting
for this group of sharing users that leaves the group write access bit
set to "true". That is the purpose of the umask setting, to define
what set of permission bits the system should apply to a newly created
file or directory in the absense of the creator explicitly supplying a
permission bit pattern for the new file.

[Aragorn]

On 27.02.2021 at 02:28, Rich scribbled:

> K. Venken <karel....@domain.invalid> wrote:
> > Lew Pitcher wrote:
> > My (incorrect) thought was that if a file is owned by a "common"
> > user things would become easier.
>
> As each "user" in the Unix security setup is distinct, there can be
> no "common" user under Unix style permissions. The only way a
> "common" user could work is if all students logged in under the same
> username and a shared password.

Actually, that's not /entirely/ correct. It is perfectly possible to
have multiple user accounts, each with their own login and password, who
all share the same UID.

But of course, they can then also forget about any privacy on account
of the permissions on their respective home directories, given that
it's the UID that determines the permissions on anything in their $HOME,
not the login name. From the vantage of the operating system, they are
all the same person, but with multiple login aliases.

The above principle was often used on multiuser machines running
certain versions of SCO Unix. For instance, the account with UID 0 had
three separate logins, each with their own password:

- root;
- admin; and
- backup.

When logging in as "root", you got a normal command-line user interface
— a Bourne shell. When logging in as "backup", you got a menu-driven
user interface specifically for making backups. I forgot what the
"admin" login got, but I suspect it may have been some other menu-driven
interface.


--
With respect,
= Aragorn =

[K. Venken]

Once a minute would probably do. A typical use case, a student uploads
an additional data file she prepared and another student uses it to
compile a calculation setup. Manual interactions can live with a minute
delay in between.

But I would rather be on the safe side, and it seems to me that find is
less efficient/fast if the shared folder is rather large (or on a rather
overly used filesystem). inotify tells you you have to do something on a
dedicated file. find searches the filesystem and needs to go over all
the files in this directory. On the other side, if a student uploads
several files, find can handle them all at once. It depends.

Immediate is not a strict requirement, but nice to have if it can
prevent unexpected errors. Eg. a student trying to edit a just uploaded
file within a minute...

[K. Venken]

Well yes, it 'll take time if there are a lot of files in the directory.
That's what I meant ;)

[K. Venken]

FWW, you can have filenames with , as well. So I need to do some more
work anyway to handle the special characters.

> Also, no need to exec a chmod, Tcl's file attributes is fully capable
> of setting the group writable bit. Although if you simply want the
> group writable bit to be on, setting an appropriate umask for these
> users would have it set to on by default without any other tools
> needed.

There are a few other things which I noted to be handled. The chmod
could error aborting the program. So there is at least a catch needed as
well. I prefer Tcl as soon as I need to do some text processing, but a
bash version would do as well.

[K. Venken]

A typical example, I started inotifywait as root in the / directory. In
another console, I created a few files and directories with spaces and
commas. This is the result:

root@matthua:/# inotifywait -r -m -e create
/export/home/shared/groups/carbon/
Setting up watches. Beware: since -r was given, this may take a while!
Watches established.
/export/home/shared/groups/carbon/ CREATE,ISDIR hello
/export/home/shared/groups/carbon/hello/ CREATE world
/export/home/shared/groups/carbon/ CREATE,ISDIR more to see
/export/home/shared/groups/carbon/more to see/ CREATE here
/export/home/shared/groups/carbon/more to see/ CREATE here,as,well

I could look for CREATE unless someone creates a file with this name, etc...

[Henrik Carlqvist]

On Fri, 26 Feb 2021 11:58:00 +0100, K Venken wrote:

> Another complication is that the home folders and thus also the shared
> folders are exported through NFS to a set of different hosts as well
> (around 30). Creating new shares or filesystems would require to modify
> the NFS /etc/exports in the server and the NFS imports (/etc/fstab) in
> all the clients as well.

I hope the the inotify solution will work for you as the umask solution
is not enough for your needs.

As you already are using NIS to distribute user accounts in your network
I would suggest that you also use NIS to distribute automount maps. By
putting all your NFS directories in automount maps provided by NIS
instead of the local /etc/fstab you will only need to edit in one place
if a new server share is added to your network.

My /etc/auto.master ends with:

+auto.master.slack142

Where auto.master.slack142 is a NIS map containing links to other NIS
maps. Different versions of Slackware might mount different directories
for tools but all machines share the same home and project directories.

The above line in auto.master and "chmod u+x /etc/rc.d/rc.autofs" are the
only modifications needed to the NFS clients to all mount the same NFS
directories. The only work left is to configure some more maps on your
NIS server(s). If you are really lazy, you do not even need to configure
any maps on your NIS server, by default automount maps any NFS server
below /net. So you can peek at /net/my_nfs_server/the_export_name.
However, I prefer to write my own automount maps to get better mount
points and to add "intr" to the NFS mounts.

regards Henrik

[Rich]

K. Venken <karel....@domain.invalid> wrote:
> Rich wrote:

>> K. Venken <karel....@domain.invalid> wrote:
>>> I have written a little script which would exactly do that.
>>>
>>> #!/usr/bin/tclsh
>>> set fh [open "|inotifywait -r -m -e create carbon" r]
>>> while { [gets $fh data]>=0} {
>>> set fn [join [lrange $data 2 end]]
>>> puts "chmod g+w \"$fn\""
>>> exec chmod g+w [file join carbon $fn]
>>> }
>>> close $fh
>>>
>>> It doesn't handle all fault conditions, it doesn't handle subdirectories
>>> properly, but in principle it sets group permissions to w on each file
>>> or directory created. Together with the setgid, it might be a solution.
>>
>> You probably will want to use the --csv output option, and the Tcllib
>> csv module to parse the csv output, as your lrange followed by file
>> join will convert filenames with embedded spaces into a false directory
>> hierarchy.
>
> FWW, you can have filenames with , as well. So I need to do some more
> work anyway to handle the special characters.

The CSV format already handles comma inside fields, so you don't have
to do anything special. Note:

$ inotifywait --csv -r -m -e create x

Setting up watches. Beware: since -r was given, this may take a while!
Watches established.

x/,CREATE,"test,test"

Note the quotes (") around the filename, those are CSV's method of
indicating that a comma inside the field is part of the data, not part
of the CSV file.

And Tcl's csv module will properly parse that CSV output as well:

$ rlwrap tclsh
% package require csv
0.8
% set in {x/,CREATE,"test,test"}
x/,CREATE,"test,test"
% csv::split $in
x/ CREATE test,test
% lindex [csv::split $in] 2
test,test

With the result that you do not have to write *any* code to handle

special characters.

>> Also, no need to exec a chmod, Tcl's file attributes is fully capable
>> of setting the group writable bit. Although if you simply want the
>> group writable bit to be on, setting an appropriate umask for these
>> users would have it set to on by default without any other tools
>> needed.
>
> There are a few other things which I noted to be handled. The chmod
> could error aborting the program. So there is at least a catch needed as
> well.

Yes, I left off 'robustness' comments.

> I prefer Tcl as soon as I need to do some text processing, but a bash
> version would do as well.

In this instance, given that inotifywait has csv output and Tcl has a
csv parsing module, which together will free you from any special
character handling, you'd be better off remaining with Tcl, you'll more
easily avoid the quoting minefield of bash scripts that way.

[Rich]

K. Venken <karel....@domain.invalid> wrote:
> Rich wrote:
>> You probably will want to use the --csv output option, and the
>> Tcllib csv module to parse the csv output, as your lrange followed
>> by file join will convert filenames with embedded spaces into a
>> false directory hierarchy.
>
> A typical example, I started inotifywait as root in the / directory. In
> another console, I created a few files and directories with spaces and
> commas. This is the result:
>
> root@matthua:/# inotifywait -r -m -e create
> /export/home/shared/groups/carbon/
> Setting up watches. Beware: since -r was given, this may take a while!
> Watches established.
> /export/home/shared/groups/carbon/ CREATE,ISDIR hello
> /export/home/shared/groups/carbon/hello/ CREATE world
> /export/home/shared/groups/carbon/ CREATE,ISDIR more to see
> /export/home/shared/groups/carbon/more to see/ CREATE here
> /export/home/shared/groups/carbon/more to see/ CREATE here,as,well
>
> I could look for CREATE unless someone creates a file with this name, etc...

Don't go down the rabbit hole of trying to code a way around the
special characters and space characters problem.

Just ask inotifywait to output csv, use Tcllib's csv parsing module,
and you will be able to handle any filenames the students create with
no extra effort on your part.

[Lew Pitcher]

I would say, to "unmasked".

> That is the purpose of the umask setting, to define what set
> of permission bits the system should apply to a newly created file or
> directory

Not really. umask /removes/ permissions, it does not /add/ them. For a
newly created file to have GROUP WRITE permission, the creating process
must create the file with the GROUP WRITE permission bit set /and/ the
user's umask (at the time of file creation) must not mask that permission
bit.

> in the absense of the creator explicitly supplying a permission bit
> pattern for the new file.

The creator /always/ explicitly supplies the permission bit pattern for a
newly created file. It's just that the "creator" isn't the user; it is
the process that creat(2)s the file.

[Grant Taylor]

On 2/27/21 1:36 AM, K. Venken wrote:
> Immediate is not a strict requirement, but nice to have if it can
> prevent unexpected errors. Eg. a student trying to edit a just uploaded
> file within a minute...

My personal opinion is that people probably won't run into the less than
one minute catch very often. If / when they do, go get something to
drink. It will be fixed by the time that you get back.

[Grant Taylor]

On 2/26/21 3:58 AM, K Venken wrote:

> NIS is used to distribute the accounts

I missed this.

I've not heard of NIS being used in production in quite a while.

Thank you for the data point.

Can I ask if it's NIS or NIS+?

[K. Venken]

Thanks for the tips. I wasn't to fond on jumping in the rabbit hole
anyway. The tcl library and csv option is the way to go. I'll use that.

[K. Venken]

I use autofs on my laptop, so I was aware of the /net path. On the hosts
the shared access is needed for, it is however not activated. Also, Nis
does not distribute the auto maps on these systems. I'll take it into
consideration.

[K. Venken]

Grant Taylor wrote:
> On 2/26/21 3:58 AM, K Venken wrote:
>> NIS is used to distribute the accounts
>
> I missed this.
>
> I've not heard of NIS being used in production in quite a while.
>
> Thank you for the data point.
>
> Can I ask if it's NIS or NIS+?

I can't tell... This sounds not so smart, but lets have some background
of the system.

The hosts on which the shared folder is needed are part of a (small)
cluster. The master splits the world into the external world for the
students laptops (linux, windows, apple) and the internal network for
the calculation nodes. The master serves the files and accounts (NIS/NFS
server) to the internal nodes (NIS/NFS clients). This way it creates an
internal cluster wide consistent view on files and accounts. NIS is only
exposed internally. The calculation nodes are always on line. The
students upload their files (SCP or NFS) and can then start calculations
(using SLURM)

I used the default configuration in Slackware, which I thought were NIS
and NFS-3, but I may be wrong.

[K. Venken]

...

Thanks for all the help, comments and suggestions. They were very
useful. Using inotify seemed to be the solution. To have a resume for
setting up a shared folder, may it be helpful for someone else:


1) Create a group, eg carbon
addgroup -g 1506 carbon

2) Create a folder for this group, owned by group carbon
mkdir /export/home/shared/groups/carbon
chown root:carbon /export/home/shared/groups/carbon

3) Set the proper permissions
chmod 770 /export/home/shared/groups/carbon
chmod g+s /export/home/shared/groups/carbon

4) Add the users to this group
usermod -aG carbon user1
usermod -aG carbon user2
...

5) Run the NIS update in /var/yp
cd /var/yp; make

6) Start the inotify script (see below) which enforces group write
permission upon creating a file or directory. The script I use needs the
tcllib (see slackbuilds).

7) The script should be started when booting (to add in the
/etc/rc.d/rc.local)


A few comments about this script.

The script insists on an absolute path. This may not be required, but I
have not tested it with relative paths. The path should also be created
in advance, which may also not be really required.

Setting the file permissions can go wrong. The catch would detect this
and not crash the script. The permissions set are both read and write as
someone may have a more restrictive umask.


I have tested a few different situations, a file uploaded with SCP, a
file created on an NFS client,... with different users.



#!/usr/bin/tclsh

package require csv

set dirn ""
if {[llength $argv]==1} {
set dirn [lindex $argv 0]
}

if {([string compare [string index $dirn 0] [file separator]]!=0)} {
puts "usage: [file rootname [info script]] absolutepath"
exit 0
}

if {[file exists $dirn]==0} {
puts "$dirn does not exist"
exit 1
}

set fh [open "|inotifywait -r -m -c -e create $dirn" r]
fconfigure $fh -buffering line

while { [gets $fh data]>=0} {

if { [catch {
set items [csv::split $data]
set fn [file join [lindex $items 0] [lindex $items 2]]
file attributes $fn -permissions g+rw
} ferr] } {
puts stderr "$data\n$ferr"
}
}
close $fh

[Henrik Carlqvist]

On Sun, 28 Feb 2021 13:55:47 +0100, K. Venken wrote:
> I used the default configuration in Slackware, which I thought were NIS
> and NFS-3,

Yes, that is right.

regards Henrik

[Grant Taylor]

On 2/28/21 5:55 AM, K. Venken wrote:
> I can't tell... This sounds not so smart, but lets have some background
> of the system.

It sounds like you're using NIS(-) in a micro environment with tight
controls. -- Works for me.

> I used the default configuration in Slackware, which I thought were NIS
> and NFS-3, but I may be wrong.

Thank you for clarifying.