O_CLOEXEC

[Lewin Bormann]

Hi,

I just read open(2) and was confused about the description of the O_CLOEXEC
flag:

O_CLOEXEC (Since Linux 2.6.23)
Enable the close-on-exec flag for the new file descriptor. Specifying this
flag permits a program to avoid additional fcntl(2) F_SETFD operations to set
the FD_CLOEXEC flag. Additionally, use of this flag is essential in some
multithreaded programs since using a separate fcntl(2) F_SETFD operation to set
the FD_CLOEXEC flag does not suffice to avoid race conditions where one thread
opens a file descriptor at the same time as another thread does a fork(2) plus
execve(2).

I find it weird that it's explained with the fact that you don't have to set it
later with fcntl(), and that it's 'essential in /some/ multithreaded programs'
etc..
My question: Because I did not find something about it anywhere, I'm just
guessing that the flag has the effect of closing this fd when using a syscall
from the exec() family? -> 'close-on-exec'. Is that right?

Sorry if my question seems stupid, but I'm really confused about it :|

Thanks,
Lewin

[Richard Kettlewell]

'man fcntl' documents the effect of the close-on-exec flag:

If the FD_CLOEXEC bit is 0, the file descriptor will remain open
across an execve(2), otherwise it will be closed.

--
http://www.greenend.org.uk/rjk/

[Xavier Roche]

Le 07/01/2012 13:50, Lewin Bormann a écrit :
> My question: Because I did not find something about it anywhere, I'm just
> guessing that the flag has the effect of closing this fd when using a syscall
> from the exec() family? -> 'close-on-exec'. Is that right?

Yes.

More precisely, when you spawn a new process, you typically fork() and
exec*(). The fork() call preserves (dup) all file descriptors, and
exec*() keeps file descriptors opened for the new application (for
purely historical reasons) unless they have the FD_CLOEXEC (added later
in POSIX) flag.

The default, fixed behavior (ie. you can not change this insane^W
default mode), is to open files (open, fopen) with the FD_CLOEXEC flag
cleared, that is, to KEEP all file descriptors opened when executing
another application, which is generally not desirable (except for
stdin/out/err for obvious reasons).

The O_CLOEXEC feature is rather "new" (on Linux, it was introduced on
2.6.23 if I am not mistaken - using it before will make open to return
an error) but allows to atomically open a file with the FD_CLOEXEC flag
set. Setting this flag just after the open() is prone to race conditions
(ie. if another thread is doing fork/exec just after an open, the
corresponding flag will remain open), so the O_CLOEXEC mask is the only
clean solution to do things correctly. You do not need to fcntl() when
using it, of course.

[Rainer Weikusat]

Xavier Roche <xro...@free.fr.NOSPAM.invalid> writes:
> Le 07/01/2012 13:50, Lewin Bormann a écrit :
>> My question: Because I did not find something about it anywhere, I'm just
>> guessing that the flag has the effect of closing this fd when using a syscall
>> from the exec() family? -> 'close-on-exec'. Is that right?
>
> Yes.
>
> More precisely, when you spawn a new process, you typically fork() and
> exec*(). The fork() call preserves (dup) all file descriptors, and
> exec*() keeps file descriptors opened for the new application (for
> purely historical reasons)

This is, of course, code language for "that how it was designed and I
disagree STRONGLY with that".

[...]

> The default, fixed behavior (ie. you can not change this insane^W
> default mode), is to open files (open, fopen) with the FD_CLOEXEC flag
> cleared, that is, to KEEP all file descriptors opened when executing
> another application, which is generally not desirable (except for
> stdin/out/err for obvious reasons).

There are two possible default behaviours here, namely, 'file
descriptors are inherited' and 'file descriptors are not
inherited'. In a real-world application, each mode will be desired for
some file descriptors and undesired for others. In particular,
'stdin/out/err' have no special semantics beyond 'usual convention'.

[William Ahern]

Rainer Weikusat <rwei...@mssgmbh.com> wrote:
<snip>

> There are two possible default behaviours here, namely, 'file
> descriptors are inherited' and 'file descriptors are not
> inherited'. In a real-world application, each mode will be desired for
> some file descriptors and undesired for others. In particular,
> 'stdin/out/err' have no special semantics beyond 'usual convention'.

Except the latter behavior makes far more sense. The simple heuristic which
says that inherited descriptors remain inheritable while new descriptors are
not inheritable still solves the stdin/stdout/stderr case rather cleanly.
What we really care about is _intermediate_, dumb processes inheriting a
sane policy.

The processes which spawn the dumb ones still have the bothersome task of
using F_SETFD for all their other descriptors. Once we consider the
requirements of antecedant processes, and not just the children, the latter
case scores much better in almost all scenarios--if not all scenarios.

[Rainer Weikusat]

William Ahern <wil...@wilbur.25thandClement.com> writes:
> Rainer Weikusat <rwei...@mssgmbh.com> wrote:
> <snip>
>> There are two possible default behaviours here, namely, 'file
>> descriptors are inherited' and 'file descriptors are not
>> inherited'. In a real-world application, each mode will be desired for
>> some file descriptors and undesired for others. In particular,
>> 'stdin/out/err' have no special semantics beyond 'usual convention'.
>
> Except the latter behavior makes far more sense.

This statement doesn't make sense: The desiscion for one or the other
is completely arbitrary and at best, one could claim that 'the latter
is far more useful' and this, in turn, depends heavily on the affected
code.