Handling the posix_spawn() file descriptor hell
Xavier Roche
I have a problem which might be considered as rather obvious for many of
you, but I just can not find a clean way to handle it.
When spawning a new process in a parent process, there are situations
where you do not want spawned processes to inherit parent's file
descriptors (for example, when the parent already opened a very large
number of file descriptors)
Is there a clean way to acheive this, with posix_spawn(), in a
multithreaded environment ?
The "obvious" solution could be to create all possible file descriptors
with FD_CLOEXEC, but when dealing with external libraries, you do not
always have the control of every single file descriptor created
(fopen(), socket(), ..), unfortunately. And there is no way to make
"FD_CLOEXEC" the default behaviour within a process (1)
With fork()/exec(), a (rather dirty, but working) solution is to close
in the fork()'ed child all file descriptors between 3 and 4096 (or 3 to
sysconf(_SC_OPEN_MAX) with most environments) with close(), after
dup2()'ing stdin/stdout/stderr, and then go on with the exec() thing.
The posix_spawn() function offers a "similar" feature, through a
"posix_spawn_file_actions_t" opaque structure, which let you
close/open/dup2 all file descriptors you want.
But there's a little (rather) annoying issue with this API:
- there is no way to tell "please close every single remaining file
descriptor"
- closing a file descriptor which was not already opened just breaks
your process (2)
- you can not be nasty and, in the parent, scan every single file
descriptor with fcntl(fd, F_GETFD, 0) and ask to close the ones not
returning FD_CLOEXEC _and_ not returning -1 (not opened), because in a
multithreaded environment a file descriptor might be created under your
feet (leading a file descriptor left opened) or, worse, a file
descriptor might be closed under your feet (leading to break your
process with exit code 127 -- see (2) again)
- in a similar way, forcing every single file descriptor to have the
FD_CLOEXEC on the parent is not only dirty, it is also hopeless for the
same reasons
I must have missed something really obvious, some very trivial feature
documented in the standard, but I just can not get it, and re-reading
endlessly the detailed spec is definitely not helpful.
Anyone ?
(my apologies for the poor syntax and grammar)
(1)
"The FD_CLOEXEC file descriptor flag associated with the new file
descriptor _shall_ be cleared unless the O_CLOEXEC flag is set in oflag."
<http://www.opengroup.org/onlinepubs/9699919799/functions/open.html>
(2)
"If this error occurs after the calling process successfully returns
from the posix_spawn() or posix_spawnp() function, the child process may
exit with exit status 127."
<http://www.opengroup.org/onlinepubs/009695399/functions/posix_spawn.html>
Xavier Roche
Anyone ?
(2)
"(..) an error value shall be returned as described by close(), dup2(),
and open(), respectively (or, if the error occurs after the calling
process successfully returns, the child process _shall_ exit with exit
status 127"
<http://www.opengroup.org/onlinepubs/009695399/functions/posix_spawn.html>
David Schwartz
wrote:
> When spawning a new process in a parent process, there are situations
> where you do not want spawned processes to inherit parent's file
> descriptors (for example, when the parent already opened a very large
> number of file descriptors)
>
> Is there a clean way to acheive this, with posix_spawn(), in a
> multithreaded environment ?
Yes, set close on exec for each file descriptor you don't want
inherited.
> The "obvious" solution could be to create all possible file descriptors
> with FD_CLOEXEC, but when dealing with external libraries, you do not
> always have the control of every single file descriptor created
> (fopen(), socket(), ..), unfortunately. And there is no way to make
> "FD_CLOEXEC" the default behaviour within a process (1)
No, not "all possible file descriptors" only those you don't want
inherited. External libraries should set close on exec if they want
the descriptors closed on an exec. If they don't set it, then they are
either buggy or don't want those descriptors closed.
> With fork()/exec(), a (rather dirty, but working) solution is to close
> in the fork()'ed child all file descriptors between 3 and 4096 (or 3 to
> sysconf(_SC_OPEN_MAX) with most environments) with close(), after
> dup2()'ing stdin/stdout/stderr, and then go on with the exec() thing.
That's very dirty, since there's no way to know whether those
descriptors should be closed or not. So that will also close
descriptors that shouldn't be closed. If only there was some simple
way to set a flag on a descriptor that indicates whether or not it
should be closed on an exec and close only those descriptors that are
flagged. Oh, wait, there is. It's called 'close on exec'.
> The posix_spawn() function offers a "similar" feature, through a
> "posix_spawn_file_actions_t" opaque structure, which let you
> close/open/dup2 all file descriptors you want.
>
> But there's a little (rather) annoying issue with this API:
> - there is no way to tell "please close every single remaining file
> descriptor"
Of course not. That would be absurd. That would close even descriptors
that you had no way to know *should* be closed and break inter-
operation with properly-coded libraries.
> - closing a file descriptor which was not already opened just breaks
> your process (2)
> - you can not be nasty and, in the parent, scan every single file
> descriptor with fcntl(fd, F_GETFD, 0) and ask to close the ones not
> returning FD_CLOEXEC _and_ not returning -1 (not opened), because in a
> multithreaded environment a file descriptor might be created under your
> feet (leading a file descriptor left opened) or, worse, a file
> descriptor might be closed under your feet (leading to break your
> process with exit code 127 -- see (2) again)
Right.
> - in a similar way, forcing every single file descriptor to have the
> FD_CLOEXEC on the parent is not only dirty, it is also hopeless for the
> same reasons
No, not every single descriptor. Every single descriptor that *should*
be closed on an exec. Some should be, some shouldn't be. And the
standard provides a way to say which is which. You're right, you don't
want to close all descriptors, only those that should be. And the
platform already does this for you.
> I must have missed something really obvious, some very trivial feature
> documented in the standard, but I just can not get it, and re-reading
> endlessly the detailed spec is definitely not helpful.
Set 'close on exec' for all descriptors that should be closed on an
exec. It's really that simple.
If you want to do some real defensive coding, on startup, close any
file descriptors you may have inherited but don't want.
DS
Xavier Roche
> inherited. External libraries should set close on exec if they want
> the descriptors closed on an exec. If they don't set it, then they are
Unfortunately many third-party libraries just do "fopen" or "socket"
without further setup, leading to get the default (no 'close on exec'
flag) behaviour.
> Set 'close on exec' for all descriptors that should be closed on an
> exec. It's really that simple.
> If you want to do some real defensive coding, on startup, close any
> file descriptors you may have inherited but don't want.
Humm, so there's no way to handle this cleanly with third-party
libraries and arbitrary child processes [ie. process with startup that
can not be modified] as I feared.
Jens Thoms Toerring
I am having the same problem regularly, i.e. having to use li-
braries that are buggy in David's sense in that they don't set
the close-on-exec flag on files they open - even when these
files are never meant to be left open for completely unrelated
processes invoked via fork()/exec() - mostly device files in
what I am doing, that no spawned unrelated program has any
reason to inherit. And usully I am getting no reaction at all
from the writers of those (often closed source) libraries, or
reactions that clearly show at that they don't understand that
this can be a serious problem or don't even know what the close-
on-exec flag is:-(
Since I am getting this problem mostly with a program that is
(until now) only used under Linux and I am not using threads
I use a (horrible) hack at the moment: first figure out which
function calls in that library may open files, then, each time
before calling these functions, make up a list of all open files
(by going through /proc/self/fd, that's where it gets non-por-
table), then call the function and check again which files are
open afterwards and set the close-on-exec flag on the files
that got opened in between. That works under the constraints
listed above but is clearly unsatisfactory, to say the least.
If anyone has a better idea on how to handle these kinds of
situations in a somewhat better way I also would be very
grateful to hear about it!
Regards, Jens
--
\ Jens Thoms Toerring ___ j...@toerring.de
\__________________________ http://toerring.de
Ian Collins
> Xavier Roche <xro...@free.fr.nospam.invalid> wrote:
>
>> Humm, so there's no way to handle this cleanly with third-party
>> libraries and arbitrary child processes [ie. process with startup that
>> can not be modified] as I feared.
>
> (until now) only used under Linux and I am not using threads
> I use a (horrible) hack at the moment: first figure out which
> function calls in that library may open files, then, each time
> before calling these functions, make up a list of all open files
> (by going through /proc/self/fd, that's where it gets non-por-
> table), then call the function and check again which files are
> open afterwards and set the close-on-exec flag on the files
> that got opened in between. That works under the constraints
> listed above but is clearly unsatisfactory, to say the least.
>
> If anyone has a better idea on how to handle these kinds of
> situations in a somewhat better way I also would be very
> grateful to hear about it!
Have you considered interposing your own open/socket/pipe/close
functions? You could then maintain your own list of open files or just
set close on exec where appropriate.
--
Ian Collins
Golden California Girls
Don't know about the libraries you are using, but that is the mantra of the guys
over in c.l.c They write "standard C" for mythical machines and refuse to
believe that the O/S has any place in their code, while at the same time touting
how close to machine code C is.
Likely the people who write them have no clue about *nix and write them for doze
machines. Or have no clue that their code might get called by a shell like
program or by a program that creates a deamon.
I actually think the issue is that *nix needs a process runtime flag so that all
opens that don't specify can be controlled by the flag. First call you make in
your code changes the default for open, to close on exec. FD 0, 1 and 2 will
already be open then so they won't change. The rest will be what you expect.
David Schwartz
wrote:
> I actually think the issue is that *nix needs a process runtime flag so that all
> opens that don't specify can be controlled by the flag. First call you make in
> your code changes the default for open, to close on exec. FD 0, 1 and 2 will
> already be open then so they won't change. The rest will be what you expect.
That will break perfectly working code that doesn't mess with the
close on exec flag because it doesn't want its file descriptors
closed. The problem is that we are stuck with a nonsensical default.
Making the default changeable will cause the problem to persist by
making libraries that are currently working become broken.
DS
Jens Thoms Toerring
No, I actually haven't - perhaps I am a bit too awed too mess
around with such basic functions;-) Thanks for pointing out that
possibility, I'll look if and how I can use this proposition!
Golden California Girls
It would simply swap which libraries are broken with a different set of
libraries that are broken and only if the caller changed the default.
I can see a program that may need both ways. On some calls to exec, close the
library files. On other calls to exec, leave the library files open. It is the
caller that knows this, not the library.
David Schwartz
wrote:
> I can see a program that may need both ways. On some calls to exec, close the
> library files. On other calls to exec, leave the library files open. It is the
> caller that knows this, not the library.
No, it would be the library. Only the library knows why it opened the
files. The caller, in principle, has no idea what the library intends
to do with the file.
One could imagine, for the sake of argument, a library that was a thin
wrapper for opening a file of some kind. In this case, it might well
be the caller that knows whether the descriptor should be closed or
not. In this case, either the caller would have to tell the library
how to set the close-on-exec flag or it would be the caller's
responsibility to set the flag correctly after the wrapper returns. If
neither has been done, we are again back to a broken library API or
broken library implementation.
DS
Golden California Girls
> On Feb 16, 5:59 pm, Golden California Girls <gldncag...@aol.com.mil>
> wrote:
>
>> I can see a program that may need both ways. On some calls to exec, close the
>> library files. On other calls to exec, leave the library files open. It is the
>> caller that knows this, not the library.
>
> No, it would be the library. Only the library knows why it opened the
> files. The caller, in principle, has no idea what the library intends
> to do with the file.
The caller may know that the use of the library is over. Translation, the
library will never be called again.
The caller may know or suspect that the library will be called again, but that
any files the library opened point to data that would be random garbage to the
new program or worse overwrite important data to the original program.
The caller may also know the library will be called again and the files will be
relevant.
Only the caller can know this. The library can't know that it will ever be
called again. To assume so is an error in the implementation.
> One could imagine, for the sake of argument, a library that was a thin
> wrapper for opening a file of some kind. In this case, it might well
> be the caller that knows whether the descriptor should be closed or
> not. In this case, either the caller would have to tell the library
> how to set the close-on-exec flag or it would be the caller's
> responsibility to set the flag correctly after the wrapper returns. If
> neither has been done, we are again back to a broken library API or
> broken library implementation.
Yes it does come down to a broken API, one that doesn't have a close up shop call.
Rather obvious would be calls as you describe above, but in those cases the fd
is being passed out of the library. If the library opens some pipes so it can
perhaps talk to other instances of the library and this is done internally that
is the bad situation. Can you imagine the problems when the caller forks and
exec's a shell and this stuff propagates down the line?!
Hopefully no one writes a library this way, but we also know human nature.
Xavier Roche
> I am having the same problem regularly, i.e. having to use li-
> braries that are buggy in David's sense in that they don't set
Most third-party libraries I'm using are in this case, unfortunately.
And I suppose that most programmers are just unaware of the problem.
Is it me or there is a last "minor" issue regarding standard C library
functions ?
You can not "fopen"
(<http://www.opengroup.org/onlinepubs/009695399/functions/fopen.html>)
atomically a file with FD_CLOEXEC set. You can set this flag (fcntl)
after calling fopen() using fcntl(fileno(..)), but not in one shot.
It means that you will one day or another end up with a race condition
if another thread creates a process just at the wrong timing.
This can have issues more than cosmetic ones (such as pipes not
correctly detecting the other end closing, and so on)
David Schwartz
wrote:
> > No, it would be the library. Only the library knows why it opened the
> > files. The caller, in principle, has no idea what the library intends
> > to do with the file.
> The caller may know that the use of the library is over. Translation, the
> library will never be called again.
So what? Just because he'll never call the library again it does not
follow that the library is done. The library may be in the process of
completing some operation and still need to intercept a signal or
something.
> The caller may know or suspect that the library will be called again, but that
> any files the library opened point to data that would be random garbage to the
> new program or worse overwrite important data to the original program.
That's ridiculous. You're saying the caller would close the
descriptors because the library is too broken to set close-on-exec
even though it will overwrite important data if close-on-exec is not
set -- and that's *not* a defect in the library?!
If a library will trash important data if a file descriptor is not set
close-on-exec, and it fails to set it close-on-exec, it's broken.
> The caller may also know the library will be called again and the files will be
> relevant.
The caller has no way of knowing what the relevancy of the files will
be to future calls. The library does. The caller has no idea why the
library opened the files at that level of detail. That the library
even uses files is typically an implementation detail.
> Only the caller can know this. The library can't know that it will ever be
> called again. To assume so is an error in the implementation.
It makes no difference whether the library will ever be called again.
If the library knows how to use the descriptors in an exec'd process,
it should not set close on exec and neither should the called. If the
library knows how to use the descriptors, whether or not it will be
called again, it is wrong for the caller to close them.
> > One could imagine, for the sake of argument, a library that was a thin
> > wrapper for opening a file of some kind. In this case, it might well
> > be the caller that knows whether the descriptor should be closed or
> > not. In this case, either the caller would have to tell the library
> > how to set the close-on-exec flag or it would be the caller's
> > responsibility to set the flag correctly after the wrapper returns. If
> > neither has been done, we are again back to a broken library API or
> > broken library implementation.
> Yes it does come down to a broken API, one that doesn't have a close up shop call.
Even a "close up shop" call may leave open file descriptors that the
library will come back and close later. The library may not be able to
shut down completely at that instant -- it may have logging to do,
operations to sync or finish, or who knows what.
> Rather obvious would be calls as you describe above, but in those cases the fd
> is being passed out of the library. If the library opens some pipes so it can
> perhaps talk to other instances of the library and this is done internally that
> is the bad situation. Can you imagine the problems when the caller forks and
> exec's a shell and this stuff propagates down the line?!
If the library is a thin wrapper on open, then either it's the
caller's responsibility to close the descriptor or it's the library's.
If it's the caller's, then the caller can set close-on-exec too. There
is, however, on case where it's rational for the caller to dig around
in descriptors the library internally opened.
> Hopefully no one writes a library this way, but we also know human nature.
True that.
DS
David Schwartz
wrote:
> It means that you will one day or another end up with a race condition
> if another thread creates a process just at the wrong timing.
> This can have issues more than cosmetic ones (such as pipes not
> correctly detecting the other end closing, and so on)
One would hope the newly-exec'd process would close any file
descriptors it had no interest in. Sadly, there is no good way to do
this. I guess you can loop through 'getdtablesize' and 'close' every
descriptor you don't know about or plan to use.
DS
Jens Thoms Toerring
> Jens Thoms Toerring wrote:
> > I am having the same problem regularly, i.e. having to use li-
> > braries that are buggy in David's sense in that they don't set
> > the close-on-exec flag on files they open.
> Most third-party libraries I'm using are in this case, unfortunately.
> And I suppose that most programmers are just unaware of the problem.
Yes, it looks like that. What's worse is that some also can't be
convinced to learn:-(
> Is it me or there is a last "minor" issue regarding standard C library
> functions ?
> You can not "fopen"
> (<http://www.opengroup.org/onlinepubs/009695399/functions/fopen.html>)
> atomically a file with FD_CLOEXEC set. You can set this flag (fcntl)
> after calling fopen() using fcntl(fileno(..)), but not in one shot.
> It means that you will one day or another end up with a race condition
> if another thread creates a process just at the wrong timing.
It's already a problem with open(2) since FD_CLOEXEC is automati-
cally cleared on open() (and it's not listed as one of the flags
you could use with open() in POSIX, O_CLOEXEC seems to be Linux-
specific), so you only can set it afterwards and it's not going
to be an atomic operation...
The only method to get around that problem I can see at the moment
would be to have a mutex that gets locked before a file is opened
and its FD_CLOEXEC set and that also gets locked before a call of
fork(). Another point why it would have been better if FD_CLOEXEC
had been made the default setting, but that can't be changed any-
more since it might break quite a number of programs...
Geoff Clare
> It's already a problem with open(2) since FD_CLOEXEC is automati-
> cally cleared on open() (and it's not listed as one of the flags
> you could use with open() in POSIX, O_CLOEXEC seems to be Linux-
> specific)
O_CLOEXEC is in the new (2008) POSIX standard.
--
Geoff Clare <net...@gclare.org.uk>
James Kanze
> On Feb 16, 11:20 am, Xavier Roche <xro...@free.fr.NOSPAM.invalid>
> wrote:
[...]
> > The "obvious" solution could be to create all possible file
> > descriptors with FD_CLOEXEC, but when dealing with external
> > libraries, you do not always have the control of every
> > single file descriptor created (fopen(), socket(), ..),
> > unfortunately. And there is no way to make "FD_CLOEXEC" the
> > default behaviour within a process (1)
> No, not "all possible file descriptors" only those you don't
> want inherited. External libraries should set close on exec if
> they want the descriptors closed on an exec. If they don't set
> it, then they are either buggy or don't want those descriptors
> closed.
> > With fork()/exec(), a (rather dirty, but working) solution
> > is to close in the fork()'ed child all file descriptors
> > between 3 and 4096 (or 3 to sysconf(_SC_OPEN_MAX) with most
> > environments) with close(), after dup2()'ing
> > stdin/stdout/stderr, and then go on with the exec() thing.
> That's very dirty, since there's no way to know whether those
> descriptors should be closed or not. So that will also close
> descriptors that shouldn't be closed. If only there was some
> simple way to set a flag on a descriptor that indicates
> whether or not it should be closed on an exec and close only
> those descriptors that are flagged. Oh, wait, there is. It's
> called 'close on exec'.
This sounds wrong to me. The external library doesn't know what
files should be left open on exec, because it knows nothing
about the exec. It only knows what exec it invokes. So given
the default (backwards, IMHO, but we're stuck with it for
historical reasons), the only real solution is to close all
files you don't want open (and anything the process you're
starting doesn't know about, and hasn't documented, shouldn't be
left open) after the fork and before the exec.
A library may have not positionned FD_CLOEXEC because it exec's
to processes which need to use its file descriptors. That
doesn't mean that they should remain open accross other exec's.
Whether a file descriptor should remain open or not depends on
the target of the exec, not (just) on the file descriptor.
> > The posix_spawn() function offers a "similar" feature, through a
> > "posix_spawn_file_actions_t" opaque structure, which let you
> > close/open/dup2 all file descriptors you want.
> > But there's a little (rather) annoying issue with this API:
> > - there is no way to tell "please close every single remaining file
> > descriptor"
> Of course not. That would be absurd. That would close even
> descriptors that you had no way to know *should* be closed and
> break inter- operation with properly-coded libraries.
No. What you want is the possibility to say: ensure all file
descriptors except those in a given list are closed. You don't
want processes inheriting file descriptors they don't know
about.
> > - closing a file descriptor which was not already opened
> > just breaks your process (2)
And that's a real problem. If I understand the specification
correctly, you could for each file descriptor do an addopen on
"/dev/null", then and addclose; according to the specification,
an addopen will cause any previously open file on the file
descriptor to be closed before trying the open. But quite
frankly, that sounds horrendous.
Alternatively, you could spawn a small program that you own,
which does the closes, and then spawns (or even fork/execs) the
program you actually want. That's probably what I'd do.
Ideally, of course, you'd have a create_process function, which
creates a process without starting it, with no files open; you
would then have a command to dup one of your file descriptors
into it, and probably one to open a file directly in it. And
finally a command to start it, once you'd finished setting it
up. Failing that, the best I can see is to always spawn a small
process of your own, which straightens things up the way you
want, then spawns or fork/exec's to the actual target program.
> If you want to do some real defensive coding, on startup,
> close any file descriptors you may have inherited but don't
> want.
That's probably the best solution, but how many programs do
that? Remember that you might not be the author of the program
you're starting (unless it is a small process of your own, as
described above).
--
James Kanze (GABI Software) email:james...@gmail.com
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
David Schwartz
> > That's very dirty, since there's no way to know whether those
> > descriptors should be closed or not. So that will also close
> > descriptors that shouldn't be closed. If only there was some
> > simple way to set a flag on a descriptor that indicates
> > whether or not it should be closed on an exec and close only
> > those descriptors that are flagged. Oh, wait, there is. It's
> > called 'close on exec'.
> This sounds wrong to me. The external library doesn't know what
> files should be left open on exec, because it knows nothing
> about the exec. It only knows what exec it invokes. So given
> the default (backwards, IMHO, but we're stuck with it for
> historical reasons), the only real solution is to close all
> files you don't want open (and anything the process you're
> starting doesn't know about, and hasn't documented, shouldn't be
> left open) after the fork and before the exec.
If the external library doesn't know whether it's own files should be
closed on an exec, something is badly wrong with it. Either it knows
to manage those files across an exec, in which case they should remain
open, or it doesn't, in which case they shouldn't.
> A library may have not positionned FD_CLOEXEC because it exec's
> to processes which need to use its file descriptors. That
> doesn't mean that they should remain open accross other exec's.
> Whether a file descriptor should remain open or not depends on
> the target of the exec, not (just) on the file descriptor.
That's is correct. The target of an exec has to close any file
descriptors it doesn't plan to use. A library that supports keeping
file descriptors open across an exec the library doesn't make has to
provide some way to know which file descriptors it plans to continue
to use. If the library API doesn't do this, then the API is broken.
> No. What you want is the possibility to say: ensure all file
> descriptors except those in a given list are closed. You don't
> want processes inheriting file descriptors they don't know
> about.
If a library relies on you to close its file descriptors across an
exec, that would have to be a defined part of the library's API. If a
library relies on you to leave its file descriptors open across an
exec, that would also have to be a defined part of the library's API.
If the library doesn't set close-on-exec but doesn't want you to keep
its descriptors open, the library is broken (or its API is).
> Alternatively, you could spawn a small program that you own,
> which does the closes, and then spawns (or even fork/execs) the
> program you actually want. That's probably what I'd do.
Or just make sure the target program knows to close any file
descriptors it winds up with that it doesn't want.
> Ideally, of course, you'd have a create_process function, which
> creates a process without starting it, with no files open; you
> would then have a command to dup one of your file descriptors
> into it, and probably one to open a file directly in it. And
> finally a command to start it, once you'd finished setting it
> up. Failing that, the best I can see is to always spawn a small
> process of your own, which straightens things up the way you
> want, then spawns or fork/exec's to the actual target program.
That will break perfectly working libraries that rely on file
descriptors they've opened and specifically *not* set close-on-exec on
to be open in an exec'd process.
> > If you want to do some real defensive coding, on startup,
> > close any file descriptors you may have inherited but don't
> > want.
> That's probably the best solution, but how many programs do
> that? Remember that you might not be the author of the program
> you're starting (unless it is a small process of your own, as
> described above).
Only the program can know for sure that it doesn't inherit any file
descriptors from any libraries used by the programs that might launch
it. (Because any such library would have to provide you a way to know
not to close the descriptor.)
DS
Golden California Girls
> On Feb 16, 10:16 pm, Golden California Girls <gldncag...@aol.com.mil>
> wrote:
>
>>> No, it would be the library. Only the library knows why it opened the
>>> files. The caller, in principle, has no idea what the library intends
>>> to do with the file.
>
>> The caller may know that the use of the library is over. Translation, the
>> library will never be called again.
>
> So what? Just because he'll never call the library again it does not
> follow that the library is done. The library may be in the process of
> completing some operation and still need to intercept a signal or
> something.
You do understand what exec does don't you?
Xavier Roche
> O_CLOEXEC is in the new (2008) POSIX standard.
And only implemented very recently on Linux, for example (2.6.23 if I am
not mistaken - in mid- 2007). What about socket() ? pipe() ? fopen() ?
All of them have the same insane race condition because of the default
FD_CLOEXEC cleared flag.
Rainer Weikusat
Ehhh ... so fucking what? If this is a problem for something YOU are
trying to accomplish, explicit synchronization is necessary, as for
many other things. It has not been a problem I had to deal with so far
and I actually use fork from multithreaded processes at least in one
situation[*].
Thou shalt not add Windows-interfaces to UNIX(*)-systems
lightheartedly. This will only lead to Windows users, like our dear
OP, complaining even more loudly that there are still differences
between 'UNIX(*)' and 'The One True Microsoft Way'.
David Schwartz
wrote:
> >> The caller may know that the use of the library is over. Translation, the
> >> library will never be called again.
> > So what? Just because he'll never call the library again it does not
> > follow that the library is done. The library may be in the process of
> > completing some operation and still need to intercept a signal or
> > something.
> You do understand what exec does don't you?
Yes. His point was that the library might never be called again in the
same process.
But a library can also intercept libc, for example, and be active on
both sides of the 'exec'. It can even hook the 'fork' operation to
prepare things for the forthcoming 'exec', though it can't know at
fork time whether there will be an exec.
DS
Golden California Girls
You seem to be making the assumption that the exec'd program is going to be
cooperating with the library. Rather obviously after the exec the signal
handler you wrote of above isn't going to be there to get the signal, hook on
fork or not. And there is no reason to assume fork is even called. Bad juju
for the library to assume it will be around after exec. This doesn't stop
people from writing code like that and publishing it as a library for others to
use. Just because all the programs they ever wrote end with a call to exit
doesn't make it so.
David Schwartz
wrote:
> You seem to be making the assumption that the exec'd program is going to be
> cooperating with the library. Rather obviously after the exec the signal
> handler you wrote of above isn't going to be there to get the signal, hook on
> fork or not. And there is no reason to assume fork is even called. Bad juju
> for the library to assume it will be around after exec.
I think you're misunderstanding my argument. I'm simply saying that
the application can't assume the library is finished just because it
never intends to call into the library again. Any argument that
begins, "the application can close any file descriptors the library
opened because it knows they will never be accessed because it won't
call the library again" is simply incorrect. The application has no
way of knowing whether or not the library still has some way to finish
what it's doing, unless the library API specifies a way.
You said:
"The caller may know that the use of the library is over.
Translation, the
library will never be called again."
I am saying this is wrong. Just because the caller will never call the
library again, it does not follow that the library will never run any
more code. This is true even across an 'exec', as the library may be
active on both sides of the 'exec'. One can imagine a library that is
designed to share a file descriptor across a tree of processes that
use it.
DS
Golden California Girls
Just how is the library going to execute when it isn't in RAM after the exec?
Or did you forget that part?
I'm not and neither was the OP talking about a cooperative tree of processes
that all load this library and share it. We are talking about going off
elsewhere and the need to close all the files the library left open because it
didn't set close on exec. I am sure that a system call has the exact same issue
as exec.
The library designer should not be placing constraints on what a calling program
can do. It should not force the only way out via exit. In short once it
returns to is caller it can not expect to ever -- even via signal -- run ever
again. Now it may offer a close up shop function up on its API and document
that, but it if doesn't ...
If the library passes out via its API all the fd's it opens then the calling
program can set the close on exec flag before it exec's elsewhere or does a
system call and if there is a return it can restore the flag to what the library
set. If there are hidden files the library designer needs to put functions to
do that into his API.
James Kanze
> On Feb 17, 7:26 am, James Kanze <james.ka...@gmail.com> wrote:
> > > That's very dirty, since there's no way to know whether
> > > those descriptors should be closed or not. So that will
> > > also close descriptors that shouldn't be closed. If only
> > > there was some simple way to set a flag on a descriptor
> > > that indicates whether or not it should be closed on an
> > > exec and close only those descriptors that are flagged.
> > > Oh, wait, there is. It's called 'close on exec'.
> > This sounds wrong to me. The external library doesn't know
> > what files should be left open on exec, because it knows
> > nothing about the exec. It only knows what exec it invokes.
> > So given the default (backwards, IMHO, but we're stuck with
> > it for historical reasons), the only real solution is to
> > close all files you don't want open (and anything the
> > process you're starting doesn't know about, and hasn't
> > documented, shouldn't be left open) after the fork and
> > before the exec.
> If the external library doesn't know whether it's own files
> should be closed on an exec, something is badly wrong with it.
> Either it knows to manage those files across an exec, in which
> case they should remain open, or it doesn't, in which case
> they shouldn't.
It can't possibly know. Or rather, it's own files should
definitely (and always) be closed across an exec it doesn't know
about, but maybe shouldn't be closed for an exec in the library
itself. The basic problem is that whether the file should be
closed on exec depends not only on the file, but the exec.
> > A library may have not positionned FD_CLOEXEC because it
> > exec's to processes which need to use its file descriptors.
> > That doesn't mean that they should remain open accross other
> > exec's. Whether a file descriptor should remain open or not
> > depends on the target of the exec, not (just) on the file
> > descriptor.
> That's is correct. The target of an exec has to close any file
> descriptors it doesn't plan to use. A library that supports
> keeping file descriptors open across an exec the library
> doesn't make has to provide some way to know which file
> descriptors it plans to continue to use. If the library API
> doesn't do this, then the API is broken.
I think rather that it is the design of Unix that is broken
here.
[...]
> > Ideally, of course, you'd have a create_process function,
> > which creates a process without starting it, with no files
> > open; you would then have a command to dup one of your file
> > descriptors into it, and probably one to open a file
> > directly in it. And finally a command to start it, once
> > you'd finished setting it up. Failing that, the best I can
> > see is to always spawn a small process of your own, which
> > straightens things up the way you want, then spawns or
> > fork/exec's to the actual target program.
> That will break perfectly working libraries that rely on file
> descriptors they've opened and specifically *not* set
> close-on-exec on to be open in an exec'd process.
That's ridiculous, of course. Since the library doesn't know
about the exec, it can't depend on its file descriptors being
open in it.
Xavier Roche
> I think rather that it is the design of Unix that is broken
> here.
Which tend to be my conclusion, too. Or maybe a "historical" design,
rather than broken.
What I do not understand is how posix_spawn() can be broken too,
considering its late release (1999).
David Schwartz
wrote:
> Just how is the library going to execute when it isn't in RAM after the exec?
> Or did you forget that part?
There are any number of ways. For example, the library could arrange
for itself to be preloaded. The library could arrange with a helper
application to put the new executable in debug mode and forcibly
attach itself to it. The library could be the system's libc library.
It doesn't matter, the application doesn't (shouldn't) care about
library internals.
> I'm not and neither was the OP talking about a cooperative tree of processes
> that all load this library and share it. We are talking about going off
> elsewhere and the need to close all the files the library left open because it
> didn't set close on exec. I am sure that a system call has the exact same issue
> as exec.
I'm not talking about a cooperative tree of processes that all load
this library and share it. I'm talking about a library that makes a
cooperative tree of processes that cooperate for the purposes of this
library, possibly unbeknownst to the applications that call it who
don't (shouldn't) give a damn *how* the library does its job. It might
be by cooperating processes, it might not. The application should not
have to care.
> The library designer should not be placing constraints on what a calling program
> can do. It should not force the only way out via exit.
I agree.
> In short once it
> returns to is caller it can not expect to ever -- even via signal -- run ever
> again. Now it may offer a close up shop function up on its API and document
> that, but it if doesn't ...
Ridiculous. That forcing blocking behavior for things like logging FOR
NOT REASON. That's an absolutely brain-dead requirement to impose on
arbitrary libraries. The application shouldn't care how a logging
library implements logging, it should just call a 'log' function when
it wants to log. If the library arranges to do the logging in a non-
blocking way, that's perfectly fine.
> If the library passes out via its API all the fd's it opens then the calling
> program can set the close on exec flag before it exec's elsewhere or does a
> system call and if there is a return it can restore the flag to what the library
> set. If there are hidden files the library designer needs to put functions to
> do that into his API.
Yes, if you were using a library that said, as part of its API, that
you had to do that, then that would be what you had to do. Such a
library would solve its responsibility to manage its file descriptors
in a documented way, which would be wonderful. The point is, it's the
library's responsibility. It can pass it on to the application, and
then (but only then) does it become the application's responsibility..
Unless the API/documentation says otherwise (or special cases such as
thin wrappers around 'open'), the library's descriptors are an
implementation detail and the application shouldn't touch them.
DS
David Schwartz
> That's ridiculous, of course. Since the library doesn't know
> about the exec, it can't depend on its file descriptors being
> open in it.
Of course it can. If it doesn't set close-on-exec, its descriptors are
guaranteed to be open in the exec'd process.
The process you exec may expect this case, and the very first thing it
might do is call into the library to 'pick up' its file descriptors.
To the process that exec's this process, it's all a library
implementation detail and not something it should have to be concerned
about.
DS
Rainer Weikusat
> On Feb 17, 6:52 pm, David Schwartz <dav...@webmaster.com> wrote:
>> On Feb 17, 7:26 am, James Kanze <james.ka...@gmail.com> wrote:
[...]
>> That's is correct. The target of an exec has to close any file
>> descriptors it doesn't plan to use. A library that supports
>> keeping file descriptors open across an exec the library
>> doesn't make has to provide some way to know which file
>> descriptors it plans to continue to use. If the library API
>> doesn't do this, then the API is broken.
>
> I think rather that it is the design of Unix that is broken
> here.
posix_spawn isn't functionally equivalent to fork/exec and the fact
that it isn't is documented. If someone insists on using it for things
it doesn't provide, that's an ordinary application programming error.
Rainer Weikusat
UNIX(*) processes are created by fork and insofar a different
executable than the currently running one should be executed by some
process, this process is responsible for setting up a suitable
execution environment beforehand. posix_spawn provides a subset of the
functionality of fork/exec and is intended as 'process creation and
file execution primitive that can be efficiently implemented without
address translation or other MMU services'.
You have sufficiently determined that it is, despite the superficially
greater similarity, still different from the Windows process creation
primitive.
Further, you have stated that, according to your opinion, whatever
Microsoft implements is (until Microsoft implements something
different) by definition 'correct' and that any deflection from
whatever is currently considered 'the Microsoft way' would therefore
be incorrect.
You are free to believe that. But it would be kind of you would
discuss your ideological preferences in a suitable advocacy group.
Jens Thoms Toerring
> James Kanze wrote:
> > I think rather that it is the design of Unix that is broken
> > here.
> Which tend to be my conclusion, too. Or maybe a "historical" design,
> rather than broken.
I also would guess it's for historical reasons. fork()/exec() is
very often used (and perhaps was even invented with this in mind)
to "pipe" data through unrelated programs - and then closing all
fd's on exec() automatically might have been rather annoying in
the most common case.
> What I do not understand is how posix_spawn() can be broken too,
> considering its late release (1999).
Well, it isn't "broken". The other way round, i.e. closing all
open files automatically (but do you leave STDIN, STDOUT and
STDERR open - they might have been dup()ed to files?) would
also have had its problems: first of all, it would have been
surprising to most programmers used to fork()/exec(), moreover
it would break the FD_CLOEXEC flag stuff since there's no flag
that tells "don't close this file on posix_spawn(). And then
you would have to explicitely specify the files you want to
be inherited by the new process. That would lead to the same
problem with broken libraries as we have it now, just the other
way round.
Moreover, posix_spawn() gives you an extra method to control
what happens to the files via the 'file_actions' argument. If
you know your file descriptors you can even e.g. close a file
for which you don't want to set FD_CLOEXEC in the caller...
So, to me it seems that what's really broken are libraries that
open files behind your back, don't document that and don't give
you a method to set the close-on-exec flag on these files.
I guess the only method around this problem, as I get it from
the whole discussion, is the following:
1) keep a list of all files you open in the code you control
2) before spawning a new process check if there are any open
files beside the one you opened yourself and set the close-
on-exec flag on those.
A method to find out which files are open could be:
1) sort the list of fd's you opened
2) check (e.g. with isatty() and checking for EBADF) if fd's
with values smaller than the smallest in your list are open
3) check for fd's larger than the largest in your list, stopping
when you don't find one anymore.
That should work without having to go through all possible ones
since open() always uses the smallest possible fd available.
Jens Thoms Toerring
> Further, you have stated that, according to your opinion, whatever
> Microsoft implements is (until Microsoft implements something
> different) by definition 'correct' and that any deflection from
> whatever is currently considered 'the Microsoft way' would therefore
> be incorrect.
> You are free to believe that. But it would be kind of you would
> discuss your ideological preferences in a suitable advocacy group.
Sorry, but I can't find anything in this thread by Xavier (and
I couldn't find any other posts by him here in cup) that would
just mention Windows. Are you perhaps getting him mixed up with
somebody else?
Rainer Weikusat
> Rainer Weikusat <rwei...@mssgmbh.com> wrote:
>> Further, you have stated that, according to your opinion, whatever
>> Microsoft implements is (until Microsoft implements something
>> different) by definition 'correct' and that any deflection from
>> whatever is currently considered 'the Microsoft way' would therefore
>> be incorrect.
>
>> You are free to believe that. But it would be kind of you would
>> discuss your ideological preferences in a suitable advocacy group.
>
> Sorry, but I can't find anything in this thread by Xavier (and
> I couldn't find any other posts by him here in cup) that would
> just mention Windows.
Amusingly, he is no longer using the NT6 installation for his
'discussion' of how handling 'handle inheritance' in the opposite way
of how Windows does it (see MS documentation for that) would be
"broken". The 'race condition' is unavoidable once an inheritable
objects exists in a multi-threaded process where any thread could
create another process with arbitrary attributes at any time. It is
made worse when no (simple) possibility to execute cleanup code before
the new program is started exists.
Xavier Roche
> Amusingly, he is no longer using the NT6 installation
Uh ? I don't really see the point in this remark. Have you ever heard
about "portable code" ?
> 'discussion' of how handling 'handle inheritance' in the opposite way
> of how Windows does it (see MS documentation for that)
Well, you are wrong. The WIN32 flavor *looks* less broken as you can
specify explicitly when spawning a process if you want inheritance or
not. But it is broken anyway, because there are other major problems
with this flag (ie. stdio pipes do not work, so you are screwed if you
want to play with pipes too). But all of this is off-topic here.
> It is made worse when no (simple) possibility to execute cleanup code before
> the new program is started exists.
The whole inheritance thing is broken in my opinion. If I want to spawn
an independent (without ANY file descriptor inheritance but the ones I
want) process, I can't.
Even if an hidden "magical" library WANTS not to "close on exec", I
could ENFORCE not to inherit this file descriptor, and I would get the
same semantic as a fresh new process. This would not break anything, and
this would allow to make things sane.
Golden California Girls
> On Feb 17, 11:37 pm, Golden California Girls <gldncag...@aol.com.mil>
> wrote:
>
>> Just how is the library going to execute when it isn't in RAM after the exec?
>> Or did you forget that part?
>
> There are any number of ways. For example, the library could arrange
> for itself to be preloaded. The library could arrange with a helper
> application to put the new executable in debug mode and forcibly
> attach itself to it. The library could be the system's libc library.
The exec'd program isn't a C program so libc isn't used.
As for the debug option, I wonder if that might be a way to steal root if the
program being exec'd is setuid? Someone would have thought of this and a
otherwise valid thing to do, exec a setuid program, will fail under your library
or your helper won't be able to attach to it thus killing the library. (Of
course if your helper is root, that is the road to hell.)
If preloaded doesn't involve having its own process or being part of the O/S
please explain how this library is linked to an exec'd program that doesn't call it.
> It doesn't matter, the application doesn't (shouldn't) care about
> library internals.
That's correct, all files in the library opens in the user's process, behind the
users back, should be marked close on exec. It really shouldn't be opening
anything behind the callers back in his process, but that is beside the point.
>> I'm not and neither was the OP talking about a cooperative tree of processes
>> that all load this library and share it. We are talking about going off
>> elsewhere and the need to close all the files the library left open because it
>> didn't set close on exec. I am sure that a system call has the exact same issue
>> as exec.
>
> I'm not talking about a cooperative tree of processes that all load
> this library and share it. I'm talking about a library that makes a
> cooperative tree of processes that cooperate for the purposes of this
> library, possibly unbeknownst to the applications that call it who
> don't (shouldn't) give a damn *how* the library does its job. It might
> be by cooperating processes, it might not. The application should not
> have to care.
The application always cares. (So does the library.) The library shouldn't
mess with system limit on the number of files allowed open by leaving a bunch
open across an exec nor open security holes with files to who knows what opened
that a malware program exploits after the exec. All files the library opens,
behind the callers back, in the callers process should (must) be marked close on
exec. (I'm assuming the admin has been smart enough to put the object file for
the library where the user can't link against it otherwise there is no need of
an exec to do the exploits.)
>> The library designer should not be placing constraints on what a calling program
>> can do. It should not force the only way out via exit.
>
> I agree.
>
>> In short once it
>> returns to is caller it can not expect to ever -- even via signal -- run ever
>> again. Now it may offer a close up shop function up on its API and document
>> that, but it if doesn't ...
>
> Ridiculous. That forcing blocking behavior for things like logging FOR
> NOT REASON. That's an absolutely brain-dead requirement to impose on
> arbitrary libraries. The application shouldn't care how a logging
> library implements logging, it should just call a 'log' function when
> it wants to log. If the library arranges to do the logging in a non-
> blocking way, that's perfectly fine.
If the library wants to do non-blocking io, it must have its own process. It
dang well better not depend on a process that is likely to sigsegv the next
processor cycle when it returns from the signal handler that called the library
to start the logging. You also aren't thinking clearly, how is it that the
library knows it will get the async io signal and not the callers signal handler
he put in place right after the return from the library?
As for the library itself, even if it has its own process the user could always
send a kill and make it go away.
David Schwartz
wrote:
> David Schwartz wrote:
> > On Feb 17, 11:37 pm, Golden California Girls <gldncag...@aol.com.mil>
> > wrote:
>
> >> Just how is the library going to execute when it isn't in RAM after the exec?
> >> Or did you forget that part?
>
> > There are any number of ways. For example, the library could arrange
> > for itself to be preloaded. The library could arrange with a helper
> > application to put the new executable in debug mode and forcibly
> > attach itself to it. The library could be the system's libc library.
> The exec'd program isn't a C program so libc isn't used.
Uh, sorry, it doesn't work that way. If you ask how something is
possible, then I have to explain how it's possible. I explained how
it's possible.
Sure, you can make up some other case where it's not possible. But so
what? I never argued that this always happens, simply that it can
happen and should be allowed to happen and therefore a scheme that
assumes it can't happen is broken.
> As for the debug option, I wonder if that might be a way to steal root if the
> program being exec'd is setuid? Someone would have thought of this and a
> otherwise valid thing to do, exec a setuid program, will fail under your library
> or your helper won't be able to attach to it thus killing the library. (Of
> course if your helper is root, that is the road to hell.)
The application doesn't care about this, it's the library's problem.
> If preloaded doesn't involve having its own process or being part of the O/S
> please explain how this library is linked to an exec'd program that doesn't call it.
How do you know the exec'd program doesn't call it? You're making up
cases where there is no problem. Great, this proves that there exists
at least one case where your proposed method might work. Wonderful.
> > It doesn't matter, the application doesn't (shouldn't) care about
> > library internals.
> That's correct, all files in the library opens in the user's process, behind the
> users back, should be marked close on exec. It really shouldn't be opening
> anything behind the callers back in his process, but that is beside the point.
Huh? The whole point of a library is to "just work". If this means
opening file descriptors, great. If this means setting them close-on-
exec, great. If this means making them persist across an exec, great.
It's not the application's problem. As far as the application is
concerned, the library should just be magic.
> > I'm not talking about a cooperative tree of processes that all load
> > this library and share it. I'm talking about a library that makes a
> > cooperative tree of processes that cooperate for the purposes of this
> > library, possibly unbeknownst to the applications that call it who
> > don't (shouldn't) give a damn *how* the library does its job. It might
> > be by cooperating processes, it might not. The application should not
> > have to care.
> The application always cares. (So does the library.) The library shouldn't
> mess with system limit on the number of files allowed open by leaving a bunch
> open across an exec nor open security holes with files to who knows what opened
> that a malware program exploits after the exec.
You're just making up ways something can't work. If a sensible library
does this, then it doesn't do all the stupid things you're making up.
Any technique can be broken by implementing it badly.
> All files the library opens,
> behind the callers back, in the callers process should (must) be marked close on
> exec. (I'm assuming the admin has been smart enough to put the object file for
> the library where the user can't link against it otherwise there is no need of
> an exec to do the exploits.)
Why should they be marked close-on-exec if the library knows how to
handle them existing after a call to 'exec'?! Are you arguing this is
100% completely impossible?
> > Ridiculous. That forcing blocking behavior for things like logging FOR
> > NOT REASON. That's an absolutely brain-dead requirement to impose on
> > arbitrary libraries. The application shouldn't care how a logging
> > library implements logging, it should just call a 'log' function when
> > it wants to log. If the library arranges to do the logging in a non-
> > blocking way, that's perfectly fine.
> If the library wants to do non-blocking io, it must have its own process. It
> dang well better not depend on a process that is likely to sigsegv the next
> processor cycle when it returns from the signal handler that called the library
> to start the logging. You also aren't thinking clearly, how is it that the
> library knows it will get the async io signal and not the callers signal handler
> he put in place right after the return from the library?
Again, you are saying there are cases where this is not appropriate. I
agree. But your argument is that this is *never* appropriate. Suddenly
you just make things up like that it "depends" on something. Who said
it "depended" on it?
> As for the library itself, even if it has its own process the user could always
> send a kill and make it go away.
Sure, a user can always break his own stuff by sending 'kill' signals
to random processes. A technique does not have to resist deliberate
sabotage to be legitimate.
DS
roger.f...@sun.com
wrote:
> Hi folks,
>
> I have a problem which might be considered as rather obvious for many of
> you, but I just can not find a clean way to handle it.
>
> When spawning a new process in a parent process, there are situations
> where you do not want spawned processes to inherit parent's file
> descriptors (for example, when the parent already opened a very large
> number of file descriptors)
>
> Is there a clean way to acheive this, with posix_spawn(), in a
> multithreaded environment ?
[snip]
Given the current POSIX specification, there is no reliable way.
The spec should be fixed as follows:
1. Interpret the close file action to apply only if the target
file descriptor is open at the time of the call to posix_spawn().
Closing an already-closed file descriptor should not be an error.
2. Specify a new 'closefrom' file action:
int posix_spawn_file_actions_addclosefrom(
posix_spawn_file_actions_t *file_actions,
int lowfiledes);
which causes posix_spawn() to close all currently open file
descriptors that are greater than or equal to lowfiledes.
These two changes would solve almost all of the issues
that have been discussed in this thread.
The 'closefrom' operation is commonly used when creating
daemon processes. With a bit of operating system help,
it need not be expensive.
Roger Faulkner
Sun Microsystems
Xavier Roche
> Given the current POSIX specification, there is no reliable way.
> The spec should be fixed as follows:
> 1. Interpret the close file action to apply only if the target
> file descriptor is open at the time of the call to posix_spawn().
> Closing an already-closed file descriptor should not be an error.
Yep, this would be great. Breaking on a faulty dup2() or open() is
certainly sane, but breaking on close() does not really makes sense in
many situations.
> These two changes would solve almost all of the issues
> that have been discussed in this thread.
Yep - anyone known how sending feedback to IEEE regarding the POSIX
specs can be feasible ?
The PASC mailing-list seems a bit idle to me:
<http://www.pasc.org/plato/show_archive.tpl?CALLER=mailinglists.tpl&listname=pasc-gen-l>
(and this is probably not the correct place to send feedback)
Rainer Weikusat
> On Feb 16, 2:20 pm, Xavier Roche <xro...@free.fr.NOSPAM.invalid>
> wrote:
>> I have a problem which might be considered as rather obvious for many of
>> you, but I just can not find a clean way to handle it.
>>
>> When spawning a new process in a parent process, there are situations
>> where you do not want spawned processes to inherit parent's file
>> descriptors (for example, when the parent already opened a very large
>> number of file descriptors)
>>
>> Is there a clean way to acheive this, with posix_spawn(), in a
>> multithreaded environment ?
> [snip]
>
> Given the current POSIX specification, there is no reliable way.
> The spec should be fixed as follows:
>
> 1. Interpret the close file action to apply only if the target
> file descriptor is open at the time of the call to posix_spawn().
> Closing an already-closed file descriptor should not be an error.
What makes you believe that a file action which requests closing some
descriptor would be invalid if the descriptor isn't open? Or any other
action on a descriptor, fwiw. If this was so, it would be impossible
to spawn anything reliably from a multithreaded process (another
thread could close the descriptor at any time).
Casper H.S. Dik
>What makes you believe that a file action which requests closing some
>descriptor would be invalid if the descriptor isn't open? Or any other
>action on a descriptor, fwiw. If this was so, it would be impossible
>to spawn anything reliably from a multithreaded process (another
>thread could close the descriptor at any time).
Because the standard says it is so:
If the file_actions argument is not NULL, and specifies any close,
dup2, or open actions to be performed, and if posix_spawn() or
posix_spawnp() fails for any of the reasons that would cause close(),
dup2(), or open() to fail, an error value shall be returned as
described by close(), dup2(), and open(), respectively (or, if the
error occurs after the calling process successfully returns, the child
process shall exit with exit status 127). An open file action may, by
itself, result in any of the errors described by close() or dup2(), in
addition to those described by open().
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.
Rainer Weikusat
> Rainer Weikusat <rwei...@mssgmbh.com> writes:
>
>>What makes you believe that a file action which requests closing some
>>descriptor would be invalid if the descriptor isn't open? Or any other
>>action on a descriptor, fwiw. If this was so, it would be impossible
>>to spawn anything reliably from a multithreaded process (another
>>thread could close the descriptor at any time).
>
> Because the standard says it is so:
>
> If the file_actions argument is not NULL, and specifies any close,
> dup2, or open actions to be performed, and if posix_spawn() or
> posix_spawnp() fails for any of the reasons that would cause close(),
> dup2(), or open() to fail, an error value shall be returned as
> described by close(), dup2(), and open(), respectively (or, if the
> error occurs after the calling process successfully returns, the child
> process shall exit with exit status 127). An open file action may, by
> itself, result in any of the errors described by close() or dup2(), in
> addition to those described by open().
The original poster was specifically referring to EINVAL as an error
code return from posix_spawn itself. And this should certainly not
happen for an attempt to perform an operation on a 'bad' file
descriptor. That posix_spawn is really mostly useless in multithreaded
applications, especially, multithreaded applications where some
threads may be controlled by code which exists only in compiled form,
because the specification explictly demands this in another context is
an amusing detail, though.
Xavier Roche
> What makes you believe that a file action which requests closing some
> descriptor would be invalid if the descriptor isn't open? Or any other
> action on a descriptor, fwiw.
The spec is not totally clear regarding this issue, but suggests that a
failure in one of the close()/dup2() or open() operation would result in
breaking the process with code 127. (1)
> If this was so, it would be impossible
> to spawn anything reliably from a multithreaded process (another
> thread could close the descriptor at any time).
Unfortunately this is the observed behaviour (at least on Solaris and
Linux): closing an already closed file descriptor leads to have a dead
born child (with exit code 127).
The consequence is that you can not attempt to modify any file
descriptor you do not have the control on using the
posix_spawn_file_actions_t opaque structure.
(1)
<http://www.opengroup.org/onlinepubs/009695399/functions/posix_spawn.html>
(..)
If the file_actions argument is not NULL, and specifies any close, dup2,
or open actions to be performed, and if posix_spawn() or posix_spawnp()
fails for any of the reasons that would cause close(), dup2(), or open()
to fail, an error value shall be returned as described by close(),
dup2(), and open(), respectively (or, if the error occurs after the
calling process successfully returns, the child process shall exit with
exit status 127). An open file action may, by itself, result in any of
the errors described by close() or dup2(), in addition to those
described by open().
(2)
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <spawn.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/wait.h>
#include <unistd.h>
extern char **environ;
/* error info */
#define FATAL_ERROR(WHERE) do { \
const int err = errno; \
const char *const err_s = strerror(err); \
fprintf(stderr, "** error with %d with %s: %s\n", err, WHERE, err_s); \
exit(EXIT_FAILURE); \
} while(0)
int main(void) {
int fd;
pid_t pid;
posix_spawn_file_actions_t actions;
int status;
char* argv[] = {"/bin/ps", NULL};
/* init ' actions' */
if (posix_spawn_file_actions_init(&actions) != 0) {
FATAL_ERROR("posix_spawn_file_actions_init");
}
/* open /dev/null (by default, FD_CLOEXEC is cleared) */
fd = open("/dev/null", O_RDONLY);
if (fd == -1) {
FATAL_ERROR("open");
}
/* close 'fd' on child */
if (posix_spawn_file_actions_addclose(&actions, fd) != 0) {
FATAL_ERROR("posix_spawn_file_actions_addclose");
}
/* with -DALREADY_CLOSED, close the parent fd */
#ifdef ALREADY_CLOSED
/* close in parent */
if (close(fd) != 0) {
FATAL_ERROR("close");
}
#endif
/* spawn process */
if (posix_spawn(&pid, argv[0], &actions, NULL, argv, environ) != 0) {
FATAL_ERROR("posix_spawn");
}
/* cleanup */
posix_spawn_file_actions_destroy(&actions);
/* wait for pid */
if (waitpid(pid, &status, 0) != pid) {
FATAL_ERROR("waitpid");
}
/* and print the exit code */
printf("process returned status %d\n", status);
return 0;
}
Casper H.S. Dik
>The original poster was specifically referring to EINVAL as an error
>code return from posix_spawn itself. And this should certainly not
>happen for an attempt to perform an operation on a 'bad' file
>descriptor. That posix_spawn is really mostly useless in multithreaded
>applications, especially, multithreaded applications where some
>threads may be controlled by code which exists only in compiled form,
>because the specification explictly demands this in another context is
>an amusing detail, though.
Ok, so you get EBADF. Still doesn't help you using posix_spawn.
Casper
James Kanze
wrote:
> James Kanze wrote:
> > I think rather that it is the design of Unix that is broken
> > here.
> Which tend to be my conclusion, too. Or maybe a "historical"
> design, rather than broken.
Historical might be the reason, but broken is the correct
qualifier.
> What I do not understand is how posix_spawn() can be broken
> too, considering its late release (1999).
Well, it does have to be compatible with existing functions.
But I sort of agree; I don't quite understand why they still
insisted on doing it backwards, and forcing everything in one
function, rather than creating a process without starting it,
then allowing you to "set it up" before starting it.
James Kanze
> On Feb 18, 1:18 am, James Kanze <james.ka...@gmail.com> wrote:
> > That's ridiculous, of course. Since the library doesn't know
> > about the exec, it can't depend on its file descriptors being
> > open in it.
> Of course it can. If it doesn't set close-on-exec, its
> descriptors are guaranteed to be open in the exec'd process.
If it doesn't know about the process, it can't count on it being
there, much less on its file descriptors being open. You're
just being silly.
> The process you exec may expect this case, and the very first
> thing it might do is call into the library to 'pick up' its
> file descriptors.
If the library exec's to a specific process, yes. If you exec
to 'ls', for example, no.
Rainer Weikusat
> On Feb 18, 10:26 am, Xavier Roche <xro...@free.fr.NOSPAM.invalid wrote:
>> James Kanze wrote:
>> > I think rather that it is the design of Unix that is broken
>> > here.
>
>> Which tend to be my conclusion, too. Or maybe a "historical"
>> design, rather than broken.
>
> Historical might be the reason, but broken is the correct
> qualifier.
The correct qualifier would be 'that was not how I would have done
it' (or maybe still substitute "But Microsoft hasn't copied that
!!1"). As a matter of fact, the existing behaviour is useful for
certain cases, useless but non-relevant for certain other cases and
at all unusable for yet other cases.
Which - coincidentally - happens to always be this way.
>> What I do not understand is how posix_spawn() can be broken
>> too, considering its late release (1999).
>
> Well, it does have to be compatible with existing functions.
> But I sort of agree; I don't quite understand why they still
> insisted on doing it backwards,
What you consider to be 'forwards' and 'backwards' is entirely your
personal opinion and other people might actually have a different
one. There is no 'naturally given order' for anything on a computer --
all is arbitrary. If 'everything' is inherited by default, code needs
to be written to prevent inheritance of certain things (but
posix_spawn doesn't support that), otherwise, code needs to be written
to cause certain things to be inherited. What would be more of an
advantage depends on the situation.
BTW, the only sensible definition of 'broken', insofar this refers to
software, is 'observable behaviour differs from specified behaviour'.
Anything beyond that is just an attempt to pass an opinion, usually
without arguments supporting it, as fact (if there were arguments,
marketing trick wouldn't be necessary).
Casper H.S. Dik
>On Feb 18, 10:26 am, Xavier Roche <xro...@free.fr.NOSPAM.invalid>
>wrote:
>> James Kanze wrote:
>> > I think rather that it is the design of Unix that is broken
>> > here.
>> Which tend to be my conclusion, too. Or maybe a "historical"
>> design, rather than broken.
>Historical might be the reason, but broken is the correct
>qualifier.
>> What I do not understand is how posix_spawn() can be broken
>> too, considering its late release (1999).
>Well, it does have to be compatible with existing functions.
>But I sort of agree; I don't quite understand why they still
>insisted on doing it backwards, and forcing everything in one
>function, rather than creating a process without starting it,
>then allowing you to "set it up" before starting it.
Oh you mean like fork() and exec()?
David Schwartz
> > Of course it can. If it doesn't set close-on-exec, its
> > descriptors are guaranteed to be open in the exec'd process.
> If it doesn't know about the process, it can't count on it being
> there, much less on its file descriptors being open. You're
> just being silly.
Huh? It can count on the file descriptors being there in any process,
whether it knows about the exec or not. How am I being silly?
> > The process you exec may expect this case, and the very first
> > thing it might do is call into the library to 'pick up' its
> > file descriptors.
> If the library exec's to a specific process, yes. If you exec
> to 'ls', for example, no.
Did you miss the "may" in there? I am not saying this has to happen.
I'm simply saying that it can happen, and if it does happen, it's
perfectly reasonable. Breaking it is not.
DS
Golden California Girls
> Uh, sorry, it doesn't work that way. If you ask how something is
> possible, then I have to explain how it's possible. I explained how
> it's possible.
>
> Sure, you can make up some other case where it's not possible. But so
> what? I never argued that this always happens, simply that it can
> happen and should be allowed to happen and therefore a scheme that
> assumes it can't happen is broken.
I asked how it was possible in all cases, or have you forgotten it was you that
insisted the library lives on in my process even after exec.
> Great, this proves that there exists
> at least one case where your proposed method might work. Wonderful.
I'm not proposing a method. I'm telling you that I control my process, not the
library and I dang well know, not the library, that files should be closed on
exec. I control my process space and the library had better well be able to
deal with that fact. It means I get all signals, I control all masks, I control
if files are open, I control where the thread of control goes. If the library
makes assumptions on these it better be able to handle the side effects or
document what it needs.
>>> It doesn't matter, the application doesn't (shouldn't) care about
>>> library internals.
>
>> That's correct, all files in the library opens in the user's process, behind the
>> users back, should be marked close on exec. It really shouldn't be opening
>> anything behind the callers back in his process, but that is beside the point.
>
> Huh? The whole point of a library is to "just work". If this means
> opening file descriptors, great. If this means setting them close-on-
> exec, great. If this means making them persist across an exec, great.
> It's not the application's problem. As far as the application is
> concerned, the library should just be magic.
Were it possible to write a library like that ... It isn't!
Lets make this a little more real to you. We are on system Few_Files which
allows 20 files open in a process. Application A opens one file and calls the
library that opens ten files. Now with the standard 3 we have 14 files open.
Application A is all done and exec's to Application B. Application B is going
to open 12 files but doesn't use the library or even knows of its existance. If
the library leaves its ten files open across the exec the library causes
Application B to fail. Nice design. As I say Application A knows if the files
should be left open or closed, not the library. It isn't possible to write a
magic library.
roger.f...@sun.com
>
> Because the standard says it is so:
>
> If the file_actions argument is not NULL, and specifies any close,
> dup2, or open actions to be performed, and if posix_spawn() or
> posix_spawnp() fails for any of the reasons that would cause close(),
> dup2(), or open() to fail, an error value shall be returned as
> described by close(), dup2(), and open(), respectively (or, if the
> error occurs after the calling process successfully returns, the child
> process shall exit with exit status 127). An open file action may, by
> itself, result in any of the errors described by close() or dup2(), in
> addition to those described by open().
Ok Casper, I'm going to do some standards legalism here:
The spec (above) says:, literally:
if posix_spawn() or posix_spawnp() fails for any
of the reasons that would cause close(), ... to fail,
It doesn't say that posix_spawn() *must* fail if it is
asked to close a file descriptor that is not open.
It says that *if* it fails due to a failing close(), it will
return the error value as described for close().
The implementation can legally choose for posix_spawn()
*not* to fail in this case.
IMO, this would be both rational and useful.
Roger Faulkner
Sun Microsystems
David Schwartz
wrote:
> David Schwartz wrote:
> > Uh, sorry, it doesn't work that way. If you ask how something is
> > possible, then I have to explain how it's possible. I explained how
> > it's possible.
> > Sure, you can make up some other case where it's not possible. But so
> > what? I never argued that this always happens, simply that it can
> > happen and should be allowed to happen and therefore a scheme that
> > assumes it can't happen is broken.
> I asked how it was possible in all cases, or have you forgotten it was you that
> insisted the library lives on in my process even after exec.
I explained how -- the processes that is exec'd calls into the
library. Are you saying that's impossible?!
> > Great, this proves that there exists
> > at least one case where your proposed method might work. Wonderful.
> I'm not proposing a method. I'm telling you that I control my process, not the
> library and I dang well know, not the library, that files should be closed on
> exec.
There is no way you can know whether a file that the library opened
should or should not exist after the exec. You don't know what that
file *does*, and that's what matters.
> I control my process space and the library had better well be able to
> deal with that fact. It means I get all signals, I control all masks, I control
> if files are open, I control where the thread of control goes. If the library
> makes assumptions on these it better be able to handle the side effects or
> document what it needs.
And maybe it does. Again, your argument has to be that this is
impossible.
> > Huh? The whole point of a library is to "just work". If this means
> > opening file descriptors, great. If this means setting them close-on-
> > exec, great. If this means making them persist across an exec, great.
> > It's not the application's problem. As far as the application is
> > concerned, the library should just be magic.
> Were it possible to write a library like that ... It isn't!
> Lets make this a little more real to you. We are on system Few_Files which
> allows 20 files open in a process. Application A opens one file and calls the
> library that opens ten files. Now with the standard 3 we have 14 files open.
> Application A is all done and exec's to Application B. Application B is going
> to open 12 files but doesn't use the library or even knows of its existance. If
> the library leaves its ten files open across the exec the library causes
> Application B to fail. Nice design. As I say Application A knows if the files
> should be left open or closed, not the library. It isn't possible to write a
> magic library.
Again, you've explained *one* case where this technique won't work. So
what? There may be a billion cases where a technique doesn't work. My
point is that there can exist cases where it does work, and in those
cases, it should be allowed to work. Breaking it is wrong, because it
will break the case where it works.
DS
Golden California Girls
>
> Again, you've explained *one* case where this technique won't work. So
> what? There may be a billion cases where a technique doesn't work. My
> point is that there can exist cases where it does work, and in those
> cases, it should be allowed to work. Breaking it is wrong, because it
> will break the case where it works.
Allowing it breaks other cases. Use your own words about breaking things.
David Schwartz
wrote:
> David Schwartz wrote:
It does not break anything if it is correctly implemented. It is
possible to implement it correctly.
DS
Mark Wooding
> That's is correct. The target of an exec has to close any file
> descriptors it doesn't plan to use.
No, that's not right. I can see all sorts of good reasons to want a
process to inherit a file descriptor that it's not expecting to use
itself.
* I might just hand it the read end of a pipe, as a way of finding out
when it and all its progeny have finished.
* I might hand it a pipe or something, leave a handy dropping in the
environment, and expect some child that it creates to make use of
this file descriptor later.
Basically, it's not right to randomly close file descriptors just
because you don't know what they're doing. Someone else might have put
them there for good reasons that you don't know about.
-- [mdw]
David Schwartz
> Basically, it's not right to randomly close file descriptors just
> because you don't know what they're doing. Someone else might have put
> them there for good reasons that you don't know about.
Then you have an insoluble problem. If you encounter a file descriptor
you don't know about, there's two possibilities:
1) You must close it. It was created by a race between 'socket' and
'fcntl(FD_CLOEXEC)'. If it stays open for the life of your process,
things will break, as the port it's bound to will remain unusable.
2) You must not close it. It was put there for good reasons that you
don't know about.
So what do you do? WHAT DO YOU DO?!
DS
Golden California Girls
And now we have come full circle. Right back where the OP started, the
libraries he has are not implemented correctly and the people who write them
don't get it, they think leaving files open across exec in the user process is
fine, but I just showed how, and you agreed that it breaks code.
Rainer Weikusat
> David Schwartz <dav...@webmaster.com> writes:
>
>> That's is correct. The target of an exec has to close any file
>> descriptors it doesn't plan to use.
>
> No, that's not right. I can see all sorts of good reasons to want a
> process to inherit a file descriptor that it's not expecting to use
> itself.
>
> * I might just hand it the read end of a pipe, as a way of finding out
> when it and all its progeny have finished.
This could even actually work :->. This process could continously
write to the pipe until it blocks and closing of the other descriptor
would either result in SIGPIE or a failed write with errno == EPIPE.
> * I might hand it a pipe or something, leave a handy dropping in the
> environment, and expect some child that it creates to make use of
> this file descriptor later.
>
> Basically, it's not right to randomly close file descriptors just
> because you don't know what they're doing. Someone else might have put
> them there for good reasons that you don't know about.
That's quite an elastic argument: Anything implemented in code could
be something someone would rather avoid.
Mark Wooding
> Mark Wooding <m...@distorted.org.uk> writes:
>
>> * I might just hand it the read end of a pipe, as a way of finding out
>> when it and all its progeny have finished.
>
> This could even actually work :->.
Actually, I meant to give it the write end of the pipe and wait for it
to be readable (at EOF). Sorry for the typo.
I think I might have actually used this trick at some point. (No, I'm
not particularly proud of it.)
>> Basically, it's not right to randomly close file descriptors just
>> because you don't know what they're doing. Someone else might have put
>> them there for good reasons that you don't know about.
>
> That's quite an elastic argument: Anything implemented in code could
> be something someone would rather avoid.
Yes. I'm just sending out a warning that `I don't know what this thing
is for' doesn't mean the same as `I should make this thing go away'. If
you don't know what it's for, then you just don't know whether it's
meant to be there (for some purpose that you're unaware of) or whether
it's a mistake.
If I had to choose, I'd take what seems to me to be the conservative
assumption that whoever left a file descriptor lying about for me to
inherit really meant for it to be there, and I shouldn't mess with it.
This just seems less rash than assuming that it's a mistake to be
rectified.
We can have an argument over signal dispositions next. Suppose that
SIGPIPE or SIGCHLD is set to SIG_IGN; should you switch it back again?
What about SIG_SYSTEM_SPECIFIC_THAT_YOUVE_NEVER_HEARD_OF?
-- [mdw]
Rainer Weikusat
> Rainer Weikusat <rwei...@mssgmbh.com> writes:
>> Mark Wooding <m...@distorted.org.uk> writes:
[...]
>>> Basically, it's not right to randomly close file descriptors just
>>> because you don't know what they're doing. Someone else might have put
>>> them there for good reasons that you don't know about.
>>
>> That's quite an elastic argument: Anything implemented in code could
>> be something someone would rather avoid.
>
> Yes. I'm just sending out a warning that `I don't know what this thing
> is for' doesn't mean the same as `I should make this thing go away'. If
> you don't know what it's for, then you just don't know whether it's
> meant to be there (for some purpose that you're unaware of) or whether
> it's a mistake.
The problem with this approach is that the only things where certainly
nobody will ever understand what there purpose was supposed to be are
those which have none because they were mistakes. To me, your
statements sounds a lot like "don't fix it just because it is broken,
adjust your expectations instead".
Xavier Roche
To close this nearly two years-old thread for lurkers, Eric Blake gave a
nice solution to mimic the "close all fd's from 3 to MAX_FD" with
posix_spawn().
The idea is simply to use posix_spawn_file_actions_adddup2() to dup2()
an existing handle (say, stdin) to every single fd's, closing or
remapping the target fd, and then close this it safely.
Rationale:
<http://www.opengroup.org/onlinepubs/009695399/functions/posix_spawn_file_actions_adddup2.html>
The posix_spawn_file_actions_adddup2() function shall add a dup2()
action to the object referenced by file_actions (..) as if
dup2( fildes, newfildes) had been called) when a new process is spawned
using this file actions object.
<http://www.opengroup.org/onlinepubs/9699919799/functions/dup2.html>
(..) If fildes2 is already a valid open file descriptor, it shall be
closed first (..)
Hence, to mimic the (quite dirty) post-fork() closing, loop:
const int fdmax = (int) sysconf(_SC_OPEN_MAX);
(.. initialize posix_spawn_file_actions_t actions)
for (fd = 3 ; fd < fdmax ; fd++) {
const int fakeFd = 0; /* map everything to stdin */
posix_spawn_file_actions_adddup2(&actions, fakeFd, fd);
posix_spawn_file_actions_addclose(&actions, fd);
}
(..continue with more posix_spawn_file_actions_adddup2() for stdio)
[The best solution, of course, is to ensure in every file
descriptor-related function (fopen()/open()/socket() ...) that the
CLOEXEC flag is used. But this feature is often only available in recent
systems..]
Casper H.S. Dik
>To close this nearly two years-old thread for lurkers, Eric Blake gave a
>nice solution to mimic the "close all fd's from 3 to MAX_FD" with
>posix_spawn().
I think it's a bug that posix_spawn_file_actions_addclose(&actions, fd)
causes a failure if the file is already closed during posix_spawn.
Xavier Roche
> I think it's a bug that posix_spawn_file_actions_addclose(&actions, fd)
> causes a failure if the file is already closed during posix_spawn.
The spec is anything but clear regarding what should happend if a
close() triggered by posix_spawn_file_actions_addclose() fails, but
suggests that this is fatal to the posix_spawn() call itself:
<http://www.opengroup.org/onlinepubs/009695399/functions/posix_spawn.html>
(..)
Casper H.S. Dik
>Casper H.S. Dik a écrit :
>> I think it's a bug that posix_spawn_file_actions_addclose(&actions, fd)
>> causes a failure if the file is already closed during posix_spawn.
>The spec is anything but clear regarding what should happend if a
>close() triggered by posix_spawn_file_actions_addclose() fails, but
>suggests that this is fatal to the posix_spawn() call itself:
That is correct; but it's a bug in the standard. It makes posix_spawn
nearly unusable considering that you may need to add an expensive
workaround (duping and closing 65000 descriptors isn't going to be
cheap)
Xavier Roche
> nearly unusable considering that you may need to add an expensive
> workaround (duping and closing 65000 descriptors isn't going to be
> cheap)
Yep ; the closing phase is far the most expensive one, unfortunately.
Some people would suggest closefrom() to be standardized (and an
additional action added):
https://www.opengroup.org/sophocles/show_mail.tpl?CALLER=show_archive.tpl&source=L&listname=austin-group-l&id=15102
See the SunOS doc:
http://docs.sun.com/app/docs/doc/816-5168/closefrom-3c?a=view