Practice of using fork()
Ulrich Hobelmann
some ways having one executable is nicer than having several of
them. So I thought that maybe I should avoid exec and just fork
my process and let one branch return, while the other branch
executes another function (instead of execing another image).
The problem: file descriptors. I certainly don't want my forked
process to hang on too files the other process opened, and
close-on-exec is nice, but doesn't apply to fork without exec.
What's the Unix practice here? Do people generally avoid fork
without exec? (Then I wonder why fork exists, instead of having
something like spawn_process(image, params, ...))
Or do they keep a global list of open file descriptors and close
these after the fork?
Well, maybe this style of programming is really crying for threads
instead of processes, I don't know. So far I like the messaging
approach of write/select/read and really want to save myself the
hassle of shared-memory, fun-with-debugging-and-synchronization
threads.
--
XML is a prime example of retarded innovation.
-- Erik Meijer and Peter Drayton, Microsoft Corporation
David Schwartz
"Ulrich Hobelmann" <u.hob...@web.de> wrote in message
news:3k3qbcF...@individual.net...
> I'm thinking about different ways of program organization, and in some
> ways having one executable is nicer than having several of them. So I
> thought that maybe I should avoid exec and just fork my process and let
> one branch return, while the other branch executes another function
> (instead of execing another image).
That's not unreasonable.
> The problem: file descriptors. I certainly don't want my forked process
> to hang on too files the other process opened, and close-on-exec is nice,
> but doesn't apply to fork without exec.
Close all file descriptors you don't want.
> What's the Unix practice here? Do people generally avoid fork without
> exec? (Then I wonder why fork exists, instead of having something like
> spawn_process(image, params, ...))
There are a lot of reasons that 'fork' and 'exec' are two functions. One
is some people want to 'fork' without 'exec'ing. Another is that you often
need to do things in-between the 'fork' and the 'exec'.
> Or do they keep a global list of open file descriptors and close these
> after the fork?
You don't need a global list, just close every descriptor you don't
want. You can call 'getdtablesize' to get the limit. Then just loop and
close every one you aren't going to use.
> Well, maybe this style of programming is really crying for threads instead
> of processes, I don't know. So far I like the messaging approach of
> write/select/read and really want to save myself the hassle of
> shared-memory, fun-with-debugging-and-synchronization threads.
If you don't share things across threads, you don't need to do anything
special in them just because your process has multiple threads. However, you
will take some penalties in things like 'malloc'.
DS
Ulrich Hobelmann
> You don't need a global list, just close every descriptor you don't
> want. You can call 'getdtablesize' to get the limit. Then just loop and
> close every one you aren't going to use.
Hm, and that just works? I thought that getdtablesize gives you
the size of the table, not necessarily how many files there are
open (and I don't want to loop through 1000s of entries ;) ), and
that close() gives an error if you don't pass it an open file (so
while that works, it's not really clean style, and makes lots of
unneeded system calls).
> If you don't share things across threads, you don't need to do anything
> special in them just because your process has multiple threads. However, you
> will take some penalties in things like 'malloc'.
Hm, maybe that would be time to dig my own malloc out again, and
port it to use mmap()... :)
David Schwartz
> David Schwartz wrote:
>> You don't need a global list, just close every descriptor you don't
>> want. You can call 'getdtablesize' to get the limit. Then just loop and
>> close every one you aren't going to use.
> Hm, and that just works? I thought that getdtablesize gives you the size
> of the table, not necessarily how many files there are open (and I don't
> want to loop through 1000s of entries ;) ), and that close() gives an
> error if you don't pass it an open file (so while that works, it's not
> really clean style, and makes lots of unneeded system calls).
It's not elegant, but there's no alternative. The system calls are
needed.
>> If you don't share things across threads, you don't need to do
>> anything special in them just because your process has multiple threads.
>> However, you will take some penalties in things like 'malloc'.
> Hm, maybe that would be time to dig my own malloc out again, and port it
> to use mmap()... :)
If you need a high-performance multi-threaded malloc, hoard is out
there.
DS
Casper H.S. Dik
>David Schwartz wrote:
>> You don't need a global list, just close every descriptor you don't
>> want. You can call 'getdtablesize' to get the limit. Then just loop and
>> close every one you aren't going to use.
>Hm, and that just works? I thought that getdtablesize gives you
>the size of the table, not necessarily how many files there are
>open (and I don't want to loop through 1000s of entries ;) ), and
>that close() gives an error if you don't pass it an open file (so
>while that works, it's not really clean style, and makes lots of
>unneeded system calls).
It's what people used to do when the number of fds was small;
now that apps can have thousands, this is not always feasible.
Solaris now has "closefrom()" which efficiently closes all
descriptors above a certain number and fdwalk which gives you
a callback for each open fd.
Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.
Barry Margolin
Ulrich Hobelmann <u.hob...@web.de> wrote:
> David Schwartz wrote:
> > You don't need a global list, just close every descriptor you don't
> > want. You can call 'getdtablesize' to get the limit. Then just loop and
> > close every one you aren't going to use.
>
> Hm, and that just works? I thought that getdtablesize gives you
> the size of the table, not necessarily how many files there are
> open (and I don't want to loop through 1000s of entries ;) ), and
> that close() gives an error if you don't pass it an open file (so
> while that works, it's not really clean style, and makes lots of
> unneeded system calls).
You basically have two choices: keep a list of all the files that were
opened so you know specifically which ones to close, or just try to
close everything and ignore the errors.
There's no "close-on-fork" flag, as you pointed out, so there's no way
for the system to automatically tell which desciptors you want to share
between the parent and child and which you don't, so it can't automate
this for you.
--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
Mr. Uh Clem
> "Ulrich Hobelmann" <u.hob...@web.de> wrote in message
> news:3k3qbcF...@individual.net...
...
>>The problem: file descriptors. I certainly don't want my forked process
>>to hang on too files the other process opened, and close-on-exec is nice,
>>but doesn't apply to fork without exec.
>
>
> Close all file descriptors you don't want.
>
As I recently learned and was discussed in the the thread
"password (& host?) file lookups, daemons and stray descriptors"
<42bb3ac4$0$5702$9a6e...@news.newshosting.com> (June 26),
some library routines such as getpwuid() open descriptors and
leave them open, with static references to them in the library.
When you fork, the reference in the library is duplicated too.
Reusing the fd that the library still thinks it has can cause
trouble.
The common wisdom of indiscriminately closing all fds you
don't care about after a fork (as shown in many examples of
how to start a daemon) seems wrong. The only place you can get
away with this is at the start of your freshly exec'd program,
or just prior to exec'ing another program.
--
Clem
"If you push something hard enough, it will fall over."
- Fudd's first law of opposition
David Schwartz
"Mr. Uh Clem" <uhc...@DutchElmSt.invalid> wrote in message
news:42de4972$0$10990$9a6e...@news.newshosting.com...
>> Close all file descriptors you don't want.
> As I recently learned and was discussed in the the thread
> "password (& host?) file lookups, daemons and stray descriptors"
> <42bb3ac4$0$5702$9a6e...@news.newshosting.com> (June 26),
> some library routines such as getpwuid() open descriptors and
> leave them open, with static references to them in the library.
> When you fork, the reference in the library is duplicated too.
> Reusing the fd that the library still thinks it has can cause
> trouble.
You can't call functions like 'getpwuid' after you 'fork'.
> The common wisdom of indiscriminately closing all fds you
> don't care about after a fork (as shown in many examples of
> how to start a daemon) seems wrong. The only place you can get
> away with this is at the start of your freshly exec'd program,
> or just prior to exec'ing another program.
What would you suggest?
DS
Gordon Burditt
>
>> As I recently learned and was discussed in the the thread
>> "password (& host?) file lookups, daemons and stray descriptors"
>> <42bb3ac4$0$5702$9a6e...@news.newshosting.com> (June 26),
>> some library routines such as getpwuid() open descriptors and
>> leave them open, with static references to them in the library.
>> When you fork, the reference in the library is duplicated too.
>> Reusing the fd that the library still thinks it has can cause
>> trouble.
>
> You can't call functions like 'getpwuid' after you 'fork'.
What functions are "like getpwuid"? How do I identify them? Any
function that leaves files open? Does that include fopen() and
open()? It seems to me that "like getpwuid" depends too much on
internal implementation and not on the interface or documentation.
Gordon L. Burditt
David Schwartz
"Gordon Burditt" <gordon...@burditt.org> wrote in message
news:11dtdaj...@corp.supernews.com...
>>>> Close all file descriptors you don't want.
>> You can't call functions like 'getpwuid' after you 'fork'.
> What functions are "like getpwuid"? How do I identify them? Any
> function that leaves files open? Does that include fopen() and
> open()? It seems to me that "like getpwuid" depends too much on
> internal implementation and not on the interface or documentation.
Any function that is not known to be safe between 'fork' and 'exec' must
not be called. You are very limited in what you can safely do between a
'fork' and an 'exec', just as you are in many other contexts such as signal
handlers. If, after a 'fork', neither process calls 'exec', you have a lot
of potential problems with shared structures. That's just a fact.
Sorry. I wish it wasn't so.
DS
Barry Margolin
"David Schwartz" <dav...@webmaster.com> wrote:
But what if you never exec()? I think you're confusing fork() with
vfork(). The latter is only supposed to be used if you plan on calling
exec() almost immediately.
Gordon Burditt
>
>>> You can't call functions like 'getpwuid' after you 'fork'.
>
>> What functions are "like getpwuid"? How do I identify them? Any
>> function that leaves files open? Does that include fopen() and
>> open()? It seems to me that "like getpwuid" depends too much on
>> internal implementation and not on the interface or documentation.
>
> Any function that is not known to be safe between 'fork' and 'exec' must
>not be called. You are very limited in what you can safely do between a
>'fork' and an 'exec', just as you are in many other contexts such as signal
>handlers. If, after a 'fork', neither process calls 'exec', you have a lot
>of potential problems with shared structures. That's just a fact.
What "shared structures"? The problem with getpwuid() is a problem
with shared file descriptors (which share a common file pointer),
or with closing a library routine's file descriptor behind it's back.
Where does fork() create memory sharing? It makes COPIES, logically,
anyway. It doesn't create shared memory where a write by one is seen
by the other process (copy-on-write, if used, tears down the sharing
when one process writes).
Gordon L. Burditt
David Schwartz
"Barry Margolin" <bar...@alum.mit.edu> wrote in message
news:barmar-62331C....@comcast.dca.giganews.com...
> But what if you never exec()?
If you never 'exec' and keep running in both processes, there are quite
a few things you can't (portably, with guaranteed safety) do.
> I think you're confusing fork() with
> vfork(). The latter is only supposed to be used if you plan on calling
> exec() almost immediately.
There are more problems with 'vfork', but there are problems with 'fork'
as well. Consider DNS functions like 'gethostbyname' that might use a
socket. After 'fork', both processes are trying to use the same TCP
connection.
DS
David Schwartz
> What "shared structures"? The problem with getpwuid() is a problem
> with shared file descriptors (which share a common file pointer),
> or with closing a library routine's file descriptor behind it's back.
The file descriptor table is a structure. After the 'fork' it is shared
(though modifications are not propogated).
> Where does fork() create memory sharing? It makes COPIES, logically,
> anyway. It doesn't create shared memory where a write by one is seen
> by the other process (copy-on-write, if used, tears down the sharing
> when one process writes).
I didn't say it created memory sharing. I said that after a fork there
are shared structures. One of them is the table of file descriptors.
The problem is that everything should be copy on write after the 'fork'.
But some thing can't be copy on write, like file descriptors. And some
things that could be copy on write might not be so simply because the
underlying libraries don't realize, like a client identifier (issued by a
server, generated randomly, or whatever) that's expected to be unique for
each client.
DS
Barry Margolin
"David Schwartz" <dav...@webmaster.com> wrote:
Good point. Unfortunately, there's no list of "fork-safe" functions,
it's totally implementation dependent.
Villy Kruse
Gordon Burditt <gordon...@burditt.org> wrote:
>
> What "shared structures"? The problem with getpwuid() is a problem
> with shared file descriptors (which share a common file pointer),
> or with closing a library routine's file descriptor behind it's back.
>
If you call endpwent() when you are done reading password file the
file is closed. If need be you can then re-open the password file
after fork.
Same for calling endhostent() after gethostbyxxx() and about a handfull
of other endxxxent() functions.
Villy
Henry Townsend
> There are more problems with 'vfork', but there are problems with 'fork'
> as well. Consider DNS functions like 'gethostbyname' that might use a
> socket. After 'fork', both processes are trying to use the same TCP
> connection.
True, but you've changed the rules in the middle of the game. Your
previous statement was about what can be done "between fork and exec",
i.e. *after* the fork. Now you're talking about creating a resource
*before* a fork and the problems associated with using it after.
--
Henry Townsend
David Schwartz
"Henry Townsend" <henry.t...@not.here> wrote in message
news:TdGdnf-y6u9...@comcast.com...
> David Schwartz wrote:
Huh?
The presumption is that you have a program that's been running and doing
all kinds of things. Now you want to 'fork' and have neither have call
'exec'. You expect each of the two processes to be able to continue doing
things normally without having a problem.
My point is that many library functions may not work right if you do
this.
DS
Gordon Burditt
>> with shared file descriptors (which share a common file pointer),
>> or with closing a library routine's file descriptor behind it's back.
>
> The file descriptor table is a structure.
It's not a structure in the user's program, and you're assuming things
about the implementation of the OS that are not necessarily true,
although that is a fairly obvious implementation.
>After the 'fork' it is shared
>(though modifications are not propogated).
In the case of the file position indicator for shared file descriptors,
modifications *ARE* propogated, and this can cause problems. (For
example, after the fork(), both processes do a sequential scan of
the password file using getpwent(). They are likely to stomp all
over each other, especially if the library is actually using a flat
file for its lookups. Although if the seeking is done in dbm files,
the problems could be just as bad.)
>> Where does fork() create memory sharing? It makes COPIES, logically,
>> anyway. It doesn't create shared memory where a write by one is seen
>> by the other process (copy-on-write, if used, tears down the sharing
>> when one process writes).
>
> I didn't say it created memory sharing. I said that after a fork there
>are shared structures. One of them is the table of file descriptors.
>
> The problem is that everything should be copy on write after the 'fork'.
>But some thing can't be copy on write, like file descriptors. And some
>things that could be copy on write might not be so simply because the
>underlying libraries don't realize, like a client identifier (issued by a
>server, generated randomly, or whatever) that's expected to be unique for
>each client.
So far, it seems all the problems come from unexpected file descriptor
sharing (which includes sockets connected to servers).
Gordon L. Burditt
David Schwartz
>>> What "shared structures"? The problem with getpwuid() is a problem
>>> with shared file descriptors (which share a common file pointer),
>>> or with closing a library routine's file descriptor behind it's back.
>> The file descriptor table is a structure.
> It's not a structure in the user's program, and you're assuming things
> about the implementation of the OS that are not necessarily true,
> although that is a fairly obvious implementation.
Huh? By definition the file descriptor table is a structure. A table of
things is a type of structure.
>>After the 'fork' it is shared
>>(though modifications are not propogated).
> In the case of the file position indicator for shared file descriptors,
> modifications *ARE* propogated, and this can cause problems. (For
> example, after the fork(), both processes do a sequential scan of
> the password file using getpwent(). They are likely to stomp all
> over each other, especially if the library is actually using a flat
> file for its lookups. Although if the seeking is done in dbm files,
> the problems could be just as bad.)
Right.
>> The problem is that everything should be copy on write after the
>> 'fork'.
>>But some thing can't be copy on write, like file descriptors. And some
>>things that could be copy on write might not be so simply because the
>>underlying libraries don't realize, like a client identifier (issued by a
>>server, generated randomly, or whatever) that's expected to be unique for
>>each client.
> So far, it seems all the problems come from unexpected file descriptor
> sharing (which includes sockets connected to servers).
That is far from the only problem. Another example of a type of problem
is PID caching.
DS
Gordon Burditt
>>>> with shared file descriptors (which share a common file pointer),
>>>> or with closing a library routine's file descriptor behind it's back.
>
>>> The file descriptor table is a structure.
>
>> It's not a structure in the user's program, and you're assuming things
>> about the implementation of the OS that are not necessarily true,
>> although that is a fairly obvious implementation.
>
> Huh? By definition the file descriptor table is a structure.
Unless, of course, it's NOT a table, but it's a file descriptor SQL
database, or a file descriptor linked list, or a file descriptor
btree, or whatever. Where are you guaranteed that there *IS* a "file
descriptor table"? In any case, it's not part of the user's program.
>A table of
>things is a type of structure.
I'd think a table of things is a type of ARRAY. Possibly, even
usually, an array of structures.
Gordon L. Burditt
Barry Margolin
gordon...@burditt.org (Gordon Burditt) wrote:
> >>>> What "shared structures"? The problem with getpwuid() is a problem
> >>>> with shared file descriptors (which share a common file pointer),
> >>>> or with closing a library routine's file descriptor behind it's back.
> >
> >>> The file descriptor table is a structure.
> >
> >> It's not a structure in the user's program, and you're assuming things
> >> about the implementation of the OS that are not necessarily true,
> >> although that is a fairly obvious implementation.
> >
> > Huh? By definition the file descriptor table is a structure.
>
> Unless, of course, it's NOT a table, but it's a file descriptor SQL
> database, or a file descriptor linked list, or a file descriptor
> btree, or whatever. Where are you guaranteed that there *IS* a "file
> descriptor table"? In any case, it's not part of the user's program.
All those are types of structures, in the general sense of the word.
They're not "C structs", but that's irrelevant.
David Schwartz
>> Unless, of course, it's NOT a table, but it's a file descriptor SQL
>> database, or a file descriptor linked list, or a file descriptor
>> btree, or whatever. Where are you guaranteed that there *IS* a "file
>> descriptor table"? In any case, it's not part of the user's program.
> All those are types of structures, in the general sense of the word.
> They're not "C structs", but that's irrelevant.
And though the file descriptor table is not part of the program in the
sense of the executable itself, it is part of the program in the sense of
the running instance of the program.
DS
j...@invalid.address
> "Gordon Burditt" <gordon...@burditt.org> wrote in message
> news:11dvmf2...@corp.supernews.com...
> > So far, it seems all the problems come from unexpected file
> > descriptor sharing (which includes sockets connected to servers).
>
> That is far from the only problem. Another example of a type of
> problem is PID caching.
We have our disagreements, but I mostly agree with you in this thread
except that I think you stated your position a little too strongly
when you said "You can't call functions like 'getpwuid' after you
'fork'".
Surely you can, if you know what to do first. It's perfectly
reasonable to call fork() without calling exec(), and do some pretty
non-trivial stuff. The trick is knowing how things work on the
platform being used. That could be said of most things in
programming.
Joe
Barry Margolin
wrote:
> Surely you can, if you know what to do first. It's perfectly
> reasonable to call fork() without calling exec(), and do some pretty
> non-trivial stuff. The trick is knowing how things work on the
> platform being used. That could be said of most things in
> programming.
The problem is that it can be difficult to perform this trick, since the
issues that have been brought up are almost universally undocumented.
As a result, they're also subject to change from release to release.
Furthermore, POSIX programming is supposed to be portable. Does POSIX
ever suggest that there's a problem calling any of these functions
across forks? This is a pretty annoying gotcha.
David Schwartz
<j...@invalid.address> wrote in message
news:m3irz3k...@localhost.localdomain...
>> That is far from the only problem. Another example of a type of
>> problem is PID caching.
> Surely you can, if you know what to do first. It's perfectly
> reasonable to call fork() without calling exec(), and do some pretty
> non-trivial stuff. The trick is knowing how things work on the
> platform being used. That could be said of most things in
> programming.
When I say "you can't", I mean portably.
DS
j...@invalid.address
j...@invalid.address
> In article <m3irz3k...@localhost.localdomain>, j...@invalid.address
> wrote:
>
> > Surely you can, if you know what to do first. It's perfectly
> > reasonable to call fork() without calling exec(), and do some
> > pretty non-trivial stuff. The trick is knowing how things work on
> > the platform being used. That could be said of most things in
> > programming.
>
> The problem is that it can be difficult to perform this trick, since
> the issues that have been brought up are almost universally
> undocumented. As a result, they're also subject to change from
> release to release.
>
> Furthermore, POSIX programming is supposed to be portable. Does
> POSIX ever suggest that there's a problem calling any of these
> functions across forks? This is a pretty annoying gotcha.
I think that's not the only annoying gotcha in the POSIX, but that's
what defect reports are for.
Be that as it may, as I understand it, apache does pretty much what
we're talking about here. The program that originally gets run opens a
listening port and then forks a bunch of child processes which all
call accept() to handle incoming client connections, letting the
system do some of the load balancing. If I've got that right it seems
to be doable.
I'm not trying to say it's easy to accomplish, only that I think
"can't" is too strong a word for it.
Joe
David Schwartz
<j...@invalid.address> wrote in message
news:m38xzzk...@localhost.localdomain...
> "David Schwartz" <dav...@webmaster.com> writes:
>> When I say "you can't", I mean portably.
> I don't see how the two are equivalent.
I'm presuming people want to be assured their code will work, even if
it's run on a different system, even when the next version of the library
comes out. Otherwise, what's the point?
DS
Kenny McCormack
Welcome to the Humpty Dumpty world of Usenet, wherein people define terms
as they like (and assume, as HD did, that as long as they are consistent in
their usage, they're good-to-go). Try hanging around comp.lang.c for
a while and you'll see it in spades.
j...@invalid.address
Lots of people don't need that assurance.
But the point I was trying to make is that you can call getpwuid after
a fork portably, that's what the preprocessor is for. If I had to
choose between the standard and autoconf I'd probably pick autoconf.
Joe
David Schwartz
"Kenny McCormack" <gaz...@yin.interaccess.com> wrote in message
news:dbqrh5$m6g$1...@yin.interaccess.com...
>>> When I say "you can't", I mean portably.
>>I don't see how the two are equivalent.
> Welcome to the Humpty Dumpty world of Usenet, wherein people define terms
> as they like (and assume, as HD did, that as long as they are consistent
> in
> their usage, they're good-to-go). Try hanging around comp.lang.c for
> a while and you'll see it in spades.
The standard says what you can and can't do. That's its purpose. We're
talking about a function that is standardized by posix.
DS
Kenny McCormack
I will grant that you, Humpty Dumpty, Tweedledee, and Tweedledum are.
The rest of us will just get on with our lives, thankyouverymuch.
Mr. Uh Clem
Unfortunately, endpwent() did not close the socket opened by
getpwuid(). (Having the endxxxxent() functions do such a
clean-up would be a rather rational thing to do.)
--
Clem
"If you push something hard enough, it will fall over."
- Fudd's first law of opposition
Barry Margolin
"Mr. Uh Clem" <uhc...@DutchElmSt.invalid> wrote:
> Villy Kruse wrote:
>
> > On Thu, 21 Jul 2005 01:57:01 -0000,
> > Gordon Burditt <gordon...@burditt.org> wrote:
> >
> >
> >
> >>What "shared structures"? The problem with getpwuid() is a problem
> >>with shared file descriptors (which share a common file pointer),
> >>or with closing a library routine's file descriptor behind it's back.
> >>
> >
> >
> > If you call endpwent() when you are done reading password file the
> > file is closed. If need be you can then re-open the password file
> > after fork.
> >
> > Same for calling endhostent() after gethostbyxxx() and about a handfull
> > of other endxxxent() functions.
> >
> > Villy
>
> Unfortunately, endpwent() did not close the socket opened by
> getpwuid(). (Having the endxxxxent() functions do such a
> clean-up would be a rather rational thing to do.)
The problem is probably that it's using NIS, and there's no call to tell
NIS that it should close its socket.