fork() on NT! or info on porting Unix apps to NT
hamilton on BIX
>I am porting an Unix application to NT and I was wondering if anyone
>might know how I can simulate or write the fork() call on NT? Exec()
>and spawn() is not quite what I am looking for. Should I consider using
>threads?
Yes, you should consider using threads, though realistically, rewriting
an application that's architected around a fork primitive to use threads
instead may be more work than is really feasible. With a fork, parent
and child get their own copies of everything, snapshotted at the time of
the fork. From there on, each can scribble over anything they like without
concern for the effects on the other. That's not the case with threads,
obviously, where you need to consider locking down critical resources
with semaphores before you make changes, etc.
One alternative you may also wish to consider is a product called
NuTCRACKER (yes, the lower case "u" is intentional) from DataFocus,
Inc. (ph 703-631-6770). This product is still in beta (to the best
of my knowledge) but includes a porting library that simulates a fork
on top of Win32. Around Christmas I talked with David Korn (author of
the Korn shell) who told me he'd successfully used the NuTCRACKER
library to port his shell. His comment was that the fork implementation
was pretty slow (since they don't get any help from the OS to do sharing
of page-table entries to do a copy-on-write implementation as all good
UNIX systems do these day and must, instead, do a wholesale copy of the
entire process space) but did work well enough to get things going.
Regards,
Doug Hamilton hami...@bix.com Ph 508-358-5715
Hamilton Laboratories, 13 Old Farm Road, Wayland, MA 01778-3117
Kent Yang
Howdy,
I am porting an Unix application to NT and I was wondering if anyone
might know how I can simulate or write the fork() call on NT? Exec()
and spawn() is not quite what I am looking for. Should I consider using
threads? Or if you know of some ftp site that has either tips, or
FAQ on porting Unix apps to NT, can you send that information to me.
My email address is: ke...@li.loral.com. Thanks in advance.
--
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Kent Yang - Design Engineer - ke...@li.loral.com / Go
Engineering, Loral Instrumentation (San Diego, CA) / Sharks!!!
Phone: (619)674-5100 ext 4180 Fax: (619)674-5145 ==
Jbair
HA>Newsgroup: comp.os.ms-windows.programmer.misc
HA>comp.os.ms-windows.programmer.win32,comp.os.ms-windows.nt.misc
HA>comp.sys.ibm.pc.programmer
HA>Organization: Delphi Internet Services Corporation
HA>ke...@loral.cts.com (Kent Yang) writes:
HA>>I am porting an Unix application to NT and I was wondering if anyone
HA>>might know how I can simulate or write the fork() call on NT? Exec()
HA>>and spawn() is not quite what I am looking for. Should I consider using
HA>>threads?
Consensys has ported genuine Unix SVR4 to NT. I have not yet tried it
but it looks really good. They have over 100 Unix SVR4 utilites compiled
from the original source code; a native NT GUI interface to manage
multiple shells (ksh) and a dialog-box interface for every Unix command;
online SVR4 man pages in windows help format. They also have an SDK
which inclues fork() and all other Unix SVR4 system services and
function calls. They claim availability on Intel, MIPS and Alpha
systems. They are at 1-800-388-1986. Prices are $395 for the Unix
utilities, $495 for the SDK, or $695 for the whole mess.
---
ş SLMR 2.1a ş All hope abandon, ye who enter messages here.
Jbair
Arthur Kreitman
> >I am porting an Unix application to NT and I was wondering if anyone
> >might know how I can simulate or write the fork() call on NT? Exec()
> >and spawn() is not quite what I am looking for. Should I consider using
> >threads?
>
> Yes, you should consider using threads, though realistically, rewriting
> an application that's architected around a fork primitive to use threads
> instead may be more work than is really feasible. With a fork, parent
Almost every app I've seen uses fork() as part of a fork/exec sequence.
Its a rare app that really needs fork(). The unix shell are one example.
Bash, sh, ksh88, and csh would require almost a complete rewrite to
replace fork with spawns or threads (lucky for you doug). One company
I know of wrote an app that uses fork() to save process status
for checkpoint/restart. For the rest of them can you can replace fork/exec
with spawn. Just watch out for signals.
> One alternative you may also wish to consider is a product called
> NuTCRACKER (yes, the lower case "u" is intentional) from DataFocus,
You should consider this only if you have a spare $50,000. That what
DataFocus charges.
> library to port his shell. His comment was that the fork implementation
> was pretty slow (since they don't get any help from the OS to do sharing
> of page-table entries to do a copy-on-write implementation as all good
> UNIX systems do these day and must, instead, do a wholesale copy of the
That's dumb of them. NT has to support, and does, have pte's that
include a copy-on-write bit. This is how fork() is implemented in the
otherwise useless posix subsystem.
--
----
Art Kreitman Congruent Corporation
ar...@congruent.com 110 Greene Street
212-431-5100 New York, New York 10012
fax 219-1532
Jonathan Shekter
>
>Howdy,
>
>I am porting an Unix application to NT and I was wondering if anyone
>might know how I can simulate or write the fork() call on NT? Exec()
>and spawn() is not quite what I am looking for. Should I consider using
>threads? Or if you know of some ftp site that has either tips, or
>FAQ on porting Unix apps to NT, can you send that information to me.
>My email address is: ke...@li.loral.com. Thanks in advance.
>
Look carefully at what the fork is doing. There are many, many,
Unix programs that use fork as follows:
if (!fork())
exec(...);
WHY they dow this instead of just spawn()ing is beyond me
(performance reasons, perhaps fork()/exec() is faster than spawn()?), but
many seem to do just that.
Also, why don't you consider using the Posix subsystem under NT?
It does have a fork(), which uses the proper hardware/VM stuff for
efficiency.
Yet Another possibilty is starting another copy of your program,
and jumping to one of two places depending on whether or not it
is a child process. You could use named oibjects, piopes, all sorts
of things to indiciate to the starting process that it is a child...
ther is probably even an API call to check the parent PID. (I'm
sure there is, actually, but I don't know it offhand.) You could
even pass a command-line parameter (really crude!)
I have had this prblem also, but It's not really insurmountable. In
general, I find that Unix programs port very easily, even ones using
things like sockets (via WinSock on NT) or Sys V shared memory (takes
~100 lines of code to write wrapper functions around Win32's MapViewOfFile,
etc., calls.)
Jonathan Shekter, from Sunny Toronto, Canada
Chris Marriott
>
>Howdy,
>
>I am porting an Unix application to NT and I was wondering if anyone
>might know how I can simulate or write the fork() call on NT? Exec()
>and spawn() is not quite what I am looking for. Should I consider using
>threads? Or if you know of some ftp site that has either tips, or
>FAQ on porting Unix apps to NT, can you send that information to me.
>My email address is: ke...@li.loral.com. Thanks in advance.
>
Forgive my ignorance of UNIX, but could someone explain to me what
the "fork" function does, please?
Thanks,
Chris
--
--------------------------------------------------------------------------
| Chris Marriott, Warrington, UK | Author of SkyMap v2 shareware |
| Internet: ch...@chrism.demon.co.uk | astronomy program for Windows. |
| CompuServe: 100113,1140 | Mail me for details! |
| Author member of Association of Shareware Professionals (ASP) |
| Windows, C/C++ consultancy undertaken, anywhere in the world. |
--------------------------------------------------------------------------
Mark Smith
> Forgive my ignorance of UNIX, but could someone explain to me what
> the "fork" function does, please?
Creates a copy of the current process. The old (parent) process gets
a return value equal to the new (child) process ID, the new process
gets a return value of 0. It's like heavyweight multi-threading.
--- Mark Smith ----------------------- msm...@discreet.demon.co.uk ---
hamilton on BIX
>Forgive my ignorance of UNIX, but could someone explain to me what
>the "fork" function does, please?
Fork() creates a child process which is an exact copy of the parent,
including the entire memory space (instructions and data), registers,
any open file handles, etc. The return value to the parent is the
process id of the child; the return value to the child is 0.
Tomas Olovsson
Ok,
So how would I port an application that uses the fact that fork() and
exec() is not an atomic operation? Suppose I would like to execute a
new program having its standard input and/or output connected somewhere
(for example executing the command "xxxx > outfile" or to send an e-mail
with the contents received from my program):
fd = .... /* a file, pipe or socket for example */
if ((pid = fork()) == 0) { /* create a child process */
dup2(fd, 0); /* for the child, assign fd to stdin */
exec(whatever, ....); /* have the child execute another program
keeping stdin connected to the socket */
}
parent code; /* The partent's stdin is untouched */
Could someone give me a simple solution to this problem, for example
with the use of threads?! Please don't give me workarounds such as
stating that I could use named pipes, temporary files etc.
Also, the arguments given earlier indicate that some think (they might not
be serious though ;-) that since fork/exec are POSIX calls, we shouldn't
allow them to be mixed with WIN32 calls for example. Why??? It should really
be my problem to decide what libraries (or subsystems for that matter) I would
like to use in my program. It's almost like stating that if you use X-windows
you are not allowed to use character I/O! I find it hard to see good technical
reasons for not allowing an application to use 2 subsystems at the same time.
(Or I should at least be able to use the POSIX subsystem and system calls
to NT, even though they seem to be unpublished at this date).
Best regards
Tomas
--
___ _ Dept of Computer Engineering
| _ ,___ _ _ | | | _ _ _ _ ,_ Chalmers Univ of Technology
| |_| | | | |_\ _> |_| |_ |_| \/ _> _> |_| | | S-412 96 Gothenburg, SWEDEN
_____________________________...@ce.chalmers.se_____
hamilton on BIX
>So how would I port an application that uses the fact that fork() and
>exec() is not an atomic operation? Suppose I would like to execute a
>new program having its standard input and/or output connected somewhere
>(for example executing the command "xxxx > outfile" or to send an e-mail
>with the contents received from my program):
> fd = .... /* a file, pipe or socket for example */
> if ((pid = fork()) == 0) { /* create a child process */
> dup2(fd, 0); /* for the child, assign fd to stdin */
> exec(whatever, ....); /* have the child execute another program
> keeping stdin connected to the socket */
> }
> parent code; /* The partent's stdin is untouched */
Here's a simple example program that I use in a course I teach ("NT for
UNIX droids" :-) that demonstrates opening a pipe, attaching the input end
to stdout for a child, starting the child, then echoing anything he writes
to my own stdout, finishing by printing total number of bytes that came
thru the pipe. You'll notice that the "trick" is that we use the
DuplicateHandle call to make an inheritable copy of the pipe input
handle and then use SetStdHandle to make that copy stdout, which is
then inherited by the child as his stdout. The reason our printf
will go where it's supposed to is because the C run-time libary
initialization gets invoked before main() is called and does its
own snapshotting of the initial stdout value.
#include <windows.h>
#include <stdio.h>
#define PipeSize 4096
void cdecl main( void )
{
char *c = GetCommandLine();
while (*c && *c != ' ')
c++;
while (*c == ' ')
c++;
if (*c)
{
char buffer[PipeSize];
DWORD length, total = 0;
HANDLE thisProcess = GetCurrentProcess(),
pipeout, pipein, childout, Stdout;
PROCESS_INFORMATION child;
static STARTUPINFO startup;
startup.cb = sizeof(startup);
CreatePipe(&pipeout, &pipein, NULL, PipeSize);
DuplicateHandle(thisProcess, pipein, thisProcess, &childout,
0 /* Ignored */, TRUE /* Inheritable */,
DUPLICATE_CLOSE_SOURCE | DUPLICATE_SAME_ACCESS);
Stdout = GetStdHandle(STD_OUTPUT_HANDLE);
SetStdHandle(STD_OUTPUT_HANDLE, childout);
CreateProcess(NULL, c, NULL, NULL, TRUE /* Inherit handles */,
0, NULL, NULL, &startup, &child);
CloseHandle(childout);
while (ReadFile(pipeout, buffer, sizeof(buffer), &length, NULL))
{
total += length;
WriteFile(Stdout, buffer, length, &length, NULL);
}
CloseHandle(pipeout);
printf("total read = %d bytes\n", total);
WaitForSingleObject(child.hProcess, INFINITE);
}
else
printf("usage: pipe command\n");
ExitProcess(0);
}
>Could someone give me a simple solution to this problem, for example
>with the use of threads?! Please don't give me workarounds such as
>stating that I could use named pipes, temporary files etc.
In the more general case where the objective of the fork() is not
merely to immediately exec a new program, threads can be an alternative,
though the design problems can be somewhat tricky. Basically, you have
to identify all the resources (variables, etc.) that the threads have
to share and semaphore any use of those shared variables so one thread
doesn't trample the value of something another thread is trying to use.
In cases where the important semantic is the snapshotting that fork()
provides, then that must be recreated by per-thread copies of the
relevant variables.
The toughest cases are things that cannot easily be either shared or
duplicated, e.g., current directories. Obviously, the brute force approach
of simply going thru all 26 possible drives, setting the current directories
anytime a thread wants to do something that might depend on the settings
is not a great solution. In a case like this, you may build a more
complex "handle" mechanism where, by comparing a single int, determine
if the process settings match his own desired settings, etc.
>Also, the arguments given earlier indicate that some think (they might not
>be serious though ;-) that since fork/exec are POSIX calls, we shouldn't
>allow them to be mixed with WIN32 calls for example. Why??? It should really
>be my problem to decide what libraries (or subsystems for that matter) I would
>like to use in my program. It's almost like stating that if you use X-windows
>you are not allowed to use character I/O! I find it hard to see good technical
>reasons for not allowing an application to use 2 subsystems at the same time.
>(Or I should at least be able to use the POSIX subsystem and system calls
>to NT, even though they seem to be unpublished at this date).
The reason you can't mix calls to different subsystems is because the
subsystems maintain a lot of "state" information on your behalf. Every
time you create a thread, for example, so does the Win32 subsystem, just
so it can stay in sync with you. It's best to think of this relationship
between your app and the subsystem as a true client-server relationship;
it's not like you're just calling a bunch of random stand-alone subroutines.
Here's a simple analogy: I'm sure you've observed that if you mix calls to
stdio with low-level i/o, your output can be garbled. Things get out of
order because the stdio routines didn't know that you also would be doing
i/o. It's like that (but more complex!) with the subsystems.
If you were to begin mixing in calls to one of the other subsystems, Win32
would have no way of tracking the changes you're making to your process
state. What handles are valid now? What's your memory image look like,
etc.?
Regards,
Jonathan Shekter
Tomas Olovsson <olov...@ce.chalmers.se> wrote:
>
>Ok,
>
>So how would I port an application that uses the fact that fork() and
>exec() is not an atomic operation? Suppose I would like to execute a
>new program having its standard input and/or output connected somewhere
>(for example executing the command "xxxx > outfile" or to send an e-mail
>with the contents received from my program):
>
> fd = .... /* a file, pipe or socket for example */
> if ((pid = fork()) == 0) { /* create a child process */
> dup2(fd, 0); /* for the child, assign fd to stdin */
> exec(whatever, ....); /* have the child execute another program
> keeping stdin connected to the socket */
> }
> parent code; /* The partent's stdin is untouched */
>
>
>Could someone give me a simple solution to this problem, for example
>with the use of threads?! Please don't give me workarounds such as
>stating that I could use named pipes, temporary files etc.
>
One of the parameters to theWin32 call CreateProcess() is a
pointer to a structure of type STARTUPINFO. If you set the
STARTF_USESTDHANDLES bit in the dwFlags member of this structure, the
new processs will use whatever you specify in the hStdInput, hStdOutput,
and hStdError members of this structure.
There you go. That's how you can redirect stuff. You can also
redirect other handles; set the inherit handles bit and the new process
will get all of your handles, oincluding pipes, files, etc. If you
change one of these handles in your program, create the new process with
inherit flags, and then change your handle back, you will effectively
have redirected that handle fore the child process.
I agree that NT should have included a fork() in the Win32 API
but it's so easy to work around... Not really a problem. CreateProcesS()
does it all.
Chris Marriott
>Fork() creates a child process which is an exact copy of the parent,
>including the entire memory space (instructions and data), registers,
>any open file handles, etc. The return value to the parent is the
>process id of the child; the return value to the child is 0.
I'm probably missing something obvious, but could you give me an example
of a situation that this would be useful for? I can't think of a use for
this facility!
kar...@informix.com
> One of the parameters to theWin32 call CreateProcess() is a
>pointer to a structure of type STARTUPINFO. If you set the
>STARTF_USESTDHANDLES bit in the dwFlags member of this structure, the
>new processs will use whatever you specify in the hStdInput, hStdOutput,
>and hStdError members of this structure.
>
> There you go. That's how you can redirect stuff. You can also
>redirect other handles; set the inherit handles bit and the new process
>will get all of your handles, oincluding pipes, files, etc. If you
>change one of these handles in your program, create the new process with
>inherit flags, and then change your handle back, you will effectively
>have redirected that handle fore the child process.
>
> I agree that NT should have included a fork() in the Win32 API
>but it's so easy to work around... Not really a problem. CreateProcesS()
>does it all.
>
> Jonathan Shekter, from Sunny Toronto, Canada
about current state information - global variables, in otherwords snapshot ?
Correct me if I'm wrong. A fork() does the following:
(a) Freeze the parent.
(b) Snapshot the entire memory space (including code+data) into another
memory space.
(c) Flow the parent with fork() returning 0
(d) Flow the child with fork() returning child process id
(e) Now you have two exactly similar processes except that the way they flow depends
on what fork() returns on each. That's the only reason why both the processes
should be different.
(f) All global/local variable references are done on each of their local memory space
and does not influence each other. Because of system file handle table replication,
it goes without saying that redirected handles are snapped to child and can use them.
To do all this in NT, you can do CreateProcess(CurrentProgram) - Good. But if you make it
jump to the location after the if (fork()) call then it would suffice flow control. After
jumping you must copy all the global/local variables from the parent through a pipe/common
file. Ok, how does CurrentProgram know it's a parent or child to jump ? Pass a special command
line character in CreateProcess() inorder that the jump takes place. This should sort of
work. The whole thing cannot be put in a fork() call. Application needs some additional
checking - Am I right ?
Karthik
Nathaniel Mishkin
: Here's a simple example program that I use in a course I teach ("NT for
: UNIX droids" :-) that demonstrates opening a pipe, attaching the input end
: to stdout for a child, starting the child, then echoing anything he writes
: to my own stdout, finishing by printing total number of bytes that came
: thru the pipe. You'll notice that the "trick" is that we use the
: DuplicateHandle call to make an inheritable copy of the pipe input
: handle and then use SetStdHandle to make that copy stdout, which is
: then inherited by the child as his stdout.
Just curious: Is there some reason you chose to use SetStdHandle()
instead of just passing the child its standard handles through the
STARTUPINFO and use the STARTF_USESTDHANDLES flag? They appear
functionally equivalent and the latter approach strikes me as cleaner
(in that you don't have to jerk the parent's state around).
hamilton on BIX
Is there some reason? No, not really. If you prefer using the
STARTUPINFO approach, that's just as good. I happened to begin
working on NT before that was added to CreateProcess (it went in
at the July '92 beta, I think), however, so I just tend to think
in terms of the way I learned to do it.
OTOH, is this really jerking the parent's state around? Oh, I
don't know. Once you've retrieved the std handles you've inherited,
why do care what they get set to, except for purposes of passing
them on to your children? I mean, you weren't planning on re-issuing
GetStdHandle calls to get stdout every time you wanted to print,
were you? Also, in a multi-threaded app, you still need to treat
the inheritable handles as a critical resource, either way, dup'ing
inheritable copies of any open handles you want to pass to a child
and then doing the CreateProcess call only when you own the right
to do so.
On the whole, I had somewhat mixed feelings when MS added the
STARTF_USESTDHANDLES flag to the STARTUPINFO structure. Here, they
were fixing something that wasn't really broken. It seemed like
there were other things that really could have benefitted a bit
more from the attention.
hamilton on BIX
>Correct me if I'm wrong. A fork() does the following:
>(a) Freeze the parent.
>(b) Snapshot the entire memory space (including code+data) into another
> memory space.
>(c) Flow the parent with fork() returning 0
>(d) Flow the child with fork() returning child process id
:
:
> Am I right ?
Almost. The parent gets the child's pid; the child gets 0. (But I
think you already knew that and just made a typo.)
Jim Edwards-Hewitt
>So how would I port an application that uses the fact that fork() and
>exec() is not an atomic operation? Suppose I would like to execute a
>new program having its standard input and/or output connected somewhere
>(for example executing the command "xxxx > outfile" or to send an e-mail
>with the contents received from my program):
>
> fd = .... /* a file, pipe or socket for example */
> if ((pid = fork()) == 0) { /* create a child process */
> dup2(fd, 0); /* for the child, assign fd to stdin */
> exec(whatever, ....); /* have the child execute another program
> keeping stdin connected to the socket */
> }
> parent code; /* The partent's stdin is untouched */
>
>
>Could someone give me a simple solution to this problem, for example
>with the use of threads?! Please don't give me workarounds such as
>stating that I could use named pipes, temporary files etc.
Use CreateProcess, specifying the StartInfo parameter.
-- Jim
---------------------------------------------------------------------------
Jim Edwards-Hewitt j...@visix.com | Aieee! Death from above!
Visix Software Inc. ...!uupsi!visix!jim | -- Sam&Max
---------------------------------------------------------------------------
Frank Whaley
>
> fd = .... /* a file, pipe or socket for example */
> if ((pid = fork()) == 0) { /* create a child process */
> dup2(fd, 0); /* for the child, assign fd to stdin */
> exec(whatever, ....); /* have the child execute another program
> keeping stdin connected to the socket */
> }
> parent code; /* The partent's stdin is untouched */
>
>Could someone give me a simple solution to this problem, for example
>with the use of threads?!
Is this a solution?
old_stdin = dup(0);
dup2(fd, 0);
spawnl(P_NOWAIT, whatever, ....);
dup2(old_stdin, 0);
---
Frank Whaley, Senior Programmer, Autodesk
f...@autodesk.com
Plus je bois, mieux je chante!
Nathaniel Mishkin
: STARTUPINFO approach, that's just as good. I happened to begin
: working on NT before that was added to CreateProcess (it went in
: at the July '92 beta, I think), however, so I just tend to think
: in terms of the way I learned to do it.
Well at least I know now why the examples are written not using it
(they were presumably written before the feature existed).
: OTOH, is this really jerking the parent's state around?
I guess it's just my preferences for doing things parametrically where
possible. I'm not sure I follow your multi-threaded examples (and I'm
not sure it's worth belaboring the point). Presumably you were trying
to head off my rebuttal about how using the STARTUPINFO works better
in MT apps. Certainly there are some MT apps where using the
STARTUPINFO approach isn't sufficient to guarantee overall
correctness. However, for many it is and for those same apps, _not_
using the approach increases the chances of errors with hard-to-detect
explanations.
Of course the best part of the CreateProcess() API is that if you pass
FALSE as the value of "fInheritHandles" and pass handles in the
STARTUPINFO, the handles still don't get passed. I had thought "ah,
this is just what I want--an easy way to make sure I pass _only_ the
std in/out/err handles and mask out the rest". Bzzz! Wrong. Besides
being unhappy that this feature in fact doesn't exist, I hate APIs
that make it easy to say things that don't make any sense (pass
handles and don't pass handles). Of course what we're really seeing
is that the "fInheritHandles" feature is part of the real NT system
call interface whereas the "standard handles" feature is part of the
WIN32 abstraction. Stir them up and see what happens.
--
-- Nat Mishkin
Atria Software, Inc.
mis...@atria.com
hamilton on BIX
>In article <hamilton....@BIX.com> hami...@BIX.com writes:
>>Fork() creates a child process which is an exact copy of the parent,
>>including the entire memory space (instructions and data), registers,
>>any open file handles, etc. The return value to the parent is the
>>process id of the child; the return value to the child is 0.
>I'm probably missing something obvious, but could you give me an example
>of a situation that this would be useful for? I can't think of a use for
>this facility!
The UNIX shells use this extensively when creating pipelines. Suppose
any particular stage is allowed to do "anything", i.e., execute any
arbitrary internal/external command, including looping thru the values
stored in shell variables, etc. How do you do that?
On UNIX, the answer is the the shells fork to generate complete copies
of the entire memory space but, at the same time, to free up each stage
to run at its own pace as a separate process. One free benefit of the
fork is that any particular stage can scribble over his copies of all
the shell variables without ever having to consider semaphoring anything.
Doing this kind of thing with threads is genuinely faster, but (from
personal experience) I can tell you it's more complex.
Regards,
Jim Marshall
||>Fork() creates a child process which is an exact copy of the parent,
||>including the entire memory space (instructions and data), registers,
||>any open file handles, etc. The return value to the parent is the
||>process id of the child; the return value to the child is 0.
||
||I'm probably missing something obvious, but could you give me an example
||of a situation that this would be useful for? I can't think of a use for
||this facility!
||
A classic example is that of a process listening on a socket,
and forking a child to handle a connection, whilst the parent
continues listening.
Yes, it can be done with threads, but it's not a straightforward
port from Unix code (using fork()) to NT code, using threads. The
other implication of using a threaded approach is that an
exception in one thread will shaft all the threads of that process.
BNAG!
CreateProcess doesn't do the job because it doesn't copy the
entire process context, so you may have a bucketful of initialisation
to perform every time you do create a process.
The POSIX subsystem is a joke. Well, no, let me rephrase that. The
POSIX subsystem is fine, as long as you don't need any comms capability
accessible by your code.
--
Cheers,
Jim
christopher kolb
>>Fork() creates a child process which is an exact copy of the parent,
>>including the entire memory space (instructions and data), registers,
>>any open file handles, etc. The return value to the parent is the
>>process id of the child; the return value to the child is 0.
>I'm probably missing something obvious, but could you give me an example
>of a situation that this would be useful for? I can't think of a use for
>this facility!
I had this code laying around from a lecture I gave a few years ago.
This code implements a UNIX TCP server that echos all input (ie, it writes
back to the client everything it reads from the client). The code is pretty
short and simple.
The general idea is that the parent process is responsible for accepting
connections from clients and then forking off a child process to deal with
each client. This way the parent never blocks on a client, and can fork
off N child processes which could all be running in parallel.
Chris
-----
Christopher Kolb <ko...@psi.com>
Performance Systems International, Inc.
510 Huntmar Park Drive
Herndon, VA, 22070
(703)904-4100 x1225
-----
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <stdio.h>
#include <netdb.h>
#include "defs.h"
main(argc, argv)
int argc;
char *argv[];
{
char buf[BUFSIZ]; // i/o buffer
fd_set iomask; // i/o event mask
int cc; // number of bytes (octets!)
int s, t; // tcp/ip socket descriptors
servent *sp; // pointer to service entry
sockaddr_in sin; // internet socket address
// Create a stream socket that uses the Internet TCP protocol.
if ((s = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) == -1)
goodbye("socket: %s", errmsg ());
// Bind the socket to the default IP address for the machine,
// and the well known service port for this TCP service.
memset((char*) &sin, 0, sizeof sin);
sin.sin_addr.s_addr = htonl(INADDR_ANY);
sin.sin_family = AF_INET;
if ((sp = getservbyname(MYSERVICE, "tcp")) == NULL)
sin.sin_port = htons(DEFPORT);
else
sin.sin_port = sp -> s_port;
if (bind(s, (sockaddr *) &sin, sizeof sin) == -1)
goodbye("bind: %s", errmsg());
// Allow up to 5 connection requests to be waiting in the queue.
if (listen(s, 5) == -1)
goodbye("listen: %s", errmsg());
// The event loop.
while (1) {
// Wait for an I/O event. In this case we are waiting for
// for a client to attempt a connection.
FD_ZERO(&iomask);
FD_SET(s, &iomask);
if (select(s + 1, &iomask, 0, 0, 0) == -1)
goodbye("select: %s", errmsg ());
else if (FD_ISSET(s, &iomask)) {
// Accept the connection. The call to accept returns the
// file/socket descriptor that should be used to communicate
// with the client.
int slen = sizeof sin;
if ((t = accept(s, (sockaddr *) &sin, &slen)) == -1)
goodbye("accept: %s", errmsg());
// Create a child process to handle the client so that the
// parent process can immediately wait for the next client.
switch (fork()) {
case -1:
// An error. Complain and exit.
goodbye("fork: %s", errmsg());
case 0:
// The child process executes this code.
while ((cc = recv(t, buf, BUFSIZ, 0)) > 0)
if (send(t, buf, cc, 0) == -1)
goodbye("send: %s", errmsg());
if (cc == -1)
goodbye("recv: %s", errmsg());
exit(0);
default:
// The parent process executes this code.
if (close(t) == -1)
goodbye("close: %s", errmsg());
}
}
}
}
Arthur Kreitman
> Ok,
>
> So how would I port an application that uses the fact that fork() and
> exec() is not an atomic operation? Suppose I would like to execute a
> new program having its standard input and/or output connected somewhere
> (for example executing the command "xxxx > outfile" or to send an e-mail
> with the contents received from my program):
> c code example deleted.....
>
> Could someone give me a simple solution to this problem, for example
> with the use of threads?! Please don't give me workarounds such as
> stating that I could use named pipes, temporary files etc.
There is an example in the MS Win32 Programmers Reference Vol 2. that
does exactly what you asking. Its not the same code as you provided, but
it does the same thing.
Ian Kemmish
>>Correct me if I'm wrong. A fork() does the following:
>>(a) Freeze the parent.
>>(b) Snapshot the entire memory space (including code+data) into another
>> memory space.
System V (i.e. most modern UNIXes) just maps the memory copy-on-write
in the child process. THis ought not be difficult on NT, and
of course it makes fork() less slow and removes the need for
hacks like vfork().
>>(c) Flow the parent with fork() returning 0
>>(d) Flow the child with fork() returning child process id
> :
> :
--
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Ian Kemmish 18 Durham Close, Biggleswade, Beds SG18 8HZ
i...@eeyore.dircon.co.uk Tel: +44 767 601 361
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
hamilton on BIX
The quotes you attribute to me were actually what I was quoting
from someone else's message. Yes, I do certainly understand the
advantage of copy-on-write. Matter of fact, the rest of that message
I posted (or if not that one, others in that thread) did explicitly
point out that no one does complete memory copies on any modern
UNIX system; it's always done with page table manipulations and
copy-on-write strategies.
The problem providing fork() on NT is not just a matter of doing
copy-on-write. (Actually, the NT kernel on which the Win32 subsystem
is built, provides that functionality already.) The problem is (or
so it's claimed by the MS people) how to share a message queue.
Arthur Kreitman
> Correct me if I'm wrong. A fork() does the following:
> (a) Freeze the parent.
> (b) Snapshot the entire memory space (including code+data) into another
> memory space.
> (c) Flow the parent with fork() returning 0
more stuff about fork deleted.
> checking - Am I right ?
You're right, but it doesn't matter. For all but a small fraction of
cases, everything done with fork/exec can be done on nt with spawn without
much work.
Andre Beck
In article <hamilton....@BIX.com>, hami...@BIX.com (hamilton on BIX) writes:
|>
|>ke...@loral.cts.com (Kent Yang) writes:
|>
|>>I am porting an Unix application to NT and I was wondering if anyone
|>>might know how I can simulate or write the fork() call on NT? Exec()
|>>and spawn() is not quite what I am looking for. Should I consider using
|>>threads?
|>
|>an application that's architected around a fork primitive to use threads
|>instead may be more work than is really feasible. With a fork, parent
|>the fork. From there on, each can scribble over anything they like without
|>concern for the effects on the other. That's not the case with threads,
|>obviously, where you need to consider locking down critical resources
|>with semaphores before you make changes, etc.
|>
|>One alternative you may also wish to consider is a product called
|>NuTCRACKER (yes, the lower case "u" is intentional) from DataFocus,
|>of my knowledge) but includes a porting library that simulates a fork
|>on top of Win32. Around Christmas I talked with David Korn (author of
|>the Korn shell) who told me he'd successfully used the NuTCRACKER
|>library to port his shell. His comment was that the fork implementation
|>was pretty slow (since they don't get any help from the OS to do sharing
|>of page-table entries to do a copy-on-write implementation as all good
|>UNIX systems do these day and must, instead, do a wholesale copy of the
|>
Seeing all this about how to emulate fork() on NT I wonder what happened
to the POSIX subsystem ? Why does nobody use it ? Why is the CLI made up
from an ugly DOS-Prompt instead of a POSIX Shell ? (okay, this tends
into advocacy direction already, but I _REALLY_ hate those driveletters and
backslashes).
--
+-o-+--------------------------------------------------------+-o-+
| o | \\\- Brain Inside -/// | o |
| o | ^^^^^^^^^^^^^^ | o |
| o | Andre' Beck (ABPSoft) Andre...@IRS.Inf.TU-Dresden.de | o |
+-o-+--------------------------------------------------------+-o-+
Ross Ridge
>Fork() creates a child process which is an exact copy of the parent,
>including the entire memory space (instructions and data), registers,
>any open file handles, etc. The return value to the parent is the
>process id of the child; the return value to the child is 0.
ch...@chrism.demon.co.uk writes:
>I'm probably missing something obvious, but could you give me an example
>of a situation that this would be useful for? I can't think of a use for
>this facility!
Ever want a programme to do two or more things at the same time? That's
an obvious use of fork(). With respect to Windows NT however, fork()
is primarily only useful for Unix compatibility. Under Unix fork() is
the only way to create a new process. Unix breaks down the "spawn"
function into two parts, fork and exec. First you need to call fork()
(split into two processes), and then in the child process call exec()
(replace the current running programme with another.) As I understand
there were certain design advantages in the Unix kernel by doing it this
way, but it also makes it easier to set up the new programme's
environment without mucking with the "spawning" programme's
environment.
Ross Ridge
Karthikeyan Guruswamy
>Seeing all this about how to emulate fork() on NT I wonder what happened
>to the POSIX subsystem ? Why does nobody use it ? Why is the CLI made up
>from an ugly DOS-Prompt instead of a POSIX Shell ? (okay, this tends
>into advocacy direction already, but I _REALLY_ hate those driveletters and
>backslashes).
It's the basic issue - If you POSIX on NT, you cannot access other subsystems
like Win32 API.
Karthik
Standard Disclaimer: 'These opinions are my own and does reflect in anyway the
ideas or opinions of my organization'
hamilton on BIX
>Seeing all this about how to emulate fork() on NT I wonder what happened
>to the POSIX subsystem ? Why does nobody use it ?
The POSIX subsystem is pretty minimal. No support for graphics, no
networking, no threads, not even a debugger. Since most of us work in
a competitive world where the product that offers the best combination
of functionality, performance, etc., is the one that sells, we're all
pretty much forced to use Win32. A POSIX app is just not going to be
competitive on an NT platform.
The cases where POSIX may be appropriate boil down to (a) captive
markets, e.g., an MIS shop building products for in-house use (where
top management may be satisfied that development costs/schedule should
dictate the strategy) or (b) "quickie" tools ports from UNIX for personal
or departmental use.
>Why is the CLI made up from an ugly DOS-Prompt instead of a POSIX Shell ?
Obviously, the question is retorical, since we both know that the problem
is that the MS marketers don't attach as much importance to a good shell
as those of us coming from a UNIX background do. But also, I think they
view this as a niche requirement (compared to mainstream stuff like WP,
spreadsheets, etc.) that's probably too small for Microsoft's attention.
But never fear, there are several alternatives to that numbingly brain-
dead cmd.exe from my own company (which offers a C shell that'll blow
away the one on UNIX) and from other sources (including MKS, HIPPO
and Consensys). Can't speak for the other guys, but certainly I do
offer free demo disks if you care to take a closer look.
Regards,
Doug Hamilton hami...@bix.com Ph 508-358-5715 FAX 508-358-1113
Jim Marshall
||
|| You're right, but it doesn't matter. For all but a small fraction of
||cases, everything done with fork/exec can be done on nt with spawn without
||much work.
||
There speaks a man with no existing unix/fork()-based code :-)))))))
I'd appreciate it if you could tell my code that I can port it
"without much work"...
Our socket-based comms-oriented code requires:
* the addition of a message-dispatching loop, plus associated
window-proc code,
* the use of a command line arg to indicate child,
* re-initialisation of data in each child that is 'spawned'
* restructuring/reorganisation of a lot of the code to
cope with the above...
Perhaps what you meant is that your specific code ported without
much work - we certainly have had a lot of grief in trying to port
our unix code to NT, firstly through abortive attempts to use the
posix patch (no useful comms for our requirements), then through
the steps listed above.
IMO, the lack of a "real" (unix-like) fork() is a major hassle.
--
Cheers,
Jim
r...@infograph.com
>The problem providing fork() on NT is not just a matter of doing
>copy-on-write. (Actually, the NT kernel on which the Win32 subsystem
>is built, provides that functionality already.) The problem is (or
>so it's claimed by the MS people) how to share a message queue.
MS also told me that there was a question of how to duplicate the user
interface. For instance, if you have an application up which has say 2
windows and a dialog box up, the user pushes a button on the dialog box
and this results in a fork(), do they share the windows and dialog box,
or does the child get its own windows and dialog box?
I am pretty unfamiliar with how fork() works in similar situations under
say X -- I am just repeating a concern which an MS engineer conveyed to
me...
rjc
Andre Beck
In article <2k36vt$9...@infmx.informix.com>, kar...@informix.com (Karthikeyan Guruswamy) writes:
|>
|>In article <2k2tf0$10...@irzr17.inf.tu-dresden.de> Andre...@IRS.Inf.TU-Dresden.DE writes:
|>>Seeing all this about how to emulate fork() on NT I wonder what happened
|>>to the POSIX subsystem ? Why does nobody use it ? Why is the CLI made up
|>>from an ugly DOS-Prompt instead of a POSIX Shell ? (okay, this tends
|>>into advocacy direction already, but I _REALLY_ hate those driveletters and
|>>backslashes).
|>
|>It's the basic issue - If you POSIX on NT, you cannot access other subsystems
|>like Win32 API.
|>
Doing even more advocacy, I answer "What for do I need Win32 ?" What I really
want to say here is, give the POSIX API a working XServer and all the problems
are solved. Porting would simply mean recompile. I know there are several
X Servers and a Xlib on W32 emulation available, but a Free XServer would
be greatest...
Olav Tollefsen
In article <2kdhs0$1l...@irzr17.inf.tu-dresden.de>
be...@irzr17.inf.tu-dresden.de (Andre Beck) writes:
> Doing even more advocacy, I answer "What for do I need Win32 ?" What I really
> want to say here is, give the POSIX API a working XServer and all the problems
> are solved. Porting would simply mean recompile. I know there are several
> X Servers and a Xlib on W32 emulation available, but a Free XServer would
> be greatest...
If you find that you can live with the POSIX 1003.1 API and be competitive,
fine. The Win32 API does however contain a lot of added value over the
POSIX API. Don't you want your server to be able to run nicely as a Windows
NT service? Maybe you would want your server to log messages in the
Windows NT Event logger? Perhaps you would be interested in using the
nice integrated security functions available in Windows NT. Portability
will often need to traded against functionality.
Olav Tollefsen
Digital Equipment Corporation, Norway
Greg DeMichillie
> But never fear, there are several alternatives to that numbingly brain-
> dead cmd.exe from my own company (which offers a C shell that'll blow
> away the one on UNIX) and from other sources (including MKS, HIPPO
> and Consensys). Can't speak for the other guys, but certainly I do
> offer free demo disks if you care to take a closer look.
I just wanted to add a whole-hearted (an unsolicited) endorsement of
Hamilton C Shell, a truly exceptional product. And I'll add that it's
the only shell out there that is available on all of the NT releases
(Intel, MIPS, Alpha etc...)
-greg
--
Greg DeMichillie Microsoft AFX
gre...@microsoft.com All opinions are mine, not Microsoft's
Arthur Kreitman
>
> There speaks a man with no existing unix/fork()-based code :-)))))))
Not true at all. If you knew what was in our product porfolio, you
wouldn't say that.
> I'd appreciate it if you could tell my code that I can port it
> "without much work"...
For a large fee, we could assign one of our staff members to solve
your problem.
> Our socket-based comms-oriented code requires:
>
> * the addition of a message-dispatching loop, plus associated
> window-proc code,
only if you want to add functionality that didn't exist in the unix version.
...dumb porting stuff deleted....
>
> Perhaps what you meant is that your specific code ported without
> much work - we certainly have had a lot of grief in trying to port
> our unix code to NT, firstly through abortive attempts to use the
> posix patch (no useful comms for our requirements), then through
> the steps listed above.
We've be able to convert most of the standard unix networking demons
into multithreaded services without much hassle. You just have to have
a little more insight than you've shown.
> IMO, the lack of a "real" (unix-like) fork() is a major hassle.
It is, but so what. It means that for applications structured around
a different system model you have to do more than type "make".
Phil Hystad
fork() is a nice feature, of UNIX, and it has many uses in UNIX and the
most prevalent seem to be in command shell piping and in server code
(where you have common input and then fork() to do work).
However, fork() is not a solution for every problem and certainly I find
it difficult to think of why I would want to fork() and then duplicate
the user interface...this just doesn't seem to make sense. Considering
that MS would want to create a fork() procedure in WIN32, they can easily
do so, avoiding lots of problems, by allowing a few limitations. Such
justifiable limitations seem to be things like a "windows interface".
phil
=============================================================================
p...@esca.com || 206-822-6800 || 206-889-1809 x2023 (voice mail)
=============================================================================
Hans Peter Kluender
: MS also told me that there was a question of how to duplicate the user
: interface. For instance, if you have an application up which has say 2
: windows and a dialog box up, the user pushes a button on the dialog box
: and this results in a fork(), do they share the windows and dialog box,
: or does the child get its own windows and dialog box?
There is third possibility which is more common under UNIX,
but probably appears to be useless to windows-programmers:
The child does not use any windows or boxes at all.
It silently does some work and then tells the parent what it did.
The parent controls the mouse and controls the user interface,
but it defers complicated and time-consuming jobs to
designated childs, which should never be seen by anyone else.
The advantage is that the parent can continue to draw and beep
and do everything to be a good entertainer for the user,
while the real work is done sinently in the background.
: I am pretty unfamiliar with how fork() works
: in similar situations under say X -- I am just repeating
: a concern which an MS engineer conveyed to me...
After the fork(), each process knows wether it's parent or child.
One of them, usually the child, simply does not use the X-connection.
This ist easily to program since both processes share the same code,
and a simple if(fork()) { .. use connection .. } else { .. don't .. }
it neither difficult to write nor to read, at least for UNIX programmers.
If the child exec()s and thus leaves the control of the program,
it's a good idea to set the close-on-exec-bit in the X-connection.
this means that if the child execs, it can't use the connection.
If the child wants do have it's own user interface, it may
open a new connection to the server and create it's own windows.
But the more common case is that the child works sinlently,
never showing something to the user or requesting input.
_ _ __ __ __ ______________________________________________
|_|\ \/ // \ / /
_ \ // /\ \\ \ Hans Peter Kluender iXOS Software GmbH
| | / \\ \ \ \\ \ email j...@ixos.de Bretonischer Ring 12
| |/ /\ \\ \/ /_\ \ Tel +49 89 46005199 D-85630 Grasbrunn
|___/ \_\\__/|___/ Fax +49 89 46005185 Germany
Mitsu Hadeishi
: r...@infograph.com writes:
Seems to me that from the foregoing discussion most of what
is done with fork() under Unix can be done faster and more efficiently
using a CreateThread() call. After all, after creating a new thread,
you still share code and data, the difference being that global data
is not copied-on-write. But it seems to me that the kind of mindless
simplicity of fork() is generally functional overkill; in particular,
what you really want is not copy-on-write for ALL variables, but
shared memory for some variables, and local storage for others. But
then you can accomplish the latter using thread local storage, and
the former by simply using semaphores to coordinate access to shared
variables.
Mitsu Hadeishi
General Partner
Open Mind Research
mi...@openmind.com
Brad Wilson
>> r...@infograph.com writes:
>>
>> : MS also told me that there was a question of how to duplicate the user
>> : interface. For instance, if you have an application up which has say 2
>> : windows and a dialog box up, the user pushes a button on the dialog box
>> : and this results in a fork(), do they share the windows and dialog box,
>> : or does the child get its own windows and dialog box?
>>
>> There is third possibility which is more common under UNIX,
>> but probably appears to be useless to windows-programmers:
>> The child does not use any windows or boxes at all.
>> It silently does some work and then tells the parent what it did.
Gosh ... this just sounds too much like threads.
A question: who cares? Really ... threads are there. You don't need to
go through contortions to get threads to work (they're in the OS). They
have some admitted shortcomings (mostly the number available) ... this
will most likely be addressed in Daytona.
Can this thread die now? Please?
--
Brad Wilson Share and Enjoy! Voice: (800)282-4FTP Fax: (508)659-6297
Not speaking for FTP Software, Inc. Internet Email: wil...@ftp.com
"A little government and a little luck are necessary in life,
but only a fool trusts either of them." -P.J. O'Rourke
Arthur Kreitman
> MS also told me that there was a question of how to duplicate the user
> interface. For instance, if you have an application up which has say 2
stuff deleted
> I am pretty unfamiliar with how fork() works in similar situations under
> say X -- I am just repeating a concern which an MS engineer conveyed to
> me...
X has no problem, every one knows to make the handles (ie the sockets
that talk to the X server) close on exec. No problem.
James Vlcek
>Seems to me that from the foregoing discussion most of what
>is done with fork() under Unix can be done faster and more efficiently
>using a CreateThread() call. After all, after creating a new thread,
>you still share code and data, the difference being that global data
>is not copied-on-write.
The cited article (reposted in its entirety! please learn to edit
quoted articles) discussed a use of fork() in which the entire child
code is located in the same image as that of the parent. As has been
oft noted here, such a use of fork() is rare; it is much more common
for fork() to be followed quickly (but not immediately) by an exec().
Note, BTW, that threading is also available under UNIX (at least
Solaris 2.x) as well, allowing the same sort of approach in deference
to fork() if you prefer.
>But it seems to me that the kind of mindless
>simplicity of fork() is generally functional overkill; in particular,
>what you really want is not copy-on-write for ALL variables, but
>shared memory for some variables, and local storage for others. But
>then you can accomplish the latter using thread local storage, and
>the former by simply using semaphores to coordinate access to shared
>variables.
fork(), being a generator of heavyweight processes, shouldn't really
be compared to threads (which, again, are available as well in UNIX).
If you did, however, want to use fork in such a context, you can
use shared memory (with semaphores) to coordinate IPC between the
parent and child. This might have an advantage over threads in that
the sharing of memory is explicit rather than implicit, and that
exceptions generated (or buggy behavior) in one thread would not
affect the others so dramatically as if they shared a common context.
Jim Vlcek
vl...@epimbe.com
Andre Beck
In article <CLMvG...@infograph.com>, r...@infograph.com writes:
|>
|>MS also told me that there was a question of how to duplicate the user
|>interface. For instance, if you have an application up which has say 2
|>windows and a dialog box up, the user pushes a button on the dialog box
|>and this results in a fork(), do they share the windows and dialog box,
|>or does the child get its own windows and dialog box?
|>
|>I am pretty unfamiliar with how fork() works in similar situations under
|>say X -- I am just repeating a concern which an MS engineer conveyed to
|>me...
|>
With X, the child would probably have a file descriptor of the connection
which is equal to it's parents fd - making a good reason to disturb Xlib
completely. Thus, one closes the connection fd prior to exec, so the problem
is solved as a neither-nor, i.e. neither is the gui duplicated nor is it
reused. However, if the forked child never execs but runs with the inherited
memory and fds, it just MUST NOT act on the connection id, even if it probably
could because all Xlib state is primarily duplicated. Such application will
eventually reopen the connection using the name of the old one, and do what
it wants after it connected - including use of it's parents windows and such.