os.fork() different in cgi-script?
Andreas Kuntzagk
following code:
#!/usr/bin/python
import os
import sys
def main():
print "Content-Type: text/plain\n\n"
print os.getpid()
childPID = os.fork()
if childPID == 0:
sys.exit()
else:
os.wait()
if __name__ == "__main__":
main()
when run on a shell, gives the result:
---------------------------
Content-Type: text/plain
20953
----------------------------
but run as a cgi-script it gives:
---------------------------
21039
Content-Type: text/plain
21039
---------------------------
So it looks the main() is run 2 times.
Is it so? And if yes, why?
I googled for this but only found a similar question from 1997 and no
answer.
Ciao, Andreas
Andreas Kuntzagk
this is Python 2.2.1, apache 1.3.26 on SuSE 8.1
Andreas
Michael Coleman
> def main():
> print "Content-Type: text/plain\n\n"
> print os.getpid()
>
> childPID = os.fork()
> if childPID == 0:
> sys.exit()
> else:
> os.wait()
>
> if __name__ == "__main__":
> main()
>
> when run on a shell, gives the result:
> ---------------------------
> Content-Type: text/plain
>
>
> 20953
> ----------------------------
>
> but run as a cgi-script it gives:
> ---------------------------
> 21039
> Content-Type: text/plain
>
>
> 21039
> ---------------------------
> So it looks the main() is run 2 times.
> Is it so? And if yes, why?
Probably because stdout is being buffered here. The pid got written in the buffer (but not yet actually printed), then the process was forked, then the buffer got flushed (written out) by each child.
One solution would be to make sure everything is flushed before you fork.
Mike
--
Mike Coleman, Scientific Programmer, +1 816 926 4419
Stowers Institute for Biomedical Research
1000 E. 50th St., Kansas City, MO 64110
Andreas Kuntzagk
> Probably because stdout is being buffered here. The pid got written in
> the buffer (but not yet actually printed), then the process was forked,
> then the buffer got flushed (written out) by each child.
>
> One solution would be to make sure everything is flushed before you
> fork.
Thanx, that's it. Is it possible/advisable to make stdout unbuffered?
Andreas
Erwin S. Andreasen
> following code:
> print "Content-Type: text/plain\n\n"
> print os.getpid()
>
> childPID = os.fork()
> if childPID == 0:
> sys.exit()
> else:
> os.wait()
[...]
> So it looks the main() is run 2 times.
> Is it so? And if yes, why?
The problem is buffering. When you run your script from the command
line, the standard output is a terminal and becomes line-buffered.
Thus at the time you fork, the stdout buffer has been flushed.
When you run it as a CGI script, stdout is a socket and thus not a
terminal -- so it becomes block buffered. At the time you fork, each
copy of stdout holds all of our unflushed output. When the two
processes exit they will both flush the output, resulting in it being
printed twice.
You can use sys.stdout.flush() to flush the pending output before
forking.
--
===============================================================
<er...@andreasen.org> Herlev, Denmark
<URL:http://www.andreasen.org/> <*>
===============================================================
Juha Autero
> So it looks the main() is run 2 times.
> Is it so? And if yes, why?
No, it isn't. The problem is that Python uses C STDIO library that has
buffered input and output. The text written in stdout is written in
buffer. Since child process created by fork() is almost identical to
parent process, it will also have a copy of stdout buffer. When
process exits, it closes stdout, which flushes the buffer. So, both
buffers are written to stdout and that causes everything to be printed
twice.
The reason why this doesn happen on command line is that buffers to
terminals are flushed after newline. You can see that by redirecting
script output to file:
$ python script.py >output
$ cat output
Content-Type: text/plain
4309
Content-Type: text/plain
4309
> I googled for this but only found a similar question from 1997 and no
> answer.
Which is strange since this common problem and has nothing to do with
Python.
--
Juha Autero
http://www.iki.fi/jautero/
Eschew obscurity!
Andreas Kuntzagk
>> I googled for this but only found a similar question from 1997 and no
>> answer.
>
> Which is strange since this common problem and has nothing to do with
> Python.
Not so strange, because assuming a Python feature I also included "python"
in the googlequest(tm).
Andreas
Michael Coleman
> Thanx, that's it. Is it possible/advisable to make stdout unbuffered?
Probably. I'd be more inclined to just do an explicit flush in the
few places where they are specifically needed, though.