reading stdout from child
Thomas Thiele
I have a python program. It works fine.
But: I wrote a second python program that should start these program
using popen to read it's stdout. And this works not fine, too.
But not always.
Sorry but it is little bit long.
First the wrapper ( I use my own popen for experiment ):
This program forks a child and should print it's stdoutput.
LOCALSTART.py
#------------------------------------------------------------------------------------
#! /usr/local/python/bin/python
import popen2, os, select, sys, string
class LocalWrapper:
def __init__(self):
self.listofchilds = []
def start(self, command):
self.startone(command)
def startone(self, command):
#create pipes for communication
p2cread, p2cwrite = os.pipe()
c2pread, c2pwrite = os.pipe()
errout, errin = os.pipe()
#fork new process
self.PID = os.fork()
if self.PID == 0: #**********child**********
#stdout und stdin
os.close(0)
os.close(1)
os.close(2)
if os.dup(p2cread) != 0:
sys.stderr.write('popen2: bad read dup\n')
if os.dup(c2pwrite) != 1:
sys.stderr.write('popen2: bad write dup\n')
if os.dup(errin) != 2: pass
for i in range(3, 256):
try: os.close(i)
except: pass
commlist = string.split(command, " ")
try: os.execvp( commlist[0], commlist )
finally: os._exit(1)
if self.PID > 0: #**********father**********
os.close(p2cread)
os.close(c2pwrite)
os.close(errin)
fout = os.fdopen(c2pread)
while 1:
stdoutput, errorout = "", ""
stdoutput = fout.readline() <------- ?????????
if stdoutput: print stdoutput,
if not stdoutput:
break
try: os.waitpid(self.PID, 0)
except OSError: print "no child" #pass
os.close(p2cwrite)
os.close(c2pread)
os.close(errout)
print "child is dead!!!!!!"
#******************************************
#*
Main *
#******************************************
if __name__ == "__main__":
import sys
argstr = sys.argv[1]
for arg in sys.argv[2:]: argstr = argstr + " " + arg
LocalWrapper().start(argstr)
#------------------------------------------------------------------------------------
second the main part from the other script LOCALSERVER.py.
It's actually a server derived from SocketServer.py with a class for
functionality
#------------------------------------------------------------------------------------
#******************************************
#*
Main *
#*******************************************
import sys
sys.stdout.write("HHHHHHAAAAAAAALLLLLLLLLLLLLLLOOOOOOOOOO!\n")
if __name__ == '__main__':
import os,sys,string
def usage():
print 'Usage: %s host port processors memory(mb)' %
os.path.basename(sys.argv[0])
if len(sys.argv) < 4:
usage()
else:
# Handle command line arguments
host = sys.argv[1]
port = string.atoi(sys.argv[2])
processors = string.atoi(sys.argv[3])
if len(sys.argv) >= 5: memory = string.atoi(sys.argv[4])
else: memory = 0
print "Starting local render server: Host: ", host, "Port: ", port,
"Process: ", processors, "RAM(mb): ", memory, "\n"
print "Version: ", VERSION, "\n"
executer = Local_Job_Executer(processors, memory, host) <-- this class
is not of interest
server = Local_Job_Server(host, port, executer) <-- derived from
SocketServer(multithreading)
server.serve_forever() <--- disabled: it works
#------------------------------------------------------------------------------------
When I call "LOCALSTART.py LOCALSERVER.py odin 1750 1" I expect to get
the normal screen output:
HHHHHHAAAAAAAALLLLLLLLLLLLLLLOOOOOOOOOO!
Starting local render server: Host: odin Port: 1750 Process: 1
RAM(mb): 0
Version: 25_07_2000_1
and from all other prints later in the program LOCALSERVER.py
but It works only if I disable server.serve_forever(). If the program
runs into the infinite loop I will get no output.
I cannot read the output in the warpper (see marked line <---??????? in
LOCALSTART.py).
Why? I expect that readline read when an output with newline at the end
was written?
Thank you for help
Thomas
Alex Martelli
news:397ECD5D...@muc.das-werk.de...
[snip]
You may be running into *buffering* problems. Python, like most languages,
will, by default, *buffer* its output when it is directed to a file (or
pipe); i.e.,
each 'written' datum is actually only added to a memory 'buffer', and the
whole
buffer gets _really_ written out, in one go, when it's filling up, or when
the file
is closed, or when the program is terminating. This is really crucial to
enhance
program performance, but it does sometimes interfere with programs that
want to read each other's output.
Fortunately, Python gives you the easiest way to defeat output buffering, of
any language I know of: just use the -u switch (option) on the command line.
I.e., instead of
python something.py
run
python -u something.py
and all output will become unbuffered. See if it works this way: it's a
very
cheap and easy experiment! If it does work, then this confirms it's a
buffering
problem. You can then, if you wish, attack it more directly (without having
to completely disable the buffering functionality) by explicitly *flushing*
the
output-streams at strategical points in your program -- this "flushing" will
cause
the memory-buffer to be truly "written out" as and when you request that.
Alex