Bidrectional Subprocess Communication
Emanuele D'Arrigo
I'm trying to replicate the positive results of the Client/Server
scripts from the thread "Bidirectional Networking", but this time
using a Process/SubProcess architecture.
The SubProcess, acting as a server, is working just fine. But the
Process executing the SubProcess, the client, somehow doesn't hear any
of the standard output from the SubProcess. What am I missing?
Below you can find the two short pieces of code. Thanks for your help!
Manu
-------------------------------------------
# serverTest.py
import sys
print("Starting Server!")
while True:
data = sys.stdin.readline().strip()
if(data):
print("SERVER RECV: '" + data + "'")
if(data == "Stop!"):
break
print("Server Stopped!")
-------------------------------------------
#clientTest.py
import sys
import threading
from time import sleep
from subprocess import Popen, PIPE
## Server Thread
class ListenerThread(threading.Thread):
def __init__(self, inChannel):
threading.Thread.__init__(self)
self.inChannel = inChannel
def run(self):
while True:
data = self.inChannel.readline().strip()
print("serverTest.py says: " + data)
print("Starting Client!")
server = Popen("python serverTest.py", stdin=PIPE, stdout=PIPE)
listenerThread = ListenerThread(server.stdout)
listenerThread.setDaemon(True)
listenerThread.start()
server.stdin.write("Something very meaningful!\n")
server.stdin.write("Stop!")
print("Client Stopped!")
Gabriel Genellina
escribió:
> I'm trying to replicate the positive results of the Client/Server
> scripts from the thread "Bidirectional Networking", but this time
> using a Process/SubProcess architecture.
>
> The SubProcess, acting as a server, is working just fine. But the
> Process executing the SubProcess, the client, somehow doesn't hear any
> of the standard output from the SubProcess. What am I missing?
(Pipes don't work the same as sockets, although unix-like systems try hard
to hide the differences...)
- you have to close server.stdin when you don't have more data to send.
The server will see an end-of-file and knows it has to exit the loop. Same
thing on the client side.
- you have to wait until the server answers, else it will get a "broken
pipe" error or similar.
- end-of-file, in Python, is detected when a read/readline returns an
empty string
- you sent "Stop!" without a \n - readline() on the server side would wait
forever; it doesn't matter in the code below because server.stdin is
explicitely closed.
Below are the modified version of your programs:
#clientTest.py
import sys
import threading
from time import sleep
from subprocess import Popen, PIPE
## Server Thread
class ListenerThread(threading.Thread):
def __init__(self, inChannel):
threading.Thread.__init__(self)
self.inChannel = inChannel
def run(self):
while True:
data = self.inChannel.readline()
if not data: # data=='' means eof
break
data = data.strip()
print("serverTest.py says: " + data)
print("Starting Client!")
server = Popen("python serverTest.py", stdin=PIPE, stdout=PIPE)
listenerThread = ListenerThread(server.stdout)
listenerThread.start()
server.stdin.write("Something very meaningful!\n")
server.stdin.write("Stop!")
server.stdin.close() # notify server: no more data
listenerThread.join() # wait until server closes channel
print("Client Stopped!")
# serverTest.py
import sys
print("Starting Server!")
while True:
data = sys.stdin.readline()
if not data: # data=='' means eof
break
data = data.strip()
print("SERVER RECV: '" + data + "'")
if(data == "Stop!"):
break
print("Server Stopped!")
--
Gabriel Genellina
greg
> (Pipes don't work the same as sockets, although unix-like systems try
> hard to hide the differences...)
BSD-based unixes implement pipes using socketpair(), so
pipes actually *are* sockets (or at least they used to be,
not sure whether it's still true).
--
Greg
Emanuele D'Arrigo
wrote:
> - you have to close server.stdin when you don't have more data to send.
> The server will see an end-of-file and knows it has to exit the loop.
> Same thing on the client side.
Hi Gabriel, thanks for modifying the code to make it work. I've just
tried tinkering with it to see how far it would go. On your two
statements above: does this means that any data must be sent in one
batch and then the subprocess must shut down? What I was trying to
simulate is a client/server relationship through a subprocess, where
the server (the subprocess) keeps listening and the client sends data
when it wants to (and eventually viceversa). But when the
server.stdin.close() statement is issued, the pipe is closed for good
and can't be reopened (can it?).
> - you have to wait until the server answers, else it will get a "broken
> pipe" error or similar.
So, if I want to interrogate the subprocess multiple times I must end
and restart the ListenerThread multiple times then?
In the meantime, I better brush up on my streams... =)
Gabriel Genellina
escribió:
> On Dec 14, 4:48 am, "Gabriel Genellina" <gagsl-...@yahoo.com.ar>
> wrote:
>> - you have to close server.stdin when you don't have more data to send.
>> The server will see an end-of-file and knows it has to exit the loop.
>> Same thing on the client side.
>
> Hi Gabriel, thanks for modifying the code to make it work. I've just
> tried tinkering with it to see how far it would go. On your two
> statements above: does this means that any data must be sent in one
> batch and then the subprocess must shut down? What I was trying to
> simulate is a client/server relationship through a subprocess, where
> the server (the subprocess) keeps listening and the client sends data
> when it wants to (and eventually viceversa). But when the
> server.stdin.close() statement is issued, the pipe is closed for good
> and can't be reopened (can it?).
No, not at all. You can keep writing things to the pipe - as long as the
read side keeps reading, there is no limit on how much data you can send.
Just make sure you close the writing side of the pipe when you have no
more data to send, to signal the other side it's OK to exit the read loop.
>> - you have to wait until the server answers, else it will get a "broken
>> pipe" error or similar.
>
> So, if I want to interrogate the subprocess multiple times I must end
> and restart the ListenerThread multiple times then?
No, I mean, since your example is bidirectional, the parent process must
still be alive and able to read when the subprocess replies. Given this
sequence:
parent writes "hi sub!"
child reads "hi sub!"
child writes "hi dad!"
parent reads "hi dad!"
if the parent just exits after sending "hi sub!", the child will get an
error when replying (I think it says "invalid handle" on Windows, "broken
pipe" on Linux, or something like this).
--
Gabriel Genellina
Gabriel Genellina
escribió:
But not on Linux; a visible difference is that pipes are half-duplex on
Linux (it seems that's enough for POSIX). I don't know for sure how
they're implemented on Windows but they seem to be file system objects
(they use functions like CreateFile, ReadFile, WriteFile, etc.)
--
Gabriel Genellina