Re: toaster interprocess communication (IPC) tips from bayPIGgies

[Nate Theis]

I'm not entirely sure why we need IPC. My code just takes a bunch of callback functions, and calls them at appropriate times. I'm pretty sure PyGTK has its own eventloop, so the GUI should respond when we're waiting on Wodim.

Today I'll try to document the WodimParser code as best as I can, watch my Git repo (http://git.sugarlabs.org/~natetheis/toaster/natetheis-mainline) for changes.

--Nate

On Thu, Dec 23, 2010 at 4:33 AM, Grant Bowman <gran...@gmail.com> wrote:

Jim,

Please see this code Nate has provided for Toaster.
http://git.sugarlabs.org/~natetheis/toaster/natetheis-mainline

Nate,

I am not familiar with the details of how to implement the STDIN.
STDOUT, and STDERR redirection that you have demonstrated with the
code you showed me and that we need for Toaster.  Can you please
respond to Jim's email and tell him?

Jim and Nate, would you please use the
http://groups.google.com/group/xotoaster/ list so that no discussion
goes unanswered and so that people like Alex Kleider can also
participate?

We do not have code checked in that functions yet.  As I have been
sick I have not followed through with getting Nate's code implemented
into the Toaster application or Nina's design ideas back to the group
or implemented.

I am sorry for the delays in the past weeks.  I hope we can all use
the mail list to demonstrate interest and ask the questions we need
answered in order to move forward together.

Grant Bowman


On Fri, Dec 3, 2010 at 12:14 PM, jim <j...@well.com> wrote:
>
>
>   how to implement IPC in Python on Linux?
>   There's a local open source group using python on
> fedora to create an app that looks like a classic
> threading problem: copying files has the effect of
> stalling until all files are written, and they'd
> like a progress bar.
>   They're interested in building separate processes
> rather than get into threading, and after a brief
> unproductive hunt in the os and sys modules, i
> figured some of you all would have not only answers
> but maybe best practices for writing python programs
> on the linux OS that can use InterProcess Communication
> to work together.
>
>
>
> The easy answer is to use the multiprocessing package:
> http://docs.python.org/library/multiprocessing.html
>
> Stuff looks like threads but is using multiple processes underneath.
>
>
> Try using the multiprocess module.
> The api is almost the same as the thread API, but it kicks off
> processes instead of threads
>
>
> Heh.  I just tried to use multiprocessing for the first time for hgit,
> my hg-to-git conversion program[1].  My rationale was probably similar
> to that of jim's group or many other people, i.e. threads are a pain,
> there's the GIL, etc.  But in the end I gave up and used threads.  The
> trouble was that everything I sent over the IPC channel had to be
> picklable, and worse, the target of a multiprocessing.Process cannot be
> any callable (as it can for thrading.Thread), but only a module-level
> function, i.e. no closures or instance methods.  These requirements
> together would pretty much force me to make all the interesting data
> global.
>
> [1] http://pypi.python.org/pypi/hgit/20101202
>
>
> How about D-BUS, or some other more general IPC solution? Python's got
> good DBUS bindings IIRC...
>
>
> Another solution I played with recently:
> http://www.rabbitmq.com/tutorial-one-python.html
>
>
>
>