Interrupting computation

[Raphael Mayoraz]

Guys,

I know this is a topic for the Python list, not the wxPython, but you
guys are so ressourceful that I dare submitting my problem to you.
As in most of the software, an 'Cancel computation' button is required
and I have trouble to make it work clean.

Here is how my application works. The Cancel computation button is, of
course, in the main thread.

MainThread (mostly the GUI)
|
ComputationThread1 (threading.Thread)
at the end of run(), wx.CallAfter(), starts the next thread
|
ComputationThread2 (threading.Thread)
|
StartProcesses (a regular class)
| | |
P1 P2 P3 (those are
multiprocessing.Process objects)

etc.

In StartProcesses, I define a multiprocessing.Manager object to collect
the list of PIDs, what is done in each process using os.getpid().
StartProcesses has also the run() method that creates P1,P2,P3 and uses
the join() method to make sure they are all terminated before going further.

Now, cancelling the computation: for processes, I get the Manager object
from StartProcesses and kill each process using os.kill(pid, 3). Not
sure the signal is the good one (I'm on Windows). For the thread, I turn
a flag off so that it knows the computation is interrupted. Basically,
it goes to wx.CallAfter() and there, instead of starting the next
thread, I just stop the computation workflow.

After some 'hit my head against the nearest wall' sessions, it works
more or less: when I cancel the computation, I get all processes (P1,
...) killed (at least, I hope, don't really know how to check), and only
the MainThread is still alive (I check this by using threading.enumerate()).

Now the problem: if I look at Windows Task manager -> Processes, I have
only one python.exe process when I start the application. When I launch
the computation, I get another one, than a few more when it gets to
running P1, P2, etc. When I cancel the computation, as all processes P1,
etc. and the ComputationThread are terminated, I should get back to only
one python process, but no, there are 2 of them! So, I'd like to know
why there is an extra python process still running, and what it is ???
When I quit the application, everything is gone (at least), but now, if
I do compute/cancel several times, without quitting the application,
everytime one extra python process is added, so after 3 loops, for
example, I have 4 python processes running.

Sorry, this is a bit complicated, but if anyone has a hint, that would
be great.

Thanks,

Raphael

[Andrea Gavana]

Hi,

I normally use the terminate() method of the multiprocessing Process
class, always from a separate thread (which monitors and eventually
kill the process itself). If you wish to check what and where is still
running, I normally do this check by looping through the processes and
using is_alive() (combined with multiprocessing.active_children()).

Andrea.

"Imagination Is The Only Weapon In The War Against Reality."
http://xoomer.alice.it/infinity77/

[Jay]

I use multiprocessing in several applications and have not seen this behavior.  That said I have not used multiprocessing.Manager before, but I think Manager itself is a Python process. If so, that might explain your extra process. Also, when I "kill" processes, I use the same method you used for threads: set a flag. With processes, I tend to send a stop message (via Queue or Pipe).

If you can share some code to reproduce the problem, I might be able to help more.

[Raphael Mayoraz]

Thanks Andrea and Jay,

Yep, multiprocessing.Manager is a process (what I ignored), and
multiprocessing.active_children() helps a lot figuring everything out.
It looks like everything is clean now (famous last word).

Raphael