Killing threads, and os.system()

[Laurent Claessens]

Hello all


I've a program that launches a lot of threads and each of them
launches a os.system("my_command").

My program also keeps a list of the launched threads, so I can make
"for" loops on the threads.


My aim is to kill everything with ctrl-C (KeyboardInterrupt).

Of course I tried to do

try:
[...]
except KeyboardInterrupt :
for task in task_list :
task.stop()
#task_list is the list of threads to be killed


It does not work.

How can I produce an "emergency" stop of all processes, including the
externals programs that were called by os.system() ?

My aim is of course to write an ultimate log file containing the status
of the program when KeyboardInterupt was raised. (if not, the unix
command "kill" does the job ;) )


Thanks for any help
have a good day
Laurent

[Laurent Claessens]

Le 31/01/2012 17:04, Dennis Lee Bieber a écrit :

> Of course, if that thread is stuck waiting for a call to os.system()
> to complete, then it can not do anything...
>
> os.system() is a rather limited, restrictive, call -- best used for
> quick one-of operations. If running Python 2.6+, I'd recommend
> converting from os.system() to subprocess.Popen(). .Popen() objects now
> have .terminate() and .kill() methods.

Ok, I'll try that Popen. Indeed I think that my threads are stuck
waiting os.system to complete.

Thanks
Laurent

[Laurent Claessens]

[John Nagle]

On 1/31/2012 8:04 AM, Dennis Lee Bieber wrote:
> ({muse: who do we have to kill
> to persuade OS designers to incorporate something like the Amiga ARexx
> "rexxport" system<G>}).

QNX, which is a real-time microkernel which looks like POSIX to
applications. actually got interprocess communication right. It
has to; everything in QNX is done by interprocess communication,
including all I/O. File systems and drivers are ordinary programs.
The kernel just handles message passing, CPU dispatching, and timers.
QNX's message passing looks more like a subroutine call than an
I/O operation, and this has important implications for efficient CPU
dispatching.

Any QNX system call that can block is really a message pass. Message
passes can be given a timeout, and they can be canceled from another
thread. The "system call" then returns with an error status. This
provides a way to keep threads from getting "stuck" in a system call.

(Unfortunately, QNX, which survived as a separate company for decades,
sold out to Harmon (car audio) a few years ago. They had no clue
what to do with an OS. They sold it to Research In Motion, the
Blackberry company, which is in the process of tanking.)

Python's thread model is unusually dumb. You can't send signals
to other threads, you can't force an exception in another thread, and
I won't even get into the appalling mess around the Global Interpreter
Lock. This has forced the use of subprocesses where, in other
languages, you'd use threads. Of course, you load a new copy of the
interpreter in each thread, so this bloats memory usage.

John Nagle

[Steven D'Aprano]

On Fri, 03 Feb 2012 00:14:33 -0800, John Nagle wrote:

> I won't even get into the appalling mess around the Global Interpreter
> Lock.

You know full well that IronPython and Jython don't have a GIL. If the
GIL was as harmful as you repeatedly tell us, why haven't you, and
everyone else, migrated to IronPython and Jython?

Oh yeah, maybe it's because CPython, even with the GIL, is significantly
faster than the two implementations without a GIL. Go figure.

But never mind the facts, spreading the FUD about Python is much more
fun. Hey, I hear that Python 3 is tanking too, and millions of Python
developers are rushing, rushing I say, to port their code to Go. Or was
it Ruby? Possibly Lua. Maybe VBScript.

> This has forced the use of subprocesses where, in other
> languages, you'd use threads.

Only if by "forced" you mean "not forced at all".

http://docs.python.org/library/multiprocessing.html
http://www.stackless.com/
http://pypi.python.org/pypi/greenlet/
http://twistedmatrix.com/trac/
http://www.gevent.org/
http://code.google.com/p/cogen/
http://www.kamaelia.org/Home
http://code.google.com/p/syncless/
http://opensource.hyves.org/concurrence/
http://www.tornadoweb.org/
http://docs.python.org/library/asyncore.html
http://pyro.sourceforge.net/
http://wiki.python.org/moin/ParallelProcessing


--
Steven

[Paul Rubin]

John Nagle <na...@animats.com> writes:
> QNX's message passing looks more like a subroutine call than an
> I/O operation,

How do they enforce process isolation, or do they decide they don't need to?