Threading in Python
Andy
going on in the Python syntax/grammar/language, perhaps
the direction should be towards improving the internals
first?
One thing Java has which Python lacks is transparent
support for multi-cpu threading. Python has threads,
but I believe it is internally implemented and has
been shown not to take advantage of an SMP machine.
In Java otoh, when you use threads on an SMP machine,
it automatically takes advantage of multiple CPUs
(like BeOS). This simple plasma benchmark applet
http://rsb.info.nih.gov/plasma/
ran around 1.8x faster on my dual-Celeron machine
and the code was not even designed specially for
multiple CPUs in mind.
Peter Hansen
>
> There seems to be a lot of (arguably) radical changes
> going on in the Python syntax/grammar/language, perhaps
> the direction should be towards improving the internals
> first?
>
> One thing Java has which Python lacks is transparent
> support for multi-cpu threading. Python has threads,
> but I believe it is internally implemented and has
> been shown not to take advantage of an SMP machine.
I'm not an expert on SMP. In fact, I know next to
nothing about SMP. (But ignorance never stopped anyone
from posting an answer on Usenet. :-)
I'm quite sure Python threads will take advantage
of whatever the native threads take advantage of,
since Python threads are *not* "internally implemented"
quite like you probably imagine. On an OS and
platform with SMP support, different Python threads
probably execute on different CPUs.
Even so, you are probably right about there
being no advantage with multiple CPUs. The reason,
however, would be the "global interpreter lock",
which is a feature of the interpreter core which
protects against the usual thread safety problems.
I haven't done enough work down at that level yet
to be able to answer the question "could Python
readily be changed to reduce the negative impact
of the global interpreter lock on multithreaded
performance?"
--
----------------------
Peter Hansen, P.Eng.
pe...@engcorp.com
David Bolen
> One thing Java has which Python lacks is transparent
> support for multi-cpu threading. Python has threads,
> but I believe it is internally implemented and has
> been shown not to take advantage of an SMP machine.
Python threads are native OS threads (which I don't think is what you
mean by internally implemented).
However, you are correct that given the global interpreter lock (only
one thread can be executing actual Python bytecodes at any given
time), it's not very amenable to SMP systems. That is, unless the
multi-processing you want to do is off in an extension module that
releases the lock before doing it's work.
I have a feeling that changing the GIL would be a very large
undertaking. In some ways less arguable than stuff like the recent
division changes, since if done right it needn't break any existing
interfaces, but it would surely be a much higher risk set of changes.
Not that it's a bad idea, mind you - just a hard one.
--
-- David
--
/-----------------------------------------------------------------------\
\ David Bolen \ E-mail: db...@fitlinxx.com /
| FitLinxx, Inc. \ Phone: (203) 708-5192 |
/ 860 Canal Street, Stamford, CT 06902 \ Fax: (203) 316-5150 \
\-----------------------------------------------------------------------/
Martin von Loewis
> There seems to be a lot of (arguably) radical changes
> going on in the Python syntax/grammar/language, perhaps
> the direction should be towards improving the internals
> first?
People have started to work on free threading, but this turned quickly
out to be quite a radical change to the internals, to a degree that
third-party code would easily break - in addition to introducing many
new bugs.
That said, a contribution to implement free threading without any
negative impacts on performance on single-processor machines, and on
portability across versions, would surely be appreciated and reviewed
by quite a number of Python contributors (including myself).
Regards,
Martin
Skip Montanaro
(Andy, I'm not picking on you.)
andy> There seems to be a lot of (arguably) radical changes going on in
andy> the Python syntax/grammar/language, perhaps the direction should
andy> be towards improving the internals first?
andy> One thing Java has which Python lacks is transparent support for
andy> multi-cpu threading. Python has threads, but I believe it is
andy> internally implemented and has been shown not to take advantage of
andy> an SMP machine.
Folks, you should all know where the Python source code is. If not, I'll
remind you where you can download it:
http://sourceforge.net/projects/python/
Python is open source. That means you can make improvements and submit them
to SF. People do it all the time. PythonLabs is not a large organization
(a half dozen developers at most, with some kibbitzing from the rest of
Digital Creations) and they have other higher priority tasks that prevent
them from implementing a virtual machine that is 10x, 5x or even 2x faster
than the current implementation at the moment.
One of the advantages of open source is anyone can hack on it. One of the
disadvantages of open source is the development community takes what it can
get in the way of skills. Most of us interested in Python are probably
application programmers. Compiler experts are a lot harder to find. It may
well be that the Python community doesn't have a JIT compiler or SMP
multi-threading guru or two laying about with a few months free to devote to
such a project. (There's Tim Peters, but I never get the impression he has
a lot of time to spare.) Sun, on the other hand, has the money necessary to
go out and hire someone (several someones, more than likely) with the
necessary skills and make it their job to add JIT compilation or efficient
SMP multi-threading to Java. In fact, you could argue that they have to do
these sorts of things to keep Java from being crushed by Microsoft. Python
is generally not under such competitive pressures. Consequently, it's
likely that you'll see more whiz bang compiler stuff in the Java environment
for the foreseeable future than in the Python environment.
That doesn't mean it won't happen, but it might not happen at the same rate
as in other environments.
--
Skip Montanaro (sk...@pobox.com)
http://www.mojam.com/
http://www.musi-cal.com/
Aahz Maruch
Andy <an...@mindgate.net> wrote:
>
>There seems to be a lot of (arguably) radical changes going on in the
>Python syntax/grammar/language, perhaps the direction should be towards
>improving the internals first?
>
>One thing Java has which Python lacks is transparent support for
>multi-cpu threading. Python has threads, but I believe it is
>internally implemented and has been shown not to take advantage of an
>SMP machine.
That's about half-true. For more information, see
http://starship.python.net/crew/aahz/
(New & Improved slideshow coming Real Soon Now (a couple or three days))
--
--- Aahz <*> (Copyright 2001 by aa...@pobox.com)
Hugs and backrubs -- I break Rule 6 http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista
Fortune cookie: Watch your relations with other people carefully, be reserved.
Ype Kingma
Aahz Maruch wrote:
>
> In article <bn787.4044$Me1....@e420r-sjo2.usenetserver.com>,
> Andy <an...@mindgate.net> wrote:
> >
> >There seems to be a lot of (arguably) radical changes going on in the
> >Python syntax/grammar/language, perhaps the direction should be towards
> >improving the internals first?
> >
> >One thing Java has which Python lacks is transparent support for
> >multi-cpu threading. Python has threads, but I believe it is
> >internally implemented and has been shown not to take advantage of an
> >SMP machine.
>
> That's about half-true. For more information, see
> http://starship.python.net/crew/aahz/
>
(Very nice python threading code.)
> (New & Improved slideshow coming Real Soon Now (a couple or three days))
> --
Meanwhile you can have the best of both worlds: www.jython.org
Some threading features:
- jython interpreter threads are Java threads,
- java's 'synchronized' is used in the jython interpreter where necessary,
- in other words: fine grained locking is used, there is no 'interpreter lock'.
Regards,
Ype
--
email at xs4all.nl