I just killed GIL!!!
sturlamolden
I just had one moment of exceptional clarity, during which realized
how I could get the GIL out of my way... It's so simple, I cannot help
wondering why nobody has thought of it before. Duh! Now I am going to
sit and and marvel at my creation for a while, and then go to bed
(it's past 2:30 a.m.) Tomorrow I will contemplate whether to sell this
little secret for big bucks, give it away for free, or just keep it to
myself... :-)
Now you are probably thinking I reinvented the gunpowder, and are
running multiple processes. Not so. I am not running parallel
processes, like parallel python or the processing module in cheese
shop. I am running multiple THREADS. In fact, I am just using
threading.Thread. The source code is pure Python, so there is no C
magic, and I only used the stuff that's already there in the standard
library. So, I just made CPython do what everyone claim to be
impossible. One single process of CPython is using all the cpu power
of my dual-core laptop.
Daniel Fetchinson
If I were you I would keep it a secret until a Hollywood producer
offers big bucks for the film rights.
Tim Daneliuk
Who would play Guido, I wonder?
--
----------------------------------------------------------------------------
Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/
Torsten Bronger
Tim Daneliuk writes:
> Daniel Fetchinson wrote:
>
>> [...]
>>
>>> I just had one moment of exceptional clarity, during which
>>> realized how I could get the GIL out of my way... It's so
>>> simple, I cannot help wondering why nobody has thought of it
>>> before. [...]
>>
>> If I were you I would keep it a secret until a Hollywood producer
>> offers big bucks for the film rights.
>
> Who would play Guido, I wonder?
Ralf Möller. No other.
Tschö,
Torsten.
--
Torsten Bronger, aquisgrana, europa vetus
Jabber ID: bro...@jabber.org
(See http://ime.webhop.org for further contact info.)
Matias Surdi
sturlamolden escribió:
Jonathan Gardner
> One single process of CPython is using all the cpu power
> of my dual-core laptop.
Are they stuck in a while loop, waiting for their resource to become
available?
Using 100% of the CPU is a bug, not a feature. If you can't rewrite
your algorithm to be disk or network bound, next optimization step is
C.
Steve Holden
Sorry. I'm definitely a skeptic on this story. Put up or shut up.
regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/
Carl Banks
wrote:
> Using 100% of the CPU is a bug, not a feature.
No it isn't. That idea is borne of the narrowmindedness of people who
write server-like network apps. What's true for web servers isn't
true for every application.
> If you can't rewrite
> your algorithm to be disk or network bound, next optimization step is
> C.
I'm sorry, but I don't like being told to use C. Perhaps I would like
the expressiveness of Python, am willing to accept the cost in
performance, but would also like to take advantage of technology to
get performance gains when I can? What's so unreasonable about that?
Look, the GIL is not a good thing. It's a trade off. Probably a good
one, too, considering the history of Python's development and the way
it simplifies writing extensions. But it's not, by itself, a good
thing.
For the record, I am not complaining about that GIL. As I said, I
understand and approve of why it's there. I am, however, complaining
about attitude that if you want to be free of the GIL you're doing
something wrong.
Carl Banks
"Martin v. Löwis"
> understand and approve of why it's there. I am, however, complaining
> about attitude that if you want to be free of the GIL you're doing
> something wrong.
If you _want_ to be free of the GIL, you are not _doing_ anything, and
that may or may not be wrong. If you are complaining about the GIL,
I think you are doing something wrong, because complaining doesn't
help progress at all. I think neither was the case in this thread -
the guy claimed that he actually did something about the GIL, and
now we are all waiting for him to also tell us what it is that he
did. I think it is somewhat wrong to not tell in the first place,
but this is free software, and choosing not to contribute isn't
inherently wrong. Maybe the guy is making fun of us; whether that
is wrong or not depends on your notion of humor.
Regards,
Martin
Marco Mariani
>>> If I were you I would keep it a secret until a Hollywood producer
>>> offers big bucks for the film rights.
>> Who would play Guido, I wonder?
>
> Ralf Möller. No other.
And the GIL killer?
Clive Owen, Matt Damon, Mark Wahlberg?
Chris
Marky Mark ftw, he could even rap about it his method.
Martin P. Hellwig
You killed the GIL, you bastard! :-)
> Hello Guys...
>
> I just had one moment of exceptional clarity, during which realized
> how I could get the GIL out of my way... It's so simple, I cannot help
> wondering why nobody has thought of it before. Duh! Now I am going to
> sit and and marvel at my creation for a while, and then go to bed
> (it's past 2:30 a.m.) Tomorrow I will contemplate whether to sell this
> little secret for big bucks, give it away for free, or just keep it to
> myself... :-)
>
Sounds good, though I have a question.
Is it transparent, meaning I can just write my code the way I used too
and it makes the decision for itself to spread the tasks on multiple
CPU's without me to worry about shared memory, spinlocks, deadlocks and
out of order inter-depending processing of subtask without heaving
having so much overhead that it has no performance advantage?
If not, what is the advantage above already present solutions?
--
mph
Steve Holden
OP is a fertile source of ideas, not all of them fully baked, shall we
say. I agree that the coyness in not immediately revealing the technique
is less than helpful, and increases my assessment that the assertion of
having "killed the GIL" will turn out to be wrong.
I'd love to be wrong about that, but the GIL *has* been the subject of
extensive efforts to kill it over the last five years, and it has
survived despite the best efforts of the developers.
sturlamolden
> If not, what is the advantage above already present solutions?
Well... I like the processing module. Except that Wintendo toy OS has
no fork() availabe for the Win32 subsystem, which makes it a bit
limited on that platform (slow at starting up, dependent on object
serialization, difficulties handling global variables, etc). The
advantage of my solution is that I can use threads and one process,
which makes it more friendly to Windows. On the other hand, the
processing module has the advantage that it cna be used on distributed
memory systems (e.g. clusters).
sturlamolden
> help progress at all. I think neither was the case in this thread -
> the guy claimed that he actually did something about the GIL, and
> now we are all waiting for him to also tell us what it is that he
> did.
Ok, I did not remove the GIL, but I found a way to remove its
notorious side effect on SMPs. So I am going to reveal it now. Forgive
my strange sence of humor at 3 o'clock in the morning.
First, think about this:
(1) What is the GIL?
(2) Where does it live?
(3) How can it be manually released?
The answer to this is:
(1) A critical section (a lock/mutex object)
(2) As a global object in Python25.dll on my computer
(3) Using Python's C API or calling methods in a ctypes.CDLL object
The Python C API has the ability to embed Python interpreters. You do
this by importing Python25.dll into the process. ctypes has the
ability to call functions in a DLL. So is it possible to embed Python
in Python? And what would be consequence be?
First, if I try to load a DLL more than once, Windows will detect this
and just give me a handle to the currently imported library. So by
importing Python25.dll with ctypes, I can just make my Python
interpreter talk to itself through its own CAPI. Yes I can create sub
interpreters using PyNew_Interpreter, but they all share the same GIL,
so it's not useful here.
So here is what I suddendly realized, and please don't laugh, its so
simple its almost silly:
I make some copies of Python25.dll, and call them Python25-1.dll,
Python25-2.dll, Python25-3.dll, Python25-4.dll, etc. Then I load them
all into my current Python process as ctypes.CDLL objects. Tada! I now
have a pool of independent Python interpreters, not sharing the GIL,
but still living in the same process.
I can make the different interpreters talk to each other, including
manipulating each other's GIL, using the using ctypes and Python's C
API for embedding.
So why does this circumvent the GIL? Because ctypes releases it before
calling functions form a CDLL object.
If I use my main interpreter to delegate a task to one of its embedded
'children', its GIL will be released while it is waiting for the
answer. Associating each embedded interpreter with a threading.Thread
is all that remains. The GIL is released while the thread operating
the child interpreter is blocked.
An there you have the answer. It's really very simple :-)
Regards,
Sturla Molden
sturlamolden
> It's april 1st again???
Not according to my calendar. This was not meant as a joke. I think I
may have solved the GIL issue. See my answer to Martin v. Löwis for a
full explanation.
sturlamolden
> Quick, write it down before the drugs wear off.
Hehe, I don't take drugs, apart from NSAIDs for arthritis. Read my
answer to Martin v. Löwis.
MRAB
wrote:
> Hallöchen!
>
> Tim Daneliuk writes:
> > Daniel Fetchinson wrote:
>
> >> [...]
>
> >>> I just had one moment of exceptional clarity, during which
> >>> realized how I could get the GIL out of my way... It's so
> >>> simple, I cannot help wondering why nobody has thought of it
> >>> before. [...]
>
> >> If I were you I would keep it a secret until a Hollywood producer
> >> offers big bucks for the film rights.
>
> > Who would play Guido, I wonder?
>
> Ralf Möller. No other.
>
American hero, Guy Ross! :-)
Hrvoje Niksic
> If I use my main interpreter to delegate a task to one of its
> embedded 'children', its GIL will be released while it is waiting
> for the answer. Associating each embedded interpreter with a
> threading.Thread is all that remains. The GIL is released while the
> thread operating the child interpreter is blocked.
Have you tackled the communication problem? The way I see it, one
interpreter cannot "see" objects created in the other because they
have separate pools of ... everything. They can communicate by
passing serialized objects through ctypes, but that sounds like the
solutions that use processes.
Tim Daneliuk
"DIE HARD WITH NO GIL - He's Back And He's Pissed..."
Rhamphoryncus
Interesting. Windows specific, but there's other ways to do the same
thing more portably.
The bigger issue is that you can't share any objects. This
effectively gives you a multiprocess model - a bit cheaper than that,
but not enough to really supply GIL-free threading.
Rhamphoryncus
> I'd love to be wrong about that, but the GIL *has* been the subject of
> extensive efforts to kill it over the last five years, and it has
> survived despite the best efforts of the developers.
sturlamolden
> Have you tackled the communication problem? The way I see it, one
> interpreter cannot "see" objects created in the other because they
> have separate pools of ... everything. They can communicate by
> passing serialized objects through ctypes, but that sounds like the
> solutions that use processes.
I see two solutions to that.
It is possible to use fine-grained locking on the objects that need to
be communicated. ctypes can pass PyObject* pointers
(ctypes.py_object), or we could resort to some C. When interpreter A
get a PyObject* pointer from B (a total of four bytes on my computer),
interpreter A can:
(1) Acquire B's GIL
(2) Increment the refcount of the PyObject*
(3) Release B's GIL
(4) Use the object as its own (synchronization is required)
Sharing PyObject* pointer has the possibility of shooting your leg
off. But I believe the small quirks can be worked out.
The other option is to use serialized Queues like the processing
module in cheese shop. I don't think it will be faster than similar
mechanisms based on IPC, because object serialization and
deserialization are the more expensive parts.
Jonathan Gardner
> On Apr 17, 3:37 am, Jonathan Gardner <jgard...@jonathangardner.net>
> wrote:
> > If you can't rewrite
> > your algorithm to be disk or network bound, next optimization step is
> > C.
>
> I'm sorry, but I don't like being told to use C. Perhaps I would like
> the expressiveness of Python, am willing to accept the cost in
> performance, but would also like to take advantage of technology to
> get performance gains when I can? What's so unreasonable about that?
>
If you're satisfied then don't take the next optimization step.
sturlamolden
> Interesting. Windows specific, but there's other ways to do the same
> thing more portably.
I believe you can compile Python as a shared object (.so) on Linux as
well, and thus loadable by ctypes.
> The bigger issue is that you can't share any objects.
Why not?
> This
> effectively gives you a multiprocess model - a bit cheaper than that,
> but not enough to really supply GIL-free threading.
That solution is safe. But I am looking into sharing objects. I don't
think its impossible.
PyObject* pointers can be passed around. GILs can be acquired and
released, refcounts increased and decreased, etc. but we have to sort
out some synchronization details for the shared objects. For one
thing, we have to make sure that a garbage collector does not try to
reclaim a PyObject* belonging to another interpreter. But here we are
talking about minor changes to CPython's source, or perhaps none at
all.
Jonathan Gardner
> I'd love to be wrong about that, but the GIL *has* been the subject of
> extensive efforts to kill it over the last five years, and it has
> survived despite the best efforts of the developers.
>
To add to that...
In my mind, I see three options for multi-process systems:
(1) Locks.
(2) A global lock (GIL)
(3) Learning to live with the possibility of things disappearing out
from under you.
In the SQL world, they chose (3). In the Java/C++/C# world, they chose
(1). I like Python's compromise a lot, even though it means in a
single process, you can only have one thread doing Python at a time.
Usually the bits I want to parallelize on are blocking system calls to
the network or disk anyway, or the result of a long calculation that
updates its result all at once. So having the OS handle the tough bits
while I program in a fantasy world where threads are an illusion is
fine with me.
Discovering a way to get rid of the GIL and not have to do (1) and (3)
is truly exciting, but I've lost hope a long time ago.
Besides, if it gets in the way I can always do something novel like, I
don't know, spawn another Python process?
Jonathan Gardner
>
> An there you have the answer. It's really very simple :-)
>
That's an interesting hack.
Now, how do the processes communicate with each other without stepping
on each other's work and without using a lock?
Once you get that solved, I am sure the entire computing world will
beat a path to your door. (Hint: This is the kind of thing Stroustrup,
Guido, and Alonzo Church have thought a lot about, just to name a
few.)
By the way, you do know that you can recompile Python from source
code, and that you have the freedom to modify that source code. If you
want to remove the GIL and see what happens, just make the calls to
acquire and release the GIL do nothing. See how far that will get you.
Jonathan Gardner
> On Apr 17, 5:46 pm, Hrvoje Niksic <hnik...@xemacs.org> wrote:
>
> > Have you tackled the communication problem? The way I see it, one
> > interpreter cannot "see" objects created in the other because they
> > have separate pools of ... everything. They can communicate by
> > passing serialized objects through ctypes, but that sounds like the
> > solutions that use processes.
>
> I see two solutions to that.
>
> It is possible to use fine-grained locking on the objects that need to
> be communicated.
And you'll pay a price for every lock/unlock operation, in addition to
the added complexity of the code (which you are already beginning to
see.) That's been tried in Python, and everyone agreed that the GIL
was the better compromise.
> The other option is to use serialized Queues like the processing
> module in cheese shop.
You mean pipes, files, and sockets?
You should check out Stackless's channels. Not a new or unique
concept, but a very powerful one that everyone should be familiar with.
Rhamphoryncus
> On Apr 17, 6:03 pm, Rhamphoryncus <rha...@gmail.com> wrote:
>
> > Interesting. Windows specific, but there's other ways to do the same
> > thing more portably.
>
> I believe you can compile Python as a shared object (.so) on Linux as
> well, and thus loadable by ctypes.
Python is compiled as a .so, but I don't believe that makes everything
private to that .so. Other means may be necessary.
Not that important though.
> > This
> > effectively gives you a multiprocess model - a bit cheaper than that,
> > but not enough to really supply GIL-free threading.
>
> That solution is safe. But I am looking into sharing objects. I don't
> think its impossible.
>
> PyObject* pointers can be passed around. GILs can be acquired and
> released, refcounts increased and decreased, etc. but we have to sort
> out some synchronization details for the shared objects. For one
> thing, we have to make sure that a garbage collector does not try to
> reclaim a PyObject* belonging to another interpreter. But here we are
> talking about minor changes to CPython's source, or perhaps none at
> all.
But can you automatically manage the reference count? ie, does your
interpreter have a proxy to the other interpreter's object, or does
the object itself gain a field indicating who owns it?
Either way you'll need to keep the number of shared objects to a
minimum, as the use of locking creates a bottleneck - only one thread
can run at a time for a given object.
sturlamolden
wrote:
> On Apr 17, 8:19 am, sturlamolden <sturlamol...@yahoo.no> wrote:
>
>
>
> > An there you have the answer. It's really very simple :-)
>
> That's an interesting hack.
>
> Now, how do the processes communicate with each other without stepping
> on each other's work and without using a lock?
Why can't I use a lock?
There is a big difference between fine-grained locking on each object
(cf. Java) and a global lock for everything (cf. CPython's GIL). Fine-
grained locking for each object has been tried, and was found to be a
significant slow down in the single-threaded case.
What if we just do fine grained locking on objects that need to be
shared?
What if we accept that "shared" objects are volatile and may suddenly
disappear (like a weakref), and trap that as an exception?
Carl Banks
wrote:
Where do you see the word "satisfied" in what I wrote?
Carl Banks
"Martin v. Löwis"
I'm fairly skeptical that it actually works. If the different
Python interpreters all import the same extension module
(say, _socket.pyd), windows won't load the DLL twice, but
only one time. So you end up with a single copy of _socket,
which will have a single copy of socket.error, socket.socket,
and so on.
The single copy of socket.error will inherit from one specific
copy of IOError. So if you import socket in a different
interpreter, and raise socket.error there, and try to catch
IOError, the exception won't be caught - because *that*
IOError is then not a base class of socket.error.
A more general approach similar to this one is known
as ".NET appdomains" or "Java isolates". It requires
the VM to refrain from having any global state, and
to associate all "global" state with the appdomain
or isolate. Python can't easily support that kind of
model, because extension modules have global state
all the time.
Even if Python supported appdomains, you still wouldn't
get any object sharing out of it. As soon as you start
to share some object (but not their types), your entire
type hierarchy gets messed up.
FWIW, Tcl implements this model precisely: Tcl is
not thread-safe in itself, but supports a
interpreter-per-thread model. I'm also skeptical that
this is any better than the GIL.
Regards,
Martin
Nick Craig-Wood
Sounds very interesting. I particularly liked this bit from the web
page - an excellent solution to fine grained locking. Sending only
immutable objects between threads is very like the functional approach
used by Erlang which is extremely good at concurrency.
------------------------------------------------------------
Which objects can be shared between threads
You probably know how to append to a list or modify a dict. When
confronted with threading though you may start to wonder, what happens
if two threads modify a list or a dict simultaneously? There's two
common answers to this:
* 1. Corruption. This is the default in C, and to a lesser degree
Java. It doesn't limit you much and can be faster, but it requires
some deep voodoo to know when you're using it right.
* 2. Locks. All the operations internally lock the object. This
makes them individually correct, but they'll be wrong again if you
try to compose them into larger operations. It also adds a
significant performance penalty.
My priority is making it easy to write correct programs, and neither
of those options are good enough. Instead, I take a third option as my
default:
* 3. Make sharing impossible, avoiding the question. list and dict
cannot be shared between threads. Any attempt to pass a list or
dict into something like a Queue will raise a TypeError. This
ensures any thread interaction is explicit, rather than
implicit. See also: The Zen of Python
Of course if you couldn't share any objects then you'd just have
processes, which are quite awkward to use. Instead, I only make
mutable objects like list and dict unshareable, while immutable int
and str objects can still be shared. Further, mutable objects that
provide an explicit API for use between threads are also shareable.
------------------------------------------------------------
--
Nick Craig-Wood <ni...@craig-wood.com> -- http://www.craig-wood.com/nick
Rhamphoryncus
> Rhamphoryncus <rha...@gmail.com> wrote:
> > On Apr 17, 7:40 am, Steve Holden <st...@holdenweb.com> wrote:
> > > I'd love to be wrong about that, but the GIL *has* been the subject of
> > > extensive efforts to kill it over the last five years, and it has
> > > survived despite the best efforts of the developers.
>
> > Yo. http://code.google.com/p/python-safethread/
>
> Sounds very interesting. I particularly liked this bit from the web
> page - an excellent solution to fine grained locking. Sending only
> immutable objects between threads is very like the functional approach
> used by Erlang which is extremely good at concurrency.
Although superficially similar, the details of Erlang are actually
pretty different. It copies all objects passed between threads - it
was originally designed for fault tolerance (entire nodes going down),
not concurrency. If you want a shared mutable object you need to use
a "process" as one, treating it as an actor. Hopefully you can do
most of it in a one-way, message driven style, as otherwise you're
going to be transforming your synchronous calls into a series of
callbacks.
If you have a node farm, you want to upgrade it incrementally, and you
want it to be fault tolerant (nodes go down at random without kill the
whole thing), Erlang is much better than safethread. That's at a
significant cost though, as it's only good at the one style.
Safethread is much better at a style useful on a single desktop.
sjde...@yahoo.com
> On 17 Apr, 15:21, "Martin P. Hellwig" <x...@xs4all.nl> wrote:
>
> > If not, what is the advantage above already present solutions?
>
> Well... I like the processing module. Except that Wintendo toy OS has
> no fork() availabe for the Win32 subsystem
Passing a NULL SectionHandle to NTCreateProcess/CreateProcessEx
results in a fork-style copy-on-write duplicate of the current process.
sturlamolden
> Passing a NULL SectionHandle to NTCreateProcess/CreateProcessEx
> results in a fork-style copy-on-write duplicate of the current process.
I know about NtCreateProcess and ZwCreateProcess, but they just create
an empty process - no context, no thread(s), no DLLs loaded, etc.
There is even an example code of how to implement fork() with
ZwCreateProcess in Nebbet's book on NT kernel internals, but
apparently it doesn't work quite well. (Right now I cannot even make
it compile, because WDK headers are fubar with invalid C; even
Microsoft's own compiler does not accept them.)
Searching with Google, I find several claims that there is a
"CreateProcessEx", which can do a COW fork of a process in the Win32
subsystem. I cannot find it documented anywhere. It is also not
exported by kernel32.dll. If you know how this function is defined and
which DLL exports it, please post it. But I suspect it does not
exist.
sjde...@yahoo.com
> On 18 Apr, 21:28, "sjdevn...@yahoo.com" <sjdevn...@yahoo.com> wrote:
>
> > Passing a NULL SectionHandle to NTCreateProcess/CreateProcessEx
> > results in a fork-style copy-on-write duplicate of the current process.
>
> I know about NtCreateProcess and ZwCreateProcess, but they just create
> an empty process - no context, no thread(s), no DLLs loaded, etc.
> There is even an example code of how to implement fork() with
> ZwCreateProcess in Nebbet's book on NT kernel internals, but
> apparently it doesn't work quite well.
It works fine for a copy-on-write process creation. It doesn't work
100% compatibly to fork. Nebbet is the best reference out there on
the method.
FWIW, NT's POSIX subsytem fork() uses (or used to use) the NULL
SectionHandle method and was POSIX certified, so it's certainly
possible.
> Searching with Google, I find several claims that there is a
> "CreateProcessEx"
Yeah my bad, I meant zwCreateProcess. It's been almost a decade now
since I used it.
sturlamolden
wrote:
> FWIW, NT's POSIX subsytem fork() uses (or used to use) the NULL
> SectionHandle method and was POSIX certified, so it's certainly
> possible.
Windows Vista Ultimate comes with Interix integrated, renamed
'Subsystem for Unix based Applications' or SUA for short. Interix is
even UNIX certified when a C compiler is installed. Windows also has a
OS/2 subsystem which has a COW fork. Yes it is possible. One may
wonder why the Win32 subsystem don't have this feature. Perhaps fork()
is unfriendly to threads, like fork on Linux used to be (or is?)
pthread unfriendly. Or perhaps M$ (MegaDollar) just did this to be
mean. I don't know. I see the lack of fork() in Win32 as one of the
major shortcomings of Windows.
Anyhow, I just downloaded the WDK which supersedes the DDK. The
examples in Nebbet's book do not build anymore, as there are invalid C
in the WDK header files. :-(
Aahz
Carl Banks <pavlove...@gmail.com> wrote:
>On Apr 17, 3:37 am, Jonathan Gardner <jgard...@jonathangardner.net>
>wrote:
>>
>
>No it isn't. That idea is borne of the narrowmindedness of people who
>write server-like network apps. What's true for web servers isn't
>true for every application.
Only when you have only one application running on a machine.
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/
Why is this newsgroup different from all other newsgroups?
Carl Banks
> In article <b6de5e65-19b9-40b4-b9b9-f4cf0e61b...@l42g2000hsc.googlegroups.com>,
> Carl Banks <pavlovevide...@gmail.com> wrote:
>
> >On Apr 17, 3:37 am, Jonathan Gardner <jgard...@jonathangardner.net>
> >wrote:
>
> >> Using 100% of the CPU is a bug, not a feature.
>
> >No it isn't. That idea is borne of the narrowmindedness of people who
> >write server-like network apps. What's true for web servers isn't
> >true for every application.
>
> Only when you have only one application running on a machine.
Needless pedantry.
"Using 100% of the CPU time a OS allow a process to have is not
necessarily a bug." Happy?
Carl Banks
Aahz
Not really; my comment is about the same level of pedantry as yours.
Jonathan's comment was clearly in the context of inappropriate CPU usage
(e.g. spin-polling). Obviously, there are cases where hammering on the
CPU for doing a complex calculation may be appropriate, but in those
cases, you will want to ensure that your application gets as much CPU as
possible by removing all unnecessary CPU usage by other apps.
Carl Banks
> In article <d1548d12-85b6-45cc-8010-0bde9c1c3...@u12g2000prd.googlegroups.com>,
> Carl Banks <pavlovevide...@gmail.com> wrote:
>
>
>
> >On Apr 20, 10:57 pm, a...@pythoncraft.com (Aahz) wrote:
> >> In article <b6de5e65-19b9-40b4-b9b9-f4cf0e61b...@l42g2000hsc.googlegroups.com>,
> >> Carl Banks <pavlovevide...@gmail.com> wrote:
> >>>On Apr 17, 3:37 am, Jonathan Gardner <jgard...@jonathangardner.net>
> >>>wrote:
>
> >>>> Using 100% of the CPU is a bug, not a feature.
>
> >>>No it isn't. That idea is borne of the narrowmindedness of people who
> >>>write server-like network apps. What's true for web servers isn't
> >>>true for every application.
>
> >> Only when you have only one application running on a machine.
>
> >Needless pedantry.
>
> >"Using 100% of the CPU time a OS allow a process to have is not
> >necessarily a bug." Happy?
>
> Not really; my comment is about the same level of pedantry as yours.
> Jonathan's comment was clearly in the context of inappropriate CPU usage
> (e.g. spin-polling).
That's far from evident. Jonathan's logic went from "I'm using 100%
CPU" to "You must be spin-polling". At best, Jonathan was making some
unsupported assumptions about the type of program sturlamolden had in
mind, and criticized him based on it. But frankly, I've seen enough
people who seem to have no conception that anyone could write a useful
program without an I/O loop that it wouldn't surprise me it he meant
it generally.
> Obviously, there are cases where hammering on the
> CPU for doing a complex calculation may be appropriate, but in those
> cases, you will want to ensure that your application gets as much CPU as
> possible by removing all unnecessary CPU usage by other apps.
Nonsense. If I'm running a background task on my desktop, say
formating a complex document for printing, I would like it to take up
as much of CPU as possible, but still have secondary priority to user
interface processes so that latency is low.
Carl Banks