Python and Boehm-Demers GC, I have code.
Neil Schemenauer
I have added the Boehm-Demers garbage collector to Python and it
works well! With the default set of extension modules all tests
pass. Also I have done some benchmarks using pystone and
pybench. The GC version of Python is just as fast as the regular
version! I was quite amazed. Here are the pybench results:
PYBENCH 0.5
Benchmark: free3 (rounds=3, warp=10)
Tests: per run per op. diff *
------------------------------------------------------------------------
BuiltinFunctionCalls: 746.83 ms 2.93 us -11.90%
BuiltinMethodLookup: 1139.33 ms 1.09 us -1.07%
CreateInstances: 1006.83 ms 11.99 us -9.09%
DictCreation: 847.17 ms 2.82 us -21.66%
IfThenElse: 1357.67 ms 1.01 us -1.84%
ListSlicing: 1116.33 ms 127.58 us +38.30%
NestedForLoops: 970.00 ms 1.29 us -1.40%
NormalClassAttribute: 1126.50 ms 0.94 us -1.46%
NormalInstanceAttribute: 1086.33 ms 0.91 us -2.75%
PythonFunctionCalls: 1083.50 ms 3.28 us +3.63%
PythonMethodCalls: 837.83 ms 5.59 us +1.23%
Recursion: 981.50 ms 39.26 us +16.37%
SimpleDictManipulation: 934.67 ms 1.56 us -7.96%
SimpleFloatArithmetic: 1112.83 ms 1.01 us +4.95%
SimpleIntegerArithmetic: 1000.83 ms 0.76 us -2.48%
SimpleListManipulation: 1213.50 ms 2.25 us +36.74%
SmallLists: 1132.67 ms 2.22 us -15.34%
SmallTuples: 1198.50 ms 2.50 us -3.44%
SpecialClassAttribute: 1132.83 ms 0.94 us -1.51%
SpecialInstanceAttribute: 1299.67 ms 1.08 us -2.44%
TupleSlicing: 2027.17 ms 3.86 us -5.52%
------------------------------------------------------------------------
Average round time: 26870.00 ms -1.39%
*) measured against: py152 (rounds=10, warp=10)
I started with Sam Rushing's patch and modified it for Python
1.5.2c. To my surprise, removing the reference counting
(Py_INCREF, Py_DECREF) actually slowed down Python a lot (over
two times for pystone). I not exactly sure why this is. Maybe
someone can enlighten me. I will try to do some more testing.
I have to fix up modules which use other mallocs (readline and
_tkinter for sure). Guido, will a patch be accepted which
changes Python code to use Py_Mem_* instead of the libc names
where approprate? I will have to go through the extension
modules and fix them by hand. Also, what about a patch that
makes using garbage collection a configure option. The changes
are quite minor. I don't know autoconf but maybe I can figure it
out.
With regards to finalization, __del__ methods are not called from
the garbage collector. If you create reference loops then too
bad for you. The garbage collector supports finalization so I
may fix this later.
A preliminary patch is attached. The patch assumes libgc.a and
gc.h are installed somewhere and that you have already run
configure. The Makefile diffs are probably okay only for a Linux
box. I must learn autoconfig. :(
Neil
diff -u -r Python-1.5.2c1/Include/mymalloc.h Python-gc/Include/mymalloc.h
--- Python-1.5.2c1/Include/mymalloc.h Fri Dec 4 11:48:10 1998
+++ Python-gc/Include/mymalloc.h Fri Jul 16 03:49:55 1999
@@ -87,6 +87,18 @@
#define _PyMem_EXTRA 0
#endif
+#ifdef Py_USE_GC
+#include <gc.h>
+#define malloc(n) GC_MALLOC(n)
+#define calloc(m,n) GC_MALLOC((m)*(n))
+#define realloc(o,n) GC_REALLOC(o,n)
+#ifdef Py_USE_GC_FREE
+ #define free(n) GC_FREE(n)
+#else
+ #define free(n)
+#endif /* Py_USE_GC_FREE */
+#endif /* Py_USE_GC */
+
#define PyMem_NEW(type, n) \
( (type *) malloc(_PyMem_EXTRA + (n) * sizeof(type)) )
#define PyMem_RESIZE(p, type, n) \
diff -u -r Python-1.5.2c1/Makefile Python-gc/Makefile
--- Python-1.5.2c1/Makefile Fri Jul 16 04:08:43 1999
+++ Python-gc/Makefile Fri Jul 16 04:08:37 1999
@@ -143,7 +143,7 @@
WITH=
# Compiler options passed to subordinate makes
-OPT= -g -O2
+OPT= -g -O2 -DPy_USE_GC
# Subdirectories where to run make recursively
SUBDIRS= Parser Objects Python Modules
diff -u -r Python-1.5.2c1/Modules/Makefile.pre Python-gc/Modules/Makefile.pre
--- Python-1.5.2c1/Modules/Makefile.pre Fri Jul 16 04:08:44 1999
+++ Python-gc/Modules/Makefile.pre Fri Jul 16 04:16:06 1999
@@ -28,7 +28,7 @@
SGI_ABI=
DEFS= -DHAVE_CONFIG_H
-LIBS= -lieee -ldl
+LIBS= -lieee -ldl -lgc
LIBM= -lm
LIBC=
diff -u -r Python-1.5.2c1/Objects/Makefile Python-gc/Objects/Makefile
--- Python-1.5.2c1/Objects/Makefile Fri Jul 16 04:08:43 1999
+++ Python-gc/Objects/Makefile Fri Jul 16 04:09:17 1999
@@ -13,7 +13,7 @@
RANLIB= ranlib
AR= ar
-DEFS= -DHAVE_CONFIG_H
+DEFS= -DHAVE_CONFIG_H -DPy_USE_GC_FREE
# === Other things that are customizable but not by configure ===
diff -u -r Python-1.5.2c1/Parser/Makefile Python-gc/Parser/Makefile
--- Python-1.5.2c1/Parser/Makefile Fri Jul 16 04:08:43 1999
+++ Python-gc/Parser/Makefile Fri Jul 16 04:08:52 1999
@@ -14,7 +14,7 @@
AR= ar
DEFS= -DHAVE_CONFIG_H
-LIBS= -lieee -ldl
+LIBS= -lieee -ldl -lgc
# === Other things that are customizable but not by configure ===
Tim Peters
> I have added the Boehm-Demers garbage collector to Python and it
> works well!
That's good, but I'm not clear on what this means. The patch didn't seem to
do more than replace malloc/calloc/realloc/free with the BDW versions; in
particular, Py_USE_GC_FREE was #define'd, and you ran tests without
significant cyclic structures, so it doesn't *appear* that anything more
happened here than that a different implementation of the malloc family got
plugged in. Did the "collection" phase of BDW find any trash at all?
> I started with Sam Rushing's patch and modified it for Python
> 1.5.2c. To my surprise, removing the reference counting
> (Py_INCREF, Py_DECREF) actually slowed down Python a lot (over
> two times for pystone). I not exactly sure why this is. Maybe
> someone can enlighten me.
My guess is that simple refcounting has excellent cache behavior in Python,
and GC doesn't. Unlike as in almost any other language, the little loop
for i in xrange(10000000):
pass
creates a huge amount of heap trash at a furious pace (even ints "are boxed"
in Python). RC can reuse the same little bit of storage over & over (the
trash is recycled as soon as it's created), while delayed GC chews up some
large amount of address space over & over instead. You can test this by
timing that loop alone (with and without refcounting).
Note too that Python does no stack allocation of locals or frames or
anything else: everything is heap-based, with the most speed-critical
allocations augmented by assorted internal fast free lists. Disable the
refcounting, and the finalizers on the "intense" objects don't get called,
and then the fast free lists don't get used either. So the allocation speed
optimizations Python implements get castrated too.
I've always said it would take a very sophisticated mark-&-sweep to beat
RC's performance for *Python* (it's not Lisp, or Smalltalk, or C++, or ...
under the covers); if true (& I haven't seen anything yet that changes my
mind <wink>), you're at the first step of very long road if you want to make
BDW the primary approach; but it would be great if BDW could be invoked
"just sometimes" to reclaim the cycles RC can't catch.
semi-encouraging-ly y'rs - tim
Neil Schemenauer
>I have to fix up modules which use other mallocs (readline and
>_tkinter for sure). Guido, will a patch be accepted which
>changes Python code to use Py_Mem_* instead of the libc names
>where approprate? I will have to go through the extension
>modules and fix them by hand.
It seems that most of the Python code was already using the
PyMem_* functions. I only had to fix a few things. The garbage
collecting version is still as fast as the regular version. I
think readline, zlib, and _tkinter are all working okay now too.
The patch is at:
Neil Schemenauer
>[Neil Schemenauer]
>> I have added the Boehm-Demers garbage collector to Python and it
>> works well!
>> ...
>
>That's good, but I'm not clear on what this means. The patch didn't seem to
>do more than replace malloc/calloc/realloc/free with the BDW versions; in
>particular, Py_USE_GC_FREE was #define'd, and you ran tests without
>significant cyclic structures, so it doesn't *appear* that anything more
>happened here than that a different implementation of the malloc family got
>plugged in. Did the "collection" phase of BDW find any trash at all?
It doesn't unless you create some. The patch creates a version
of Python that uses GC only to clean up reference cycles.
>... it would be great if BDW could be invoked
>"just sometimes" to reclaim the cycles RC can't catch.
That is what is happening now.
>semi-encouraging-ly y'rs - tim
Thanks.
Neil
BTW, _tkinter is not working perfectly yet. I think lambdas
passed as callbacks are getting collected.
Tim Peters
> ... Did the "collection" phase of BDW find any trash at all?
[Neil Schemenauer]
> It doesn't unless you create some. The patch creates a version
> of Python that uses GC only to clean up reference cycles.
Just as it appeared <wink>. So it's really no surprise that the overall
timing didn't change much in pybench, right?
> ...
> BTW, _tkinter is not working perfectly yet. I think lambdas
> passed as callbacks are getting collected.
Such as to key bindings? Harmonic convergence (I was just looking at that
in IDLE tonight!).
When you pass a lambda (or bound method object, or any callable object -- CO
for short) as a binding, Tkinter.py eventually wraps the CO in an instance
of Tkinter.CallWrapper, and the .__call__() method of that instance is
passed on to _tkinter.c's Tkapp_CreateCommand via Tkinter.Misc._register().
The interesting thing here is that nothing in Tkinter.py holds on to
anything from which the CO can be reached. Instead _tkinter.c's
Tkapp_CreateCommand bumps the CO's refcount, stuffs a pointer to the CO in a
heap-allocated new instance of a PythonCmd_ClientData struct, and passes a
pointer to the latter on to Tcl via Tcl_CreateCommand. _tkinter.c doesn't
hold on to a reference to this either: when Tkapp_CreateCommand returns,
Tcl the sole owner of the CO reference that was added, and Tcl is the only
one who knows the address of the PythonCmd_ClientData struct.
So if the CO is an embedded lambda expression, there is only one reference
to it in total, and only Tcl can reach it (which it does when the binding
triggers, and then it passes a pointer to the PythonCmd_ClientData struct
back to _tkinter.c's PythonCmd). So if BDW isn't crawling over Tcl/Tk's
memory too, the lambda will look like trash, and the PythonCmd_ClientData
struct will look like trash regardless of what flavor of CO was passed.
So BDW has to get at Tcl's memory too, or _tkinter.c has to maintain a list
of allocated PythonCmd_ClientData thingies (until PythonCmdDelete is called
back from Tcl), or ...
It's convoluted for sure, but I do expect you'll find "the answer" at the
bottom of it.
rc-is-so-simple<wink>-ly y'rs - tim
Neil Schemenauer
>Just as it appeared <wink>. So it's really no surprise that the overall
>timing didn't change much in pybench, right?
I'm surprised. The collection of reference cycles is coming for
free. I expected there to be some speed tradeoff. So far the
only problem I see is sorting out the mallocs (ie. extension
modules may use their own malloc). This should be done anyhow in
order for alternate Python malloc implemetations to be used. The
other problem is objects that expect not to be collected even
though Python has no references to them (Tkinter callbacks). I
don't think this problem is unsolvable. Things are much better
than I expected.
With some tuning maybe the gc version of Python will perform
better than the regular version.
[Tim on Tkinter callbacks being collected]
>It's convoluted for sure, but I do expect you'll find "the answer" at the
>bottom of it.
>
>rc-is-so-simple<wink>-ly y'rs - tim
This is really nasty. I am trying to allocate this data with
regular malloc. If I use malloc and free then this data should
not get collected and the GC should realize that the Python
functions still have references to them. So far I am not having
much luck with this approach. It would help if I totally
understood what was happening here. I have more work to do.
Neil
Christian Tismer
Neil Schemenauer wrote:
...
> With some tuning maybe the gc version of Python will perform
> better than the regular version.
I believe you will need to keep the RC as well, since at
the moment, I think this gives more locality than GC,
as Tim mentioned already.
Early this year, I tried to speed up the VM, and one aspect
I tried was to delay DECREFS, doing them from time to
time in bulks. Although I could measure the extra overhead
away, Python became a little slower. It appears as a rule of
thumb to me:
RC is fine. Try to get rid of objects ASAP.
If you can manage to make it rock solid, I would perhaps
like to bundle it with stackless Python. Although, the
collector which Guido proposed a while ago could perform
even better, so I'm still a bit undecided.
ciao - chris
--
Christian Tismer :^) <mailto:tis...@appliedbiometrics.com>
Applied Biometrics GmbH : Have a break! Take a ride on Python's
Kaiserin-Augusta-Allee 101 : *Starship* http://starship.python.net
10553 Berlin : PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF
we're tired of banana software - shipped green, ripens at home
Tim Peters
> ...
> The collection of reference cycles is coming for free. I expected there
> to be some speed tradeoff.
pybench and pystone have small working sets (they're primarily CPU-speed
tests); so there's very little reachable for BDW to trace, so BDW doesn't
have much to do, so runs fast when it's invoked; and with refcounting on,
most pystone/pybench needs are handled by internal free lists that don't
invoke malloc at all after initial setup, so BDW doesn't even get called
often. IOW, BDW likely isn't doing much of anything in these tests besides
some initial mallocs.
It's curious that list operations appeared to take a major hit in pybench.
This may be related to "[GC_realloc] is very likely to allocate a new
object, unless MERGE_SIZES is defined in gc_priv.h" from the BDW README.
Python uses realloc over & over when growing lists.
> So far the only problem I see is sorting out the mallocs (ie. extension
> modules may use their own malloc). This should be done anyhow in order
for
> alternate Python malloc implemetations to be used.
I don't think that's going to be happen: Python doesn't ask anyone to
change anything about how they like to handle their memory today, and Guido
has pronounced that a Major Feature on multiple occasions. Vladimir
Marangozov's PyMalloc (see his Starship page) caters to that, letting Python
use its own malloc without anyone else getting involved.
> The other problem is objects that expect not to be collected even
> though Python has no references to them (Tkinter callbacks). I
> don't think this problem is unsolvable. Things are much better
> than I expected.
On one particular platform, and that platform's version of threads (or are
you running without threads?), yes -- looks great so far <wink>.
> With some tuning maybe the gc version of Python will perform
> better than the regular version.
With or without refcounting? As before, I think refcounting's contribution
to Python's current performance is vastly under-appreciated.
[about Tkinter callbacks]
> This is really nasty. I am trying to allocate this data with
> regular malloc. If I use malloc and free then this data should
> not get collected and the GC should realize that the Python
> functions still have references to them.
This won't work. A pointer *must* be reachable from *Python's* "root set",
else BDW won't know it exists. From the BDW README again:
Note that pointers inside memory allocated by the standard "malloc"
are not seen by the garbage collector. Thus objects pointed to only
from such a region may be prematurely deallocated. It is thus
suggested that the standard "malloc" be used only for memory regions,
such as I/O buffers, that are guaranteed not to contain pointers to
garbage collectable memory.
> So far I am not having much luck with this approach.
That's only because it's a hopeless dead end <wink>. As I suggested before,
... or _tkinter.c has to maintain a list of allocated
PythonCmd_ClientData thingies (until PythonCmdDelete is called
back from Tcl), ...
IOW, this is certainly solvable, but has to be approached in a way that will
work <wink>. Let BDW do the malloc for these (else the contained CO pointer
will be invisible to it). The ClientData struct has to remain reachable
from _tkinter.c, though, until Tcl tells PythonCmdDelete it's got no more
use for it.
Another possibility is to use "standard" malloc/free for ClientData structs,
but arrange for Tkinter.py to hold on to a reference to the callable object
(so BDW can find it from Python's root set). It already holds on to the
*name* of the synthesized Tcl cmd, in a list bound to data attr
_tclCommands. Change the list to a dict mapping the name to its associated
CO, and fiddle all uses of _tclCommands accordingly. That's probably the
fastest way out of this jam (&, indeed, should also run faster than the
list.remove() approach used today).
btw-keep-in-mind-that-software-doesn't-work<wink>-ly y'rs - tim
Neil Schemenauer
> Neil Schemenauer wrote:
> ...
> > With some tuning maybe the gc version of Python will perform
> > better than the regular version.
>
> the moment, I think this gives more locality than GC,
> as Tim mentioned already.
It is my intention to leave the RCing alone. My previous
ambition to remove it was misguided. By tuning I meant doing
things like reducing how often the collector gets called. Since
RC should take care of almost everything the collector should
very infrequently. Then again, premature optimization is the
root of all programming evil. I will concentrate on making
things work right now.
> If you can manage to make it rock solid, I would perhaps
> like to bundle it with stackless Python. Although, the
> collector which Guido proposed a while ago could perform
> even better, so I'm still a bit undecided.
I don't know about rock solid but it is running pretty well for
me now. I fixed the problem with the tkinter callbacks and I can
now run Idle and Grail with no problems. All regression tests
pass. I will try to beat on it for a few days and see what what
breaks. I haven't fixed all the extension modules yet. There is
trouble if PyMem_* and malloc and friends are mixed. I suspect
that many extension modules do this (including some of mine). So
far I had to fix readline, regex, re, and some of the Python
internals. I have sent Guido a patch.
Are you familiar with the (theoretically) copying garbage
collector for Python? I can't find the name of the guy who wrote
it. I think it would be interesting to integrate this collector
with your stackless Python. That would remove the need for
keeping the separate GC stack (I think). pybench shows Python
with this collector to be 68% slower than Python 1.5.1. It
should be faster without the GC stack but it will probably still
be slower than regular Python.
I have a new version of the patch available at:
http://www.acs.ucalgary.ca/~nascheme/python/gc.html
I will probably compile some Windows binaries (for people on
compiler challanged systems). Hopefully people can try it out
and give me some feedback.
Neil
Neil Schemenauer
>pybench and pystone have small working sets (they're primarily CPU-speed
>tests); so there's very little reachable for BDW to trace, so BDW doesn't
>have much to do, so runs fast when it's invoked; and with refcounting on,
>most pystone/pybench needs are handled by internal free lists that don't
>invoke malloc at all after initial setup, so BDW doesn't even get called
>often. IOW, BDW likely isn't doing much of anything in these tests besides
>some initial mallocs.
What could I use as a better benchmark? Real applications would
be best I suppose.
>I don't think that's going to be happen: Python doesn't ask anyone to
>change anything about how they like to handle their memory today, and Guido
>has pronounced that a Major Feature on multiple occasions. Vladimir
>Marangozov's PyMalloc (see his Starship page) caters to that, letting Python
>use its own malloc without anyone else getting involved.
Perhaps I misunderstand, but I thought that the PyMem_* functions
were created for this purpose. Extension modules can use any
malloc they like but they can't mix malloc/free/realloc with
PyMem_*.
>On one particular platform, and that platform's version of threads (or are
>you running without threads?), yes -- looks great so far <wink>.
Yes, one platform, no threads. Thanks a lot Tim. Here I was
feeling pretty good about myself. Now I'm depressed.
>> With some tuning maybe the gc version of Python will perform
>> better than the regular version.
>
>With or without refcounting? As before, I think refcounting's contribution
>to Python's current performance is vastly under-appreciated.
With.
>[about Tkinter callbacks]
>> This is really nasty. I am trying to allocate this data with
>> regular malloc. If I use malloc and free then this data should
>> not get collected and the GC should realize that the Python
>> functions still have references to them.
>
>This won't work. A pointer *must* be reachable from *Python's* "root set",
>else BDW won't know it exists.
<homer>Mmmm, documentation</homer>. I think I have the problem
licked now. I use GC_malloc_uncollectable for these things.
>btw-keep-in-mind-that-software-doesn't-work<wink>-ly y'rs - tim
No kidding. I tried to compile this stuff under Windows. What a
nightmare. I'm going to have to take some time off and lick my
wounds. :)
Neil
Tim Peters
refcounting left on]
[Neil Schemenauer]
> What could I use as a better benchmark? Real applications would
> be best I suppose.
This is a real bind! People advocating changes in core areas are always
challenged to prove they wouldn't slow things down, but there's no clear way
to do that. Python is used by so many people for so many things on so many
platforms now, a credible study would be a huge undertaking.
So screw it <wink>. Find someone with an important program that's leaking
memory and see whether this fixes *that*. This (not speed) is BDW's real
strength anyway. Chalk up a few victories there, and selling will be much
easier.
A cautionary tale: Over the past month I've been poking away at tracking
leaks in IDLE. After building a capable-enough cycle-finding tool (a
feature-bloated but much faster offshoot of Lars's Plumbo.py), I found
cycles all over the place. As things turned out, though, hardly any of them
could be considered trash! The way functions point to their module globals,
which in turn point to imported modules, some of which eventually point into
sys, from whose sys.modules everthing can be reached ... an isolated cycle
was very rare indeed.
It came as a real surprise to me that "real GC" wouldn't have helped. This
may be a particularly nasty problem for GUI programs, so I'm not too quick
to generalize. OTOH, don't be too quick to discount it either <wink>.
>> Python doesn't ask anyone to change anything about how they like to
>> handle their memory today, and Guido has pronounced that a Major Feature
>> on multiple occasions. Vladimir Marangozov's PyMalloc (see his Starship
>> page) caters to that, letting Python use its own malloc without anyone
>> else getting involved.
> Perhaps I misunderstand, but I thought that the PyMem_* functions
> were created for this purpose. Extension modules can use any
> malloc they like but they can't mix malloc/free/realloc with
> PyMem_*.
No, I misunderstood you. But now that I don't, it's not worth going back to
explain how I did <wink>.
> ...
> Yes, one platform, no threads. Thanks a lot Tim. Here I was
> feeling pretty good about myself. Now I'm depressed.
Excellent -- if you can persist beyond that, this may yet turn into
something useful instead of Yet Another 48-Hour Abandoned Grand Scheme <0.9
wink>. Reading the BDW docs and comments closely should give you another
sharp jolt of helpful depression: this is > a megabyte of code spread over
> 100 files, fighting a gazillion platform peculiarities: there's nothing
easy about it.
OTOH, they've made oodles of progress over the years. and looks like it's
usable on most major platforms now. Keep pitching it as a compile-time
config option and I think it's got a chance.
[about Tkinter callbacks]
> <homer>Mmmm, documentation</homer>. I think I have the problem
> licked now. I use GC_malloc_uncollectable for these things [ClientData
> structs].
It doesn't matter whether that appeared to work, it only matters whether I
*think* that should work <wink>. Yup! That should work. Nice job, Neil --
that's got to be the minimal hack around this one.
>> btw-keep-in-mind-that-software-doesn't-work<wink>-ly y'rs - tim
> No kidding. I tried to compile this stuff under Windows. What a
> nightmare. I'm going to have to take some time off and lick my
> wounds. :)
If it's any consolation, last time I looked hard at BDW was about 6 years
ago, when I gave up after about 40 hours of trying to port it to the KSR
architecture. This was after the guy in the next office gave up after about
20 hours of trying to stop it dumping core on a SPARC/SunOS box. It's come
a long way since then!
or-we-could-view-it-as-5x-as-many-lines-to-go-wrong<wink>-ly y'rs - tim
Robin Becker
<tim...@email.msn.com> writes
>[Tim, speculates on why pystone/pybench aren't much affected by BDW w/
> refcounting left on]
>
>So screw it <wink>. Find someone with an important program that's leaking
>memory and see whether this fixes *that*. This (not speed) is BDW's real
>strength anyway. Chalk up a few victories there, and selling will be much
>easier.
>
>A cautionary tale: Over the past month I've been poking away at tracking
>leaks in IDLE. After building a capable-enough cycle-finding tool (a
>feature-bloated but much faster offshoot of Lars's Plumbo.py), I found
>cycles all over the place. As things turned out, though, hardly any of them
>could be considered trash! The way functions point to their module globals,
>which in turn point to imported modules, some of which eventually point into
>sys, from whose sys.modules everthing can be reached ... an isolated cycle
>was very rare indeed.
>
>It came as a real surprise to me that "real GC" wouldn't have helped. This
>may be a particularly nasty problem for GUI programs, so I'm not too quick
>to generalize. OTOH, don't be too quick to discount it either <wink>.
>
very interesting; I'm spending a bit of time on Zope at the moment. This
if anything will be a long running thing which shouldn't leak memory.
Any chance of an analysis? Or the (I'm sure all the features are good)
version of plumbo?
--
features-to-one-are-bugs-to-others-ly yrs
Robin Becker
Tim Peters
[Robin Becker]
> very interesting; I'm spending a bit of time on Zope at the moment. This
> if anything will be a long running thing which shouldn't leak memory.
> Any chance of an analysis?
Not from me -- I don't even know how to spell Xoap.
> Or the (I'm sure all the features are good) version of plumbo?
It's Cyclops.py, and I just uploaded it to python.org's ftp contrib site.
It should show up in the System directory later this week (send a complaint
every 5 minutes to ftpm...@python.org until it does <wink>).
that-barry-gets-too-much-sleep-ly y'rs - tim
Robin Becker
<tim...@email.msn.com> writes
>
...
>Not from me -- I don't even know how to spell Xoap.
>
>> Or the (I'm sure all the features are good) version of plumbo?
>
>It's Cyclops.py, and I just uploaded it to python.org's ftp contrib site.
>It should show up in the System directory later this week (send a complaint
>every 5 minutes to ftpm...@python.org until it does <wink>).
thanks
--
Ulysses-ly yrs Robin Becker
Barry A. Warsaw
>>>>> "TP" == Tim Peters <tim...@email.msn.com> writes:
TP> It's Cyclops.py, and I just uploaded it to python.org's ftp
TP> contrib site. It should show up in the System directory later
TP> this week (send a complaint every 5 minutes to
TP> ftpm...@python.org until it does <wink>).
TP> that-barry-gets-too-much-sleep-ly y'rs - tim
Tim was already
Pushing it with the bogus
Haiku that he sent
-Barry
Ovidiu Predescu
> I have added the Boehm-Demers garbage collector to Python and it
> works well!
Great work! However, this is not enough, there are more things to do
;-/!
More than a year ago I've added support for the Boehm's collector to
Objective-C, an object oriented language based on C, and two libraries
written in Objective-C that use reference counting as a memory tracking
policy.
What I found is that I needed to introduce weak references in these
libraries, otherwise I would have objects alive without being used
anymore by the code. The reason were two places in these libraries where
objects were kept in a global table. Having references to objects from
this table prevented the objects from being collected, even if you drop
the references to the objects from the rest of your code. The
alternative is to require the programmer to remove the references from
these global tables when he is no longer interested in keeping the
objects alive, but this is error-prone and defeats the whole purpose of
automatic garbage collection.
I don't know exactly if there are any modules in Python that keep track
of objects in global variables but if this is the case you may need to
implement something like that.
One of the libraries I'm referring to is libFoundation and you can find
it at:
http://www.geocities.com/SiliconValley/Monitor/7464/libFoundation/
The interesting piece you may want to look at is GarbageCollector.m. It
defines a way to register observers for the finalization of other
objects. The registering is done programmatically, by interested objects
(like for example the NSNotificationCenter, which allows interested
objects to be informed when a particular event is posted by another
object).
The support for weak pointers in the Objective-C runtime was done using
the typed-memory facility of the Boehm's collector. Take a look at a
description of how it works on:
http://www.geocities.com/SiliconValley/Monitor/7464/Objective-C/
If you're interested in the sources of the runtime, get a recent
snapshot of EGCS and look in the libobjc directory.
Have fun,
--
Ovidiu Predescu <ovi...@cup.hp.com>
http://andromeda.cup.hp.com/ (inside HP's firewall only)
http://www.geocities.com/SiliconValley/Monitor/7464/
Markus Kohler
[deletia]
>
> My guess is that simple refcounting has excellent cache behavior in Python,
> and GC doesn't. Unlike as in almost any other language, the little loop
>
> for i in xrange(10000000):
> pass
>
> creates a huge amount of heap trash at a furious pace (even ints "are boxed"
> in Python). RC can reuse the same little bit of storage over & over (the
> trash is recycled as soon as it's created), while delayed GC chews up some
> large amount of address space over & over instead. You can test this by
> timing that loop alone (with and without refcounting).
True.
IMHO this is one of Pythons major design flaws. There is no need to create a new
object for each step of the loop.
Smalltalk has loops at least as flexible as Python and it doesn't have this problem.
Loops in squeak (www.squeak.org) are already 2 to 3 times faster than in Python.
>
> Note too that Python does no stack allocation of locals or frames or
> anything else: everything is heap-based, with the most speed-critical
> allocations augmented by assorted internal fast free lists. Disable the
> refcounting, and the finalizers on the "intense" objects don't get called,
> and then the fast free lists don't get used either. So the allocation speed
> optimizations Python implements get castrated too.
>
> I've always said it would take a very sophisticated mark-&-sweep to beat
> RC's performance for *Python* (it's not Lisp, or Smalltalk, or C++, or ...
> under the covers);
Is there anything that would prevent someone to reimplement loops such that only one
variable is allocated ?
> if true (& I haven't seen anything yet that changes my
> mind <wink>), you're at the first step of very long road if you want to make
> BDW the primary approach; but it would be great if BDW could be invoked
> "just sometimes" to reclaim the cycles RC can't catch.
>
> semi-encouraging-ly y'rs - tim
>
>
>
>
Markus
--
Markus Kohler mailto:markus...@hp.com
Michael Hudson
[snippety snip]
>
> Is there anything that would prevent someone to reimplement loops such
> that only one variable is allocated ?
>
Yes. The fact that ints are immutable. Consider:
for i in xrange(10):
if i == 2:
j = i
if the loop only allocated one variable, then at the end of the loop
'print j' would give the answer '9'!
I guess this could be worked around by inspecting refcounts and such...
but not easily.
there's-the-small-integer-cache-to-worry-about-too-ly y'rs
Michael
Christian Tismer
Michael Hudson wrote:
>
> On 20 Jul 1999, Markus Kohler wrote:
> [snippety snip]
> >
> > Is there anything that would prevent someone to reimplement loops such
> > that only one variable is allocated ?
> >
>
> Yes. The fact that ints are immutable. Consider:
>
> for i in xrange(10):
> if i == 2:
> j = i
>
> if the loop only allocated one variable, then at the end of the loop
> 'print j' would give the answer '9'!
>
> I guess this could be worked around by inspecting refcounts and such...
> but not easily.
FYI, stackless Python already saves the internal loop count into
a counter object. The explicit creation of "i" is still there,
of course. But you might know that inteegr allocation is completely
handled through a cache, and I see not much benefit coming from
making my mutable integer object public.
This look to cheap for me, to be replaced by the hair of extra
refcount tracking.
Tim Peters
> for i in xrange(10000000):
> pass
[Markus Kohler]
> True.
> IMHO this is one of Pythons major design flaws.
What, specifically?
> There is no need to create a new object for each step of the loop.
> Smalltalk has loops at least as flexible as Python and it doesn't
> have this problem.
Sorry, I've missed what "the problem" is thought to be here. Loop overhead
is typically lareger in Python programs than in other languages, but is also
typically trivial all the same.
> Loops in squeak (www.squeak.org) are already 2 to 3 times faster
> than in Python.
All possible loops? All loops you've personally timed? One loop you timed
in a moment of boredom <wink>? An empty loop that counts to 10000000? I
don't doubt that Squeak is faster in many areas and for many reasons, but
I've been running Python for most of the decade and reducing loop overhead
is about the last thing I'd bother to aim at.
> ...
> Is there anything that would prevent someone to reimplement loops
> such that only one variable is allocated ?
In the loop above, there are actually two int objects created per iteration:
the one explicitly created by xrange and bound to "i", and a hidden "loop
counter" int object used to drive the for/__getitem__ protocol. The latter
can be (& has been, experimentally) reimplemented via an internal "mutable
int" type created for that purpose, but it didn't buy enough to be worth
adopting. Most loop bodies contain more than "pass" <wink>, and the more
work they do the less the fixed overhead matters.
unboxing-*all*-ints-now-that-would-buy-something-ly y'rs - tim
Markus Kohler
> [Tim]
> > for i in xrange(10000000):
> > pass
> > [and RC vs GC implications]
>
> [Markus Kohler]
> > True.
> > IMHO this is one of Pythons major design flaws.
>
> What, specifically?
That flaw is that loops are not implemented using closures.
Instead as far as I understand loops in Python are implemented such that
the loop counter is created and destroyed for each step of the loop.
Even if you optimize this heavily you will probably slower than with a design
that "recycles" the loop variable.
You could compare this to tail recursive versus non-tail recursive functions.
Also closures make the implementation of efficient loops more difficult they
have the advantage that loops and other control structures can be better integrated
into the OO concept. Smalltalk for example doesn't need Iterators for that reason.
>
> > There is no need to create a new object for each step of the loop.
> > Smalltalk has loops at least as flexible as Python and it doesn't
> > have this problem.
>
> Sorry, I've missed what "the problem" is thought to be here. Loop overhead
> is typically lareger in Python programs than in other languages, but is also
> typically trivial all the same.
>
> > Loops in squeak (www.squeak.org) are already 2 to 3 times faster
> > than in Python.
>
> All possible loops? All loops you've personally timed? One loop you timed
> in a moment of boredom <wink>? An empty loop that counts to 10000000? I
> don't doubt that Squeak is faster in many areas and for many reasons, but
> I've been running Python for most of the decade and reducing loop overhead
> is about the last thing I'd bother to aim at.
Lately ee have had already some examples in this group where loop overhead was
a major factor.
I measured different kind of loops and even the more general while loop was faster
in squeak than anything else in python.
Functions calls seems also to be much slower in Python ...
>
> > ...
> > Is there anything that would prevent someone to reimplement loops
> > such that only one variable is allocated ?
>
> In the loop above, there are actually two int objects created per iteration:
> the one explicitly created by xrange and bound to "i", and a hidden "loop
> counter" int object used to drive the for/__getitem__ protocol. The latter
> can be (& has been, experimentally) reimplemented via an internal "mutable
> int" type created for that purpose, but it didn't buy enough to be worth
> adopting. Most loop bodies contain more than "pass" <wink>, and the more
> work they do the less the fixed overhead matters.
Another problem with loops could be reference counting.
I'm wondering how often a reference count get's changed when running a loop ...
Michael Hudson
> "Tim Peters" <tim...@email.msn.com> writes:
>
> > [Tim]
> > > for i in xrange(10000000):
> > > pass
> > > [and RC vs GC implications]
> >
> > [Markus Kohler]
> > > True.
> > > IMHO this is one of Pythons major design flaws.
> >
> > What, specifically?
>
> That flaw is that loops are not implemented using closures.
Sorry, how would that help?
> Instead as far as I understand loops in Python are implemented such that
> the loop counter is created and destroyed for each step of the loop.
>
> You could compare this to tail recursive versus non-tail recursive functions.
>
> Also closures make the implementation of efficient loops more
> difficult they have the advantage that loops and other control
> structures can be better integrated into the OO concept. Smalltalk for
> example doesn't need Iterators for that reason.
Python is not a functional programming language. As I understand it, you
can't really have convenient closures because it is not lexically scoped
(I can give you inconvenient closures...) and there were lengthy
discussions about that some months back. I didn't follow it too closely
because I didn't know what lexically scoped meant then, but the upshot was
(I think) that lexical scoping isn't really the "Python way".
Python isn't really an OO language either, in the sense of something like
Smalltalk or Eiffel.
I have heard it said that Python is a language that "gets it's compromises
exactly right" and that about sums it up for me.
> Lately ee have had already some examples in this group where loop
> overhead was a major factor.
Really? When? Sorry, I don't want to sound disbelieving, but I don't
remember that.
> I measured different kind of loops and
> even the more general while loop was faster in squeak than anything
> else in python.
Python, as I said, is neither functional nor purely OO; it's not
particularly fast either. Fast enough, generally.
> Functions calls seems also to be much slower in Python ...
*That* I believe. Not clear what to do about it though.
> Another problem with loops could be reference counting. I'm wondering
> how often a reference count get's changed when running a loop ...
Not vastly more often than in regular code, I'd guess. It's true that
executing
1+1
spends longer twiddling refcounts than doing the work, but that's Python.
I can't decide which Peters-style ending I want here, so you're all
getting two:
not-OO-functional-or-fast-but-still-great-ly y'rs
it's-not-OO-or-functional-it's-Python-ly y'rs
Michael
Tim Peters
> IMHO this is one of Pythons major design flaws.
[Tim]
>> What, specifically?
[Markus]
> That flaw is that loops are not implemented using closures.
Yet you later note too that functions in Python are unusually slow. It's no
flaw for a language implementation to decline to build everything on its
least optimized constructs <wink>.
> Instead as far as I understand oops in Python are implemented such that
> the loop counter is created and destroyed for each step of the loop.
Yes, "for" loops work this way in Python.
> Even if you optimize this heavily you will probably slower than
> with a design that "recycles" the loop variable.
The for/__getitem__ protocol requires passing a full-blown int object, and
int objects are immutable. Trying to trick this is more trouble than it's
worth.
> You could compare this to tail recursive versus non-tail recursive
> functions.
Trust me on this one: you don't want to get anywhere near this comparison
unless you want to see Guido running in the opposite direction <wink>.
Seriously, I agree it's a good comparison; Python was designed to favor
iteration over recursion, though, and for that reason too it's not like the
languages you appear to like better.
> Also closures make the implementation of efficient loops more
> difficult
Yet another killer reason to do it that way <wink>.
> they have the advantage that loops and other control structures can be
> better integrated into the OO concept. Smalltalk for example doesn't need
> Iterators for that reason.
I happen to like the CLU/Sather/Icon approach toward iterators
(semi-coroutines)better than Python's or Smalltalk's (or C++'s, or Perl's,
or ...); Python doesn't work that way under the covers either, but is
surprisingly *close* to being able to work that way. So that way makes more
sense to me for Python; Python's implementation and its surface semantics
are very close in most areas.
>> Loop overhead is typically lareger in Python programs than in other
>> languages, but is also typically trivial all the same.
> Lately ee have had already some examples in this group where
> loop overhead was a major factor.
? I don't recall that. Last one I do recall, someone posted a pretty much
certifiably insane quadruply-nested numeric loop, recomputing range again at
each "for" level. Several replies suggested floating the range calls out of
the loops to save time. Christian Tismer quite properly pointed out that
this was pointless, since the loop overhead accounted for a tiny percent of
the runtime. Someone else suggested an algorithmic change that chopped
orders of magnitude off the runtime. An extreme outcome, but typical in
outline.
> I measured different kind of loops and even the more general
> while loop was faster in squeak than anything else in python.
Empty loops? Or did they do something? If empty, it's hard to care. If
they did something, why do you think the difference is accounted for by the
loop mechanism rather than by the guts of what the loops did? A specific
example would clarify matters.
> Functions calls seems also to be much slower in Python ...
Yes. Even worse <wink>, when a (say) Scheme person comes to Python, one of
the surest ways to speed up their first programs is to encourage them to
rewrite their lambda-slinging code as explicit inline loops. That is
plus1 = map(lambda x: x+1, somelist)
runs much slower than
plus1 = []
for x in somelist:
plus1.append(x+1)
> ...
> Another problem with loops could be reference counting.
> I'm wondering how often a reference count get's changed when
> running a loop ...
It really depends on the specifics of the loop body. In general, each
instance of a name in a line of code causes at least one refcount operation
each time the line is executed. There *is* a lot of refcount fiddling going
on, and nothing is ever exempted from it. It's also the case that nothing
in Python is stack-allocated; every object may have indefinite extent,
there's no way to tell at compile-time, and so everything is heap-allocated
"just in case". If you're looking for reasons why Python is slower than
many other languages, they're not hard to find <wink>.
darned-hard-to-change-though!-ly y'rs - tim
Tim Peters
> ...
> Python is not a functional programming language.
NOW they tell me <wink>.
> As I understand it, you can't really have convenient closures because it
> is not lexically scoped (I can give you inconvenient closures...) and
> there were lengthy discussions about that some months back. I didn't
> follow it too closely because I didn't know what lexically scoped meant
> then, but the upshot was (I think) that lexical scoping isn't really
> the "Python way".
It wasn't, but it may be. Python's 3-level scoping system (which Guido
routinely calls a 2-level system these days -- I guess because he thinks 2
sounds less contrived than 3 <wink>) was a thoroughly deliberate decision.
Today he'll tell you that he may have overreacted against the rampant abuse
of deep procedure nesting common in large Pascal programs of that long-ago
time. I tangled with him on this in email at the time, but having also
suffered the Pascal abuses, reluctantly agreed to stop harassing him about
it and just see how the 3-level scheme worked out in practice.
I think it worked out great! I was surprised. But there were two miserable
problems:
1) Despite its simplicity, the implementation of name resolution was much
slower than in most languages that support arbitrarily deep lexical nesting.
By far the worst of this got repaired by assigning locals to fixed slots
(and bytecodehacks wouldn't be nearly as much fun without that to abuse
<wink>).
2) Despite that scoping is 3-level, the *syntax* allows nesting functions &
classes to any level. This is an eternal source of confusion for people
coming from other languages, and can take a long time to stop tripping over.
So why did the syntax allow it? Because it would have complicated the
grammar to stop it! Guido may deny that today, but it's the truth <wink>.
Times are different now. The only people *likely* to abuse lexical nesting
these days are Scheme/Lisp heads, but Python's lack of a macro system will
drive them away anyway <wink>.
As far as the implementation goes, arbitrarily deep lexical closures with
indefinite extent are easy in Python -- it's a small & simple patch
(frames-- activation records --are heap-allocated anyway; all that's really
missing is a "static link" to crawl up the lexical nest). Guido even
implemented (but never released) it a few years ago. An unexpected
technical glitch killed it: it turned out that the natural implementation
created cyclic implementation structures that refcounting isn't strong
enough to clean up. Presumably Python2 could repair that by hook or by
crook.
So Schemish lexical scoping may yet become The Python Way! I wouldn't count
on it, though; and maintaining state via class instances will always be more
Pythonic.
[prematurely wise comments elided]
> not-OO-functional-or-fast-but-still-great-ly y'rs
> it's-not-OO-or-functional-it's-Python-ly y'rs
it's-guido's-way-or-the-highway-ly y'rs - tim
Markus Kohler
> On 21 Jul 1999, Markus Kohler wrote:
> > "Tim Peters" <tim...@email.msn.com> writes:
> >
> > > [Tim]
> > > > for i in xrange(10000000):
> > > > pass
> > > > [and RC vs GC implications]
> > >
> > > [Markus Kohler]
> > > > True.
> > > > IMHO this is one of Pythons major design flaws.
> > >
> > > What, specifically?
> >
> > That flaw is that loops are not implemented using closures.
>
> Sorry, how would that help?
It wouldn't help regarding performance, but increase the
expressiveness of the language.
>
> > Instead as far as I understand loops in Python are implemented such that
> > the loop counter is created and destroyed for each step of the loop.
> >
> > You could compare this to tail recursive versus non-tail recursive functions.
> >
> > Also closures make the implementation of efficient loops more
> > difficult they have the advantage that loops and other control
> > structures can be better integrated into the OO concept. Smalltalk for
> > example doesn't need Iterators for that reason.
>
> Python is not a functional programming language.
It could be extended by some functional features.
All I want is the integration of loops into the OO concept of the language.
Loops are cannot be used as any other polymorph
function if you don't implement them with closures (please tell me
if there's another way).
Smalltalk Example
iterate over a container and do something for each element looks like this:
aContainer do:[:each | each doSomething]
because do: is just a method like any other method you can redefine it for each
class.
This is very usefull because it allows to for example to loop over a Container
that contains two other Containers ("a" and "b" of an unknown type very easily:
do: aBlock
a do: aBlock.
b do: aBlick.
> As I understand it, you
> can't really have convenient closures because it is not lexically scoped
> (I can give you inconvenient closures...) and there were lengthy
> discussions about that some months back. I didn't follow it too closely
> because I didn't know what lexically scoped meant then, but the upshot was
> (I think) that lexical scoping isn't really the "Python way".
>
> Python isn't really an OO language either, in the sense of something like
> Smalltalk or Eiffel.
IMHO it's very close to Smalltalk, also Guido probably wouldn't agree with me ...
And It has all essential OO features.
The features that make it very similiar to Smalltalk are :
byte code interpreter,
all data are objects (is that really true ?),
crossplatform,
reflection api,
dynamically type checked,
In fact I cannot remember any successfull language that is closer to
Smalltalk than Python is.
>
> I have heard it said that Python is a language that "gets it's compromises
> exactly right" and that about sums it up for me.
I also like some things about Python. For example the module system, and how
easily external C functions can be integrated.
>
> > Lately ee have had already some examples in this group where loop
> > overhead was a major factor.
>
> Really? When? Sorry, I don't want to sound disbelieving, but I don't
> remember that.
It was about reading a file and put the words into some hash table.
As far as I can remember to loop overhead was a dominating factor.
And Python was much slower than perl for example.
>
> > I measured different kind of loops and
> > even the more general while loop was faster in squeak than anything
> > else in python.
>
> Python, as I said, is neither functional nor purely OO;
Ok it's not purely OO because you can define global functions.
But it's anything else is pretty much OO.
> it's not
> particularly fast either. Fast enough, generally.
>
> > Functions calls seems also to be much slower in Python ...
>
> *That* I believe. Not clear what to do about it though.
It's IMHO the overly complex function call mechanism, with it's
default values on all that other stuff. I really can imagine it's difficult to
make this fast. IMHO it's not worth the effort and maybe one should
invent Python lite that uses a simpler call mechanism :-O
>
> > Another problem with loops could be reference counting. I'm wondering
> > how often a reference count get's changed when running a loop ...
>
> Not vastly more often than in regular code, I'd guess. It's true that
> executing
>
> 1+1
>
> spends longer twiddling refcounts than doing the work, but that's Python.
>
> I can't decide which Peters-style ending I want here, so you're all
> getting two:
>
> not-OO-functional-or-fast-but-still-great-ly y'rs
> it's-not-OO-or-functional-it's-Python-ly y'rs
>
ovi...@cup.hp.com
wrote:
> IMHO it's very close to Smalltalk, also Guido probably wouldn't agree with me ...
> And It has all essential OO features.
>
>
> The features that make it very similiar to Smalltalk are :
> byte code interpreter,
> all data are objects (is that really true ?),
> crossplatform,
> reflection api,
> dynamically type checked,
One problem though with Python is the fact that it does make a distinction
between types and classes. Another is the garbage collection scheme that's
using.
> In fact I cannot remember any successfull language that is closer to
> Smalltalk than Python is.
Objective-C. Is a dynamically typed language and is compiled.
Greetings,
Greg Ewing
>
> I think it worked out great! I was surprised.
I wouldn't be quite so enthusiastic. I would say that
one can learn to live with the inconvenience, and
even almost forget that it's there most of the time.
But I still feel annoyed whenever I bump up against
it.
> and maintaining state via class instances will always be more
> Pythonic.
I don't think you can assert that until a Python
with lexical scoping has been in use for a while
and we see how people prefer to write things.
My bet is that a lot of things we write now like
b = Button(..., command = lambda self=self: self.blat(42))
would get written as
b = Button(..., command = lambda: self.blat(42))
and we would all wonder why we didn't storm
Guido's palace years ago and force him to release
his lexical scoping code at gunpoint...
Greg
Carel Fellinger
> Michael Hudson <mw...@cam.ac.uk> writes:
>> On 21 Jul 1999, Markus Kohler wrote:
>> > Lately ee have had already some examples in this group where loop
>> > overhead was a major factor.
>>
>> Really? When? Sorry, I don't want to sound disbelieving, but I don't
>> remember that.
> It was about reading a file and put the words into some hash table.
> As far as I can remember to loop overhead was a dominating factor.
> And Python was much slower than perl for example.
Well, if I remember correctly, eventually the perl version was slower
then the python optimized version:) The point was that reading bytes is
*much* slower then reading large chunks of data. So it had to do with
disk IO and not looping. Atleast that's what I remember from that thread.
(I checked with deja-news and low and behold I'm right! that is if we
refer to the same thread titled 'Speed of python' way back in feb:)
--
groetjes, carel
Markus Kohler
Sorry my Smalltalk example is wrong.
I must have coded for to long in this poor language called C ...
I must look like this :
do: aBlock
a do:[:each | aBlock value: each]
b do:[:each | aBlock value: each]
more complicated but still very easy to understand.
Greetings,
Tim Olson
>
> Sorry my Smalltalk example is wrong.
> I must have coded for to long in this poor language called C ...
> I must look like this :
>
> do: aBlock
> a do:[:each | aBlock value: each]
> b do:[:each | aBlock value: each]
>
> more complicated but still very easy to understand.
No, your original code was fine:
do: aBlock
a do: aBlock.
b do: aBlock.
No need to wrap the incoming block in yet another block.
You could also write it as:
do: aBlock
a , b do: aBlock.
which might be even clearer.
-- Tim Olson
Markus Kohler
Ah that's cool,
I didn't have the time to try it out ...
I saw code that used the #value: message
but, of course it has to be only used if you want to do addiotional
things for each element.
It seems to be really time to get back to Smalltalk for me ...
Tim Peters
[Markus Kohler]
> ...
> IMHO it's [Python] very close to Smalltalk, also Guido probably wouldn't
> agree with me ...
> And It has all essential OO features.
Python is something of an ink blot test that way: people coming from Scheme
think it's very close to that (and just needs macros and continuations and
lexical closures <wink>); people coming from pure functional languages have
a similar tale (and it just needs non-strict functions and a pile of
combinator notation); people coming from Perl -- well, let's stop before it
gets silly <wink>.
Python takes a lot of ideas from a lot of languages, but it has its own
worldview, and its own corresponding implementation. It really isn't "like"
anything else at heart, in philosophy or implementation.
I'm looking forward to the day when I drop in on another comp.lang.* group
and read "you know, this language is very close to Python, and all it really
needs is namespaces modeled as explicit dicts" <wink>.
>>> Lately we have had already some examples in this group where loop
>>> overhead was a major factor.
[Michael Hudson]
>> Really? When? Sorry, I don't want to sound disbelieving, but I don't
>> remember that.
[back to Markus]
> It was about reading a file and put the words into some hash table.
> As far as I can remember to loop overhead was a dominating factor.
> And Python was much slower than perl for example.
Ah, that one. That's not loops. It's input. Line-at-a-time input on my
platform is about 3 times faster in Perl than Python; indeed, it's faster in
Perl than in C! It's not loop mechanisms or even refcounting "to blame"
here (btw, Perl does refcounting too ...). It's that Perl breaks into the
stdio abstraction, going under the covers in platform + compiler specific
ways, peeking and poking C stdio FILE structs directly.
Very clever, but a lot of work. Most fgets implementations suck, and Perl
deserves all the credit for not settling for that. I doubt Python will do
it too, not as a matter of principle, but because nobody capable of it will
make time to do it.
would-rather-have-faster-functions-myself-but-if-wishes-were-bananas-
we'd-be-swimming-in-mushy-fruit-ly y'rs - tim