Real Problems with Python
ne...@cswcasa.com
to death of the -stupid- whitespace flamewar, since it's a) not a
problem, and b) not thoughtful.
So in effort to throttle that monster, here's a list of half-a-dozen
*real* problems with Python, along with what the current workarounds
are, and my subjective assessment of the prospects of them being
solved in the long term. Enjoy, argue, whatever.
1. Reference counting memory management
If you need to do anything with any sort of sophisticated cyclic
data structures, then reference-counting is deeply annoying. This
is not something that someone in a high-level language should have
to worry about!
In the long run, the solution is to use a conservative garbage
collection algorithm (such as the Boehm collector), and to use
reference counts to make sure that finalizers are called in
topological order. (Cyclic references with finalizers have no good
solution, period, so it's not worth worrying about them imo.)
Workarounds:
o Use JPython. It hijacks Java's garbage collection, so there are
no problems with cyclic data structures. It doesn't ever call
__del__ methods, though I don't think this is a problem in
practice.
o Use John Max Skaller's Viper
Viper is an interpreter written in OCaml, so like JPythn it
uses the host language's garbage collection. It's still an
alpha, though worth looking at if only to learn OCaml. :)
o Use Neil Schemenauer's gc patch.
Neil Schemenauer has added support for using the Boehm garbage
collector in Python. It uses reference counts in addition to the
gc for handling finalization, so things like file objects get
closes at the expected times, while still collecting cyclic
objects.
Prognosis:
Excellent. It looks quite likely that Python 3K will have
gc, and in the meantime Neil Schemenauer's patch is entirely
usable.
2. Lack of lexical scoping
Tim Peters disagrees, but I miss it a lot, even after using Python
for years. It makes writing callbacks harder, especially when
dealing with Tkinter and re.sub, os.path.walk, and basically every
time a higher-order function is the natural solution to a problem.
(Classes with __call__ methods are too cumbersome, and lambdas too
weak, when you need a small function closure that's basically an if
statement.)
There's another, more subtle problem -- much of the work that's
been done on optimizing and analyzing highly dynamic languages has
been from on the Scheme/ML/Lisp world, and almost of that work
assumes lexically-scoped environments. So using well-understood
optimization techniques is made harder by the difference.
Workarounds:
o Greg Ewing has a closure patch, that makes functions and
lambdas work without creating too much cyclic garbage.
Prognosis:
Pretty good. It looks like the consensus is gelling around adding
it eventually, if only to shut up complainers like me. :)
3. Multiple inheritance is unsound
By 'unsound' I mean that a method call to a method inherited by a
subclass of two other classes can fail, even if that method call
would work on an instance of the base class. So inheritance isn't a
type-safe operation in Python. This is the result of using
depth-first search for name resolution in MI. (Other languages,
like Dylan, do support sound versions of MI at the price of
more involved rules for name resolution.)
This makes formal analysis of program properties (for example,
value flow analyses for IDEs) somewhere between 'harder' to
'impossible'.
Workarounds:
o Don't use multiple inheritance in Python.
Prognosis:
I don't think there's any hope of this ever being fixed. It's just
too much a part of Python, and there isn't much pressure to fix
it. Just avoid using MI in most circumstances.
4. Lack of support for highly declarative styles of programming
It's often Python claimed that Python reads like pseudocode, but
very often when people are describing algorithms the natural
description is "find the solution that satisfies the following
conditions" rather than "nest for loops five deep".
If you've done any Prolog (or even SQL) programming you know how
powerful the constraint-solving mindset is. When a problem can be
represented as a set of constraints, then writing out the solution
manually feels very tedious, compared to telling the computer to
solve the problem for you. :)
Workarounds:
o Greg Ewing has implemented list comprehensions for Python. This
is a big increase in expressive power, since it starts to permit
the use of a constraint solving-style.
o Christian Tismer's Stackless Python patch enables the use of
first-class continuations in regular Python code. This lets
people easily and in pure Python create and experiment with all
sorts of funky control structures, like coroutines, generators,
and nondeterministic evaluation.
Prognosis:
Pretty good, if Stackless Python makes it in, along with list
comprehensions. (Personally, I suspect that this is the single most
important improvement possible to Python, since it opens up a
whole new category of expressiveness to the language.)
5. The type/class dichotomy, and the lack of a metaobject system.
C types and Python classes are both types, but they don't really
understand each other. C types are (kind of) primitive objects, in
that they can't be subclassed, and this is of course frustrating
for the usual over-discussed reasons. Also, if everything is an
object, then subclassing Class becomes possible, which is basically
all you need for a fully functional metaobject protocol. This
allows for some extremely neat things.
Workarounds:
o Use Jim Fulton's ExtensionClasses to write C extensions that
can be subclassed in Python code.
o Read GvR's paper "Metaclasses in Python 1.5" to write metaclasses
using the Don Beaudry hook. It's not as nice as just subclassing
Class, though.
Prognosis:
Good. The current workarounds are messy, but workable, and
long-term there's a real determination to solve the problem once
and for all.
6. The iteration protocol
The iteration protocol is kind of hacky. This is partly a function
of the interface, and partly due to the way the type-class
dichotomy prevents arranging the collection classes into a nice
hierarchy.
The design of the iteration protocol also makes looping over
recursive data structures (like trees) either slow, if done in the
obvious way and comprehensible way, or clumsy and weird-looking, if
you try to define iterator objects.
An example: Try writing a linked list class that you can iterate
over using a for loop. Then try to write a tree class that you
can iterate over in preorder, inorder, and postorderdirection.
Then make them efficient, taking no more than O(n) time and O(1)
space to iterate over all elements. Then try to reuse duplicated
code. Is the result beautiful?
Workarounds:
o Implement iterator objects and just eat the ugliness of
implementation for clean interface.
o Don't use trees or graphs, or at least don't expect to use
them with the standard loop constructs. :)
o Wait for Guido to invent a better way: Tim Peters has said on the
newsgroup that GvR is working on a better design,
Prognosis:
Good. There aren't any good short-term fixes, but the long term
outlook is probably fine. We aren't likely to ST blocks, though.
Neel
Fredrik Lundh
summary of some critical python warts]
just one footnote:
> 5. The type/class dichotomy, and the lack of a metaobject system.
>
> C types and Python classes are both types, but they don't really
> understand each other. C types are (kind of) primitive objects, in
> that they can't be subclassed, and this is of course frustrating
> for the usual over-discussed reasons. Also, if everything is an
> object, then subclassing Class becomes possible, which is basically
> all you need for a fully functional metaobject protocol. This
> allows for some extremely neat things.
note that you *can* implement real classes using relatively
clean C code. however, types are usually a better alternative,
since they provide low-level slots for most "magic" methods.
(after all, if performance isn't that critical, you might
as well stick to Python ;-)
</F>
Gerrit Holl
>
> 1. Reference counting memory management
> 2. Lack of lexical scoping
> 3. Multiple inheritance is unsound
> 4. Lack of support for highly declarative styles of programming
> 5. The type/class dichotomy, and the lack of a metaobject system.
> 6. The iteration protocol
...
7. Error messages spanning multiple LOC
$ python sample.py
Traceback (innermost last):
File "sample.py", line 3, in ?
print "this" + \
NameError: oops
$ cat sample.py
#!/usr/bin/env
print "this" + \
"that" + \
oops
It should say:
Traceback (innermost last):
File "sample.py", line 3, in ?
print "this" + \
"that" + \
oops
NameError: oops
regards,
Gerrit.
--
Homepage: http://www.nl.linux.org/~gerrit
-----BEGIN GEEK CODE BLOCK----- http://www.geekcode.com
Version: 3.12
GCS dpu s-:-- a14 C++++>$ UL++ P--- L+++ E--- W++ N o? K? w--- !O
!M !V PS+ PE? Y? PGP-- t- 5? X? R- tv- b+(++) DI D+ G++ !e !r !y
-----END GEEK CODE BLOCK----- moc.edockeeg.www//:ptth
Justin Sheehy
> 7. Error messages spanning multiple LOC
> $ python sample.py
> Traceback (innermost last):
> File "sample.py", line 3, in ?
> print "this" + \
> NameError: oops
I don't see that this is all that big a deal, let alone a Real Problem.
Slightly suboptimal behavior perhaps, but certainly not on the scale
of the issues raised in the original post, such as lack of real gc,
scoping issues, the type/class dichotomy, etc.
-Justin
Aahz Maruch
To emphasize Justin's point, this is a bug, not a design issue.
--
--- Aahz (Copyright 2000 by aa...@netcom.com)
Androgynous poly kinky vanilla queer het <*> http://www.rahul.net/aahz/
Hugs and backrubs -- I break Rule 6
Nostalgia just ain't what it used to be
Neil Schemenauer
>Python isn't perfect, and I enjoy thoughtful criticism. But I am sick
>to death of the -stupid- whitespace flamewar, since it's a) not a
>problem, and b) not thoughtful.
I agree. Thank you for injecting some intelligence into this
newgroup.
>1. Reference counting memory management
I have a new patch in the works. Just to wet your appetite:
* It is portable. It should run anywhere Python runs.
* It does not require rewriting or recompiling existing
extension modules.
* It has very low overhead.
The bad news is that:
* It only finds cycles that involve lists, tuples,
dictionaries, instances and classes and that have a least
one dictionary in them.
* __del__ methods sometimes may not be called for instances
involved in cycles.
* It requires some extra memory when doing the collection.
I still need to clean up and test the patch some more but things
are looking good. It currently is not thread safe but should be
possible to make so. If anyone is interested in helping out or
testing it, a prelimiary patch is at:
http://www.ucalgary.ca/~nascheme/python/gc-cycle.diff
Comments are definitely appreciated.
Neil
--
"Applying computer technology is simply finding the right wrench to
pound in the correct screw." -- Tim Wright, Sequent Computer Systems.
Tim Peters
telegraphically condensed -- see the original for more context]
> 1. Reference counting memory management
Change that to "trash cycles aren't reclaimed", and it will eventually get
fixed (possibly sooner than later, given Neil S's latest work). CPython
users don't want to give up the relative predictability of
reference-counting anyway. *Many* view rc as "a feature"!
> ...
> In the long run, the solution is to use a conservative garbage
> collection algorithm (such as the Boehm collector),
"No chance" in CPython -- the BDW collector is a large package of difficult
code, and can be extremely tricky to port to new platforms. Can also be
tricky to get it to play nice with extension modules. CPython has nothing
like that now, and never will (or so I predict). And BDW is an excellent
implementation of this approach, so it's exceedingly unlikely someone is
going to come out with an easier-to-live-with conservative package. Bottom
line: this is as futile as hoping to add curly braces.
> o Use JPython. It hijacks Java's garbage collection, so there are
> no problems with cyclic data structures. It doesn't ever call
> __del__ methods, though I don't think this is a problem in
> practice.
It sure is for people who rely on __del__ methods.
> 2. Lack of lexical scoping
>
> Tim Peters disagrees, but I miss it a lot, even after using Python
> for years.
Actually, Tim wholly agrees that you miss it a lot. He only disagrees when
people insist that *he* misses it a lot <0.1 wink -- they do, you know! and,
no, I don't>. I don't *object* to adding lexical scoping, though, and
Guido's primary objection has been reduced to the cyclic trash problem, so
this is back to #1 now.
> ...
> There's another, more subtle problem -- much of the work that's
> been done on optimizing and analyzing highly dynamic languages has
> been from on the Scheme/ML/Lisp world, and almost of that work
> assumes lexically-scoped environments. So using well-understood
> optimization techniques is made harder by the difference.
Na -- this is like thinking most supercomputer optimization work has focused
on speeding floating-point code, so integer code must be harder <0.9 wink>.
Python's scoping largely reduces to Scheme but where all functions are
defined at top-level (inventing unique names as needed to prevent name
clashes). More formally, you can certainly *model* Python's scoping in
Scheme, so any technique sufficient to optimize Scheme code so modeling
suffices to optimize Python too. How's that for a slick argument <wink>?
Your wallet will be returned at the door.
> Workarounds:
>
> o Greg Ewing has a closure patch, that makes functions and
> lambdas work without creating too much cyclic garbage.
Guido dislikes that implementation.
> ...
> 3. Multiple inheritance is unsound
>
> By 'unsound' I mean that a method call to a method inherited by a
> subclass of two other classes can fail, even if that method call
> would work on an instance of the base class.
Unsure what this is about; am pretty sure nobody has complained about it
before. Certainly "depth first, left to right" is an unprincipled approach,
but in practice it's more than predictable enough to use.
> ...
> Prognosis:
>
> I don't think there's any hope of this ever being fixed. It's just
> too much a part of Python, and there isn't much pressure to fix
> it. Just avoid using MI in most circumstances.
People use MI in Python all the time! It works great. AFAIK, the claim
above is the sole "pressure to fix it".
> 4. Lack of support for highly declarative styles of programming
This is the hardest for me, because I love highly declarative styles but too
often find they grate against Python's greater love of "obviousness". I
certainly favor adding list comprehensions, because they're one happy case
where greater power and greater clarity coincide.
> ...
> o Christian Tismer's Stackless Python patch enables the use of
> first-class continuations in regular Python code. This lets
> people easily and in pure Python create and experiment with all
> sorts of funky control structures, like coroutines, generators,
> and nondeterministic evaluation.
>
> Prognosis:
>
> Pretty good, if Stackless Python makes it in, along with list
> comprehensions. (Personally, I suspect that this is the single most
> important improvement possible to Python, since it opens up a
> whole new category of expressiveness to the language.)
Don't tell him I said this, but that's exactly why Guido fears it (any stuff
he may say about destabilizing the core interpreter loop is a smoke screen
<0.5 wink>): one programmer's "expressiveness" is often another's
"gibberish". Christian needs to find a "killer app" for Stackless. Perhaps
the Palm Pilot port is it. Perhaps some form of migrating computations
among machines is it. Hard to say -- who would have guessed that Zope would
prove popular <wink>. But stressing "adds unbounded new power to express
control flow in novel, unique new ways" is exactly the way to scare 99% of
this community in the other direction. Let's try lying about Stackless
instead ("it's really a floor wax, despite that it was originally sold as a
dessert topping").
> 5. The type/class dichotomy, and the lack of a metaobject system.
Certain to be fixed in P3K, almost certain not to be fixed in CPython 1.x.
> ...
> 6. The iteration protocol
>
> The iteration protocol is kind of hacky. This is partly a function
> of the interface, and partly due to the way the type-class
> dichotomy prevents arranging the collection classes into a nice
> hierarchy.
It was designed to support straightforward sequence types, and works fine
for that. It indeed doesn't generalize beyond that without major strain.
> ...
> o Wait for Guido to invent a better way: Tim Peters has said on the
> newsgroup that GvR is working on a better design,
I've been pushing Icon's generators (Sather's iterators, if that's more
familiar -- "semi-coroutines" for the abstractoids) since '91. A very clean
& general iteration protocol can be built on that (e.g., iteration over
recursive structures is as natural as breathing). It's easy to implement
given Stackless. It's even easier to implement directly. Guido warmed to
it at one point last year, but hasn't mentioned it again (but neither have
I).
Thanks again for the useful post, Neel! It was gourment food for thought.
back-to-cheeze-whiz-and-dairy-whip-ly y'rs - tim
Fredrik Lundh
> > ...
> > o Christian Tismer's Stackless Python patch enables the use of
> > first-class continuations in regular Python code. This lets
> > people easily and in pure Python create and experiment with all
> > sorts of funky control structures, like coroutines, generators,
> > and nondeterministic evaluation.
> >
> > Prognosis:
> >
> > Pretty good, if Stackless Python makes it in, along with list
> > comprehensions. (Personally, I suspect that this is the single most
> > important improvement possible to Python, since it opens up a
> > whole new category of expressiveness to the language.)
>
> Don't tell him I said this, but that's exactly why Guido fears it (any
stuff
> he may say about destabilizing the core interpreter loop is a smoke screen
> <0.5 wink>): one programmer's "expressiveness" is often another's
> "gibberish". Christian needs to find a "killer app" for Stackless.
Perhaps
> the Palm Pilot port is it. Perhaps some form of migrating computations
> among machines is it.
I can name two right away: high-performance web servers
and user interface toolkits. but I fear that nobody's going
to spend much time developing that stuff if they aren't
assured that (at least portions) of stackless python goes
into the core CPython distribution...
(on the other hand, I wouldn't be surprised if one or two
proof-of-concept killer applications could out there when
it's time to start thinking about what to put into 1.7...).
</F>
François Pinard
> > In the long run, the solution is to use a conservative garbage
> > collection algorithm (such as the Boehm collector),
> "No chance" in CPython [...]
I'm not sure of the implications of the above, but one sure thing is that I
very much like the current implications of reference counting. When I write:
for line in open(FILE).readlines():
I rely on the fact FILE will automatically get closed, and very soon since
no other references remain. I could of course use another Python line for
opening FILE, and yet another for explicitely closing it, as I was doing
in my first Python days, but I learned to like terse writings like above,
and I do not think there is any loss in legibility in this terseness.
I'm not even sure I would much attracted by an implementation of Python
in which implied __del__ actions are much delayed, as this would force me
to add some more clutter to my Python code, which I prefer short and clear.
--
François Pinard http://www.iro.umontreal.ca/~pinard
François Pinard
> > 2. Lack of lexical scoping
> >
> > Tim Peters disagrees, but I miss it a lot, even after using Python
> > for years.
> Actually, Tim wholly agrees that you miss it a lot.
Couldn't we approximate lexical scoping with classes? For the few times I
needed it, I was fairly satisfied with this solution. Surely more verbose
than in Scheme, but yet, given I do not use it often, I did not care the
extra-verbosity.
P.S. - If you see a contradiction with my previous message where I praise
terseness, please consider that common and frequent idioms are better terse,
while some verbosity is acceptable for things like lexical scoping, that
we do less often, and for which some extra documentation is welcome.
Michael Hudson
> "Tim Peters" <tim...@email.msn.com> writes:
>
> > > 2. Lack of lexical scoping
> > >
> > > Tim Peters disagrees, but I miss it a lot, even after using Python
> > > for years.
>
> > Actually, Tim wholly agrees that you miss it a lot.
>
> Couldn't we approximate lexical scoping with classes? For the few times I
> needed it, I was fairly satisfied with this solution. Surely more verbose
> than in Scheme, but yet, given I do not use it often, I did not care the
> extra-verbosity.
This is what I fairly often recommend to people who are abusing the
default argument hack, or slate Python for not doing scoping properly.
People still complain though.
It would also be nice if people got the terminology right; Python *is*
lexically scoped, because references to a name can only appear within
code that is textually contained in the establishing contruct (says he
paraphrasing from cltl2). The alternative is the poorly named "dynamic
scope" which would mean things like:
def f():
a = 1
return h()
def g():
a = 1
return h()
def h():
return a
print f(),g()
would print "1 2", which is just not true. This may seem bizarre, but
it can be useful; if you ever find yourself adding arguments to
functions only so you can propogate said options down to functions you
then call, you might have been better off with a dynamically scoped
variable (they're called "special" in common lisp). They're a bit like
acquisition in some ways. They also monumentally don't fit into the
Python way of doing things, so I'm not going to press for them.
The "problem" is that scopes don't nest - except for the scopes of
instance members, so use them instead.
Cheers,
Michael
Neel Krishnaswami
>
> It would also be nice if people got the terminology right; Python
> *is* lexically scoped, because references to a name can only appear
> within code that is textually contained in the establishing contruct
> named "dynamic scope" which would mean things like: [..examples
> elided...]
Python is most definitely neither lexically scoped nor dynamically
scoped, as either term is commonly used. I've found that describing
Python as having three scopes pretty much gets the idea across
unambiguously.
Neel
Christian Tismer
Fredrik Lundh wrote:
>
> Tim Peters <tim...@email.msn.com> wrote:
> > > ...
> > > o Christian Tismer's Stackless Python patch enables the use of
> > > first-class continuations in regular Python code. This lets
> > > people easily and in pure Python create and experiment with all
> > > sorts of funky control structures, like coroutines, generators,
> > > and nondeterministic evaluation.
Tim:
> > Don't tell him I said this, but that's exactly why Guido fears it (any
> > stuff he may say about destabilizing the core interpreter loop is a
> > smoke screen <0.5 wink>): one programmer's "expressiveness" is often
> > another's "gibberish".
> > Christian needs to find a "killer app" for Stackless.
> > Perhaps the Palm Pilot port is it. Perhaps some form
> > of migrating computations among machines is it.
Fredrik:
> I can name two right away: high-performance web servers
> and user interface toolkits. but I fear that nobody's going
> to spend much time developing that stuff if they aren't
> assured that (at least portions) of stackless python goes
> into the core CPython distribution...
The Palm project is promising, but I didn't hear back for
a week. Maybe I have to buy a Palm?
A number of Zope gurus seemed interested on SPAM8, for
the webserver issue, but also about new control structures.
I'm still waiting for Sam Rushing to show up. He caused me
to write that stuff. :-)
All in all: I need in fact a killer app. Stackless is hard to
understand, the new possibilities are hard to figure out,
and that is a showstopper unless I can provide an easier
entrance into this.
Well, Fredrik is right. I will be the "nobody", and I will
have to build a killer app. I'm planning to show it on
OSCOM2000, thinking of Eric Raimond's words: "Don't give up!"
ciao - chris
p.s.: After SPAM8, I was asked by a colleague:
"And now? Are they still stacking snames?"
--
Christian Tismer :^) <mailto:tis...@appliedbiometrics.com>
Applied Biometrics GmbH : Have a break! Take a ride on Python's
Düppelstr. 31 : *Starship* http://starship.python.net
12163 Berlin : PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF
we're tired of banana software - shipped green, ripens at home
Tim Peters
> In the long run, the solution is to use a conservative garbage
> collection algorithm (such as the Boehm collector),
[Tim]
> "No chance" in CPython [...]
[François Pinard]
> I'm not sure of the implications of the above, but one sure thing
> is that I very much like the current implications of reference
> counting. When I write:
>
> for line in open(FILE).readlines():
>
> I rely on the fact FILE will automatically get closed, and very soon
> since no other references remain. ...
Yes, that's exactly the kind of thing that makes many CPython users consider
RC to be "a feature". There's no reason at all to believe that CPython will
ever give this up; even NeilS's BDW patch for CPython retains refcounting,
simply for *speed* reasons (RC has excellent locality of reference,
especially when trash is being created and recycled at high dynamic rates).
just-repeating-"no-chance"-ly y'rs - tim
Tim Peters
> ...
> Couldn't we approximate lexical scoping with classes?
Certainly: the things Scheme does with lexical closures were *intended* to
be done with classes and explicit instance state in Python. Every now &
again someone comes along and insists the latter aren't sufficient, but they
are -- in Python. Python's flavor of classes are more powerful than most
realize at first.
> For the few times I needed it, I was fairly satisfied with this
> solution.
Me too, but after s/fairly/very/ after many times needing it. The class
approach is more flexible in practice, thanks to the explictly manipulable
state (e.g., conceptually small changes don't require juggling six levels of
nested closure -- it's often enough to just add a simple new method to
fiddle an aggregate closure state maintained in a vanilla class instance).
> Surely more verbose than in Scheme, but yet, given I do not use
> it often, I did not care the extra-verbosity.
>
> P.S. - If you see a contradiction with my previous message where
> I praise terseness, please consider that common and frequent idioms
> are better terse, while some verbosity is acceptable for things
> like lexical scoping, that we do less often, and for which some
> extra documentation is welcome.
I suspect that, like me, you're not doing any GUI programming in Python yet.
The press for fancier lambdas and lexical closures is much stronger from
people who do: they want to attach small & simple procedures to oodles &
oodles of widgets (your "common and frequent" indeed), and out-of-line class
solutions are legitimately viewed as clumsier and more obscure.
"one-obvious-way"-vs-"one-pleasant-obvious-way-depending"-ly
y'rs - tim
Alex
> > Couldn't we approximate lexical scoping with classes?
>
> Certainly
Could someone give me an example of how to do this? Probably I am
simply confused about the terminology, but don't you run into the same
sorts of namespace problems with nesting classes as you do with nesting
functions?
Alex.
Neel Krishnaswami
>
> > > Couldn't we approximate lexical scoping with classes?
>
> simply confused about the terminology, but don't you run into the same
> sorts of namespace problems with nesting classes as you do with nesting
> functions?
Certainly; here's a teeny closure in Scheme:
(define make-adder
(lambda (n)
(lambda (x) (+ n x))))
(define add-three (make-adder 3))
(add-three 6)
-> 9
(add-three 9)
-> 12
And here's the Python equivalent, using classes:
>>> class Adder:
... def __init__(self, n):
... self.n = n
... def __call__(self, x):
... return self.n + x
...
>>> add_three = Adder(3)
>>> add_three(6)
9
>>> add_three(9)
12
Basically, you use the __call__ method to let instances use the
function call syntax.
Neel
Neel Krishnaswami
> [ne...@cswcasa.com, with an excellent & thoughtful piece, herewith
> telegraphically condensed -- see the original for more context]
>
> > 1. Reference counting memory management
>
> Change that to "trash cycles aren't reclaimed", and it will eventually get
> fixed (possibly sooner than later, given Neil S's latest work).
Done -- this is what I meant anyway. :)
> > 3. Multiple inheritance is unsound
> >
> > By 'unsound' I mean that a method call to a method inherited by a
> > subclass of two other classes can fail, even if that method call
> > would work on an instance of the base class.
>
> Unsure what this is about; am pretty sure nobody has complained about it
> before. Certainly "depth first, left to right" is an unprincipled approach,
> but in practice it's more than predictable enough to use.
Here's some code:
class Foo:
x = 99
def frob(self):
print "%d bottles of beer" % self.x
class Bar:
x = "bar"
def wibble(self):
print "Belly up to the " + self.x
Note that all the operations on Foo and Bar work safely for direct
instances of Foo and Bar. But if you define a subclass like so:
class Baz(Bar, Foo):
pass
you can get type errors even though both of the superclasses have
type-safe operations:
>>> q = Baz()
>>> q.frob()
Traceback (innermost last):
[...]
TypeError: illegal argument type for built-in operation
This is only an annoyance in day-to-day work, but it becomes a big
problem if you are serious about building automatic code analysis
tools for Python (eg for CP4E). This is because the soundness theorems
needed to write things like soft typing tools no longer hold. :(
> > 4. Lack of support for highly declarative styles of programming
>
> This is the hardest for me, because I love highly declarative styles but too
> often find they grate against Python's greater love of "obviousness". I
> certainly favor adding list comprehensions, because they're one happy case
> where greater power and greater clarity coincide.
Agreed that slinging higher-order functions around like there's no
tomorrow is often very cryptic, but that's not quite what I meant. I
mean "declarative" more in the sense of SQL, and I firmly believe that
this is frequently far and away the most natural and obvious style of
describing a very large class of problems. Deeply nested for loops
aren't "obvious" either, unless you've been programming for
years.
Plus it's not just databases that can benefit -- empirically the
constraint based approach to GUIs is much simpler to work with. And
having two important and very different domains call out for this
stuff is a big sign it's significant IMO.
> > 6. The iteration protocol
> >
>
> I've been pushing Icon's generators (Sather's iterators, if that's more
> familiar -- "semi-coroutines" for the abstractoids) since '91. A very clean
> & general iteration protocol can be built on that (e.g., iteration over
> recursive structures is as natural as breathing). It's easy to implement
> given Stackless. It's even easier to implement directly. Guido warmed to
> it at one point last year, but hasn't mentioned it again (but neither have
> I).
I have only a nodding acquaintance with Icon or Sather, but I've
programmed in CLU, and really enjoyed it. I suspect that in large part
this was because of the cleanness with which iteration could be
expressed in it. Iteration is an awful big chunk of what programs
*do*, and being able to say it cleanly made writing them easier.
Neel
Alex
Oh, I see. Thanks for the clarification, Neel.
Alex.
Michael Hudson
> > > 3. Multiple inheritance is unsound
> > >
> > > By 'unsound' I mean that a method call to a method inherited by a
> > > subclass of two other classes can fail, even if that method call
> > > would work on an instance of the base class.
> >
> > Unsure what this is about; am pretty sure nobody has complained about it
> > before. Certainly "depth first, left to right" is an unprincipled approach,
> > but in practice it's more than predictable enough to use.
>
> Here's some code:
>
> class Foo:
> x = 99
> def frob(self):
> print "%d bottles of beer" % self.x
>
> class Bar:
> x = "bar"
> def wibble(self):
> print "Belly up to the " + self.x
>
> Note that all the operations on Foo and Bar work safely for direct
> instances of Foo and Bar. But if you define a subclass like so:
>
> class Baz(Bar, Foo):
> pass
>
> you can get type errors even though both of the superclasses have
> type-safe operations:
>
> >>> q = Baz()
> >>> q.frob()
> Traceback (innermost last):
> [...]
> TypeError: illegal argument type for built-in operation
>
> This is only an annoyance in day-to-day work, but it becomes a big
> problem if you are serious about building automatic code analysis
> tools for Python (eg for CP4E). This is because the soundness theorems
> needed to write things like soft typing tools no longer hold. :(
How is this different from (excuse the dodgy syntax, I haven't done
much Eiffel):
class FOO
feature
x: INTEGER
feature frob: STRING is
do
Result = interpolate("%d bottles of beer",x.to_string)
end
end
class BAR
feature
x: STRING
feature wibble: STRING is
do
Result = concatenate("Belly up to the ",x)
end
end
class BAZ
inherit FOO,BAR
end
? I mean, that won't compile, presumably, but in the case of a code
analysis tool there comes a point when it can just throw it's hands up
in the air and say "you're not playing by the rules, I give
up". Repeasted feature inheritance is not a solved problem in any
language I'm aware of.
And in this case, the code has a bug, so code analysis catching it is
surely a good thing?
not-at-understanding-yet-ly y'rs
Michael
Tim Peters
Tim recounts a bit of the history of trying to get Icon
generators / Sather iterators into Python]
[Neel Krishnaswami]
> I have only a nodding acquaintance with Icon or Sather, but I've
> programmed in CLU, and really enjoyed it.
Bingo. Sather's iterators were inspired by CLU's, but removed some
restrictions (e.g., IIRC CLU catered to only one iterator per loop, while
Sather can iterate over multiple objects-- each with its own iterator --in a
single loop). Icon's generators came from a different view of the world,
seen there as exposing a key mechanism in pattern matching engines.
However people come at this, though, some flavor of semi-coroutine keeps
getting reinvented: it's utterly natural for many kinds of
producer/consumer problems (of which iteration is one), and a sanity
lifesaver when the producer has data *and* control-flow state to keep track
of. All doable with threads or continuations, but they're gross overkill
for the little that's actually required.
> I suspect that in large part this was because of the cleanness
> with which iteration could be expressed in it. Iteration is an
> awful big chunk of what programs *do*, and being able to say it
> cleanly made writing them easier.
Yup! Here's a made-up example in pseudo-Python, for iterating over a binary
tree in postorder. This borrows the Icon "suspend" construct, which acts
like a "resumable return" (in implementation terms, the stack frame isn't
thrown away, and "the next call" merely resumes it exactly where it left
off):
class BinTree:
# with members .left and .right of type BinTree (& None means none),
# and .data of an arbitrary type
...
def traverse_post(self):
for child in self.left, self.right:
if child is not None:
suspend child.traverse_post()
suspend self.data
b = BinTree()
...
for leaf in b.traverse_post():
process(leaf)
It's hard to imagine more straightforward code for this task, although easy
to imagine terser code (e.g., that whole thihg is an idiomatic one-liner in
Icon, but "everything's a generator" in Icon and so has gobs of initially
cryptic shortcuts & special syntactic support).
Much more on this flavor of semi-coroutine can be dug out of c.l.py
archives. I've often called it "resumable functions" on c.l.py, in a mostly
futile attempt to get across how conceptually simple it is (it's impossible
to mention this without someone chiming in with "but you can do that with
continuations!", and after the first Scheme example of *that*, the whole
newsgroup flees in terror <0.5 wink>).
ever-notice-that-sicp-doesn't-mention-call/cc-even-once?-ly
y'rs - tim
Samuel A. Falvo II
>class BinTree:
> # with members .left and .right of type BinTree (& None means none),
> # and .data of an arbitrary type
> ...
> def traverse_post(self):
> for child in self.left, self.right:
> if child is not None:
> suspend child.traverse_post()
> suspend self.data
>
>b = BinTree()
>...
>for leaf in b.traverse_post():
> process(leaf)
I'm sorry, but I can't follow this code at all. What are the precise
semantics of suspend here? How does it return?
--
KC5TJA/6, DM13, QRP-L #1447
Samuel A. Falvo II
Oceanside, CA
Aahz Maruch
Samuel A. Falvo II <kc5...@garnet.armored.net> wrote:
>In article <000001bf768e$48e40580$45a0143f@tim>, Tim Peters wrote:
>>
>>class BinTree:
>> # with members .left and .right of type BinTree (& None means none),
>> # and .data of an arbitrary type
>> ...
>> def traverse_post(self):
>> for child in self.left, self.right:
>> if child is not None:
>> suspend child.traverse_post()
>> suspend self.data
>>
>>b = BinTree()
>>...
>>for leaf in b.traverse_post():
>> process(leaf)
>
>I'm sorry, but I can't follow this code at all. What are the precise
>semantics of suspend here? How does it return?
suspend *is* a return. It also -- at the same time -- stores the
program counter for b.traverse_post(), so that the next call to
b.traverse_post() starts at the line of execution immediately following
the suspend. This means that, should someone be foolish enough to call
b.traverse_post() inside the loop "for leaf...", the loop will probably
return incorrect results, just like modifying a list while you iterate
over it.
Using suspend means that people no longer have to hack __getitem__() to
"do the right thing" for a class. Doing this in a threaded environment
is dangerous, but no more so than dealing with any other class.
--
--- Aahz (Copyright 2000 by aa...@netcom.com)
Androgynous poly kinky vanilla queer het <*> http://www.rahul.net/aahz/
Hugs and backrubs -- I break Rule 6
Have you coined a word today?
William Tanksley
>In article <000001bf768e$48e40580$45a0143f@tim>, Tim Peters wrote:
>>class BinTree:
>> # with members .left and .right of type BinTree (& None means none),
>> # and .data of an arbitrary type
>> ...
>> def traverse_post(self):
>> for child in self.left, self.right:
>> if child is not None:
>> suspend child.traverse_post()
>> suspend self.data
>>b = BinTree()
>>...
>>for leaf in b.traverse_post():
>> process(leaf)
>I'm sorry, but I can't follow this code at all. What are the precise
>semantics of suspend here? How does it return?
Sam, imagine that instead of 'suspend' Tim had written 'return'. You see,
then, that this would be a deeply recursive function which would return
the data contained in the deepest node of the left-hand branch of the
tree, right? (Ignore the fact that such recursion would be rather silly
-- thanks to the 'suspend' it's not the same.)
Well, it so happens that this is precisely what this function (using
suspend instead of return) returns the first time it's called. The second
time it's called it continues what it was doing when it did the last
suspend -- look at it. It's asking each of its children to do a
post-order traversal, and after they finish, return their own data. In
other words, it's doing a depth-first search and allowing you to treat the
result as a sequence.
Kind of.
If you have more questions, call me; I teach better that way. Or you
could learn Icon -- it's a really cool language, and lots of fun. Sather
includes a similar construct, although (alas!) I never saw it as fun
there. Perhaps I was jaded, or maybe Sather didn't do it as well.
>KC5TJA/6, DM13, QRP-L #1447
--
-William "Billy" Tanksley, in hoc signo hack
Tim Peters
Tim's unelaborated example]
[Aahz Maruch, proves that you don't have to be a lambda-head to
pick up the thrust of it at once -- thanks!]
> suspend *is* a return. It also -- at the same time -- stores the
> program counter for b.traverse_post(), so that the next call to
> b.traverse_post() starts at the line of execution immediately following
> the suspend.
Right so far as it goes -- also stores away "everything else" needed, like
local vars and their current bindings. This is cheap since it's just the
state of the frame object, and half the trick of "suspend" lies simply in
not decrementing the frame object's refcount! In effect, it freezes the
function in mid-stride, returns a result to the caller, and waits to get
thawed again.
> This means that, should someone be foolish enough to call
> b.traverse_post() inside the loop "for leaf...", the loop will
> probably return incorrect results, just like modifying a list
> while you iterate over it.
Actually not a problem. Under the covers, the implementation has to
distinguish between "the first call" and "resuming calls". A first call
always gets a fresh frame of its own, just like any other call today. It's
only resuming that's new behavior on the call side (& resuming is much
cheaper than a first call -- it simply restarts the frame object from where
it suspended). So you could have dozens of iterators marching over the same
tree, and they wouldn't interfere with each other. "for" can hide all that
by magic, but it would also be Pythonic to expose the machinery so that you
*could* do fancier things yourself.
> Using suspend means that people no longer have to hack __getitem__()
> to "do the right thing" for a class.
Try even *writing* a binary tree postorder traversal via __getitem__(), then
ponder NeelK's original points again. You can't do better than faking
recursion via a hand-simulated stack (unnatural, tedious and error-prone),
or building the entire result sequence up front then passing it out a piece
at a time (can be too space-intensive, or waste time if you're simply doing
a search and will likely "get out early"). Note that binary tree traversal
is about as simple as you can get without being trivial: "resumable
functions" already make a night-and-day difference here, but are that much
more helpful the harder the task.
> Doing this in a threaded environment is dangerous, but no more so
> than dealing with any other class.
Or any other kind of call -- each iterator/generator gets its own locals, so
the only dangers are in mutating while iterating, or communicating info via
unprotected globals. The same cautions apply to __getitem__ today.
it-tends-to-"just-work"-all-by-itself-ly y'rs - tim
Mikael Olofsson
On 13-Feb-00 Christian Tismer wrote:
> Well, Fredrik is right. I will be the "nobody", and I will
> have to build a killer app. I'm planning to show it on
> OSCOM2000, thinking of Eric Raimond's words: "Don't give up!"
Christian!
Don't give up, I mean that, but I just have to give you a few Swedish
words of wisdom. The first one is from the northern part of Sweden,
known for it's calm people who speak with very few words. Here goes:
It's never too late to give up!
The second one was given to me by one of my colleagues. I don't know its
true original.
To get ready is to give up.
Now, that can be interpreted in at least two ways. One interpretion is
"you should never get ready, because then you have given up". An other
is "If you are ever to get ready, then you have to give up". Which one
you prefer is probably a matter of taste.
/Mikael
-----------------------------------------------------------------------
E-Mail: Mikael Olofsson <mik...@isy.liu.se>
WWW: http://www.dtr.isy.liu.se/dtr/staff/mikael
Phone: +46 - (0)13 - 28 1343
Telefax: +46 - (0)13 - 28 1339
Date: 14-Feb-00
Time: 08:30:39
This message was sent by XF-Mail.
-----------------------------------------------------------------------
Justus Pendleton
ne...@alum.mit.edu wrote:
> > > 3. Multiple inheritance is unsound
> > >
> > > By 'unsound' I mean that a method call to a method inherited by
a
> > > subclass of two other classes can fail, even if that method
call
> > > would work on an instance of the base class.
>
> class Foo:
> x = 99
> def frob(self):
> print "%d bottles of beer" % self.x
>
> class Bar:
> x = "bar"
> def wibble(self):
> print "Belly up to the " + self.x
>
> Note that all the operations on Foo and Bar work safely for direct
> instances of Foo and Bar. But if you define a subclass like so:
>
> class Baz(Bar, Foo):
> pass
In an earlier post you mentioned that Dylan gets this "right" but as I
understand it, in Dylan two classes with the same getter generic
function ("x" in this case) are "disjoint - they can never have a common
subclass and no object can be an instance of both classes" (from the
Dylan Reference Manual section on Slot Inheritance). It seems like
Dylan resolves this by simply saying you can't write code like that, not
by making Multiple Inheritance safer. Or am I missing something?
Sent via Deja.com http://www.deja.com/
Before you buy.
Aahz Maruch
Tim Peters <tim...@email.msn.com> wrote:
>
>[Samuel A. Falvo II, wonders about "suspend" semantics after
> Tim's unelaborated example]
>
>[Aahz Maruch, proves that you don't have to be a lambda-head to
> pick up the thrust of it at once -- thanks!]
I have to admit that I used Icon briefly in a CS class more than a
decade ago; it made much more of an impression on me than Lisp did, even
though I still have my LISPcraft book and I can't seem to find my Icon
book. (My professor was from UoA.) It was interesting trying to cast
my memory back that far.
BTW, it turns out that much of the Icon documentation is now on-line at
http://www.cs.arizona.edu/icon/
From my POV, Icon is a lot like Perl written by a Pascal programmer,
with (from my more experienced viewpoint) its almost excessive emphasis
on expressions rather than statements. I think that overall Python
doesn't go quite far enough in that direction, but adding generators
should make up much of the balance.
Martin von Loewis
> I'm not sure of the implications of the above, but one sure thing is that I
> very much like the current implications of reference counting. When I write:
>
> for line in open(FILE).readlines():
>
> I rely on the fact FILE will automatically get closed, and very soon since
> no other references remain. I could of course use another Python line for
> opening FILE, and yet another for explicitely closing it, as I was doing
> in my first Python days, but I learned to like terse writings like above,
> and I do not think there is any loss in legibility in this terseness.
Indeed. I think Tim's right here (as always:); if the request for
garbage collection is rephrased as 'reclaim cycles', everybody will be
happy.
Regards,
Martin
Bijan Parsia
[snip]
> Python's scoping largely reduces to Scheme but where all functions are
> defined at top-level (inventing unique names as needed to prevent name
> clashes). More formally, you can certainly *model* Python's scoping in
> Scheme,
[snip]
Er..Just out of curiosity, what *can't* you model in Scheme? :)
And-not-in-the-trivial-turing-machine-sense-ly y'rs,
Bijan Parsia
P.S. Hey! Where's my wallet?
Bjorn Pettersen
def readline(name):
fp = open(name)
while 1:
line = fp.readline()
if not line: break
suspend line
fp.close()
(someone please correct my semantics if I'm wrong -- I really haven't played enough
with suspend yet :-)
of-course-you-can-do-the-same-thing-with-continuations<wink>'ly y'rs
-- bjorn
Vladimir Marangozov
>
> [NeelK]
> > In the long run, the solution is to use a conservative garbage
> > collection algorithm (such as the Boehm collector),
>
> [Tim]
> > "No chance" in CPython [...]
>
> [François Pinard]
> > is that I very much like the current implications of reference
> > counting. When I write:
> >
> > for line in open(FILE).readlines():
> >
> > I rely on the fact FILE will automatically get closed, and very soon
>
> consider RC to be "a feature". There's no reason at all to believe
> that CPython will ever give this up; even NeilS's BDW patch for CPython
> retains refcounting, simply for *speed* reasons (RC has excellent
> locality of reference, especially when trash is being created and
> recycled at high dynamic rates).
>
That last one is a good one. So far, nobody has shown any evidence that
LOR is good or bad when different objects are involved (except, I
remember,
some special cases like iterating over ints that happen to be allocated
close to each other). Better yet, nobody _can_ show such evidence as far
as one stays on top of a virtual memory manager.
make-that-simply-"RC-is-faster-than-the-alternatives"-<wink>-
-and-I'll-take-it-ly y'rs
--
Vladimir MARANGOZOV | Vladimir....@inrialpes.fr
http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252
Charles Hixson
wouldn't you? Where Python doesn't have declarations of variables. I suppose that
one could interpolate a rule that said something like:
"The value of a variable inherited from more than one ancestor class must be
prefixed by the name of the class from which it was inherited."
Which shouldn't break any code that was currently safe. The other alternative would
be to require that those variables be renamed (in essence, the Eiffel solution) ..
Thus:
for Foo.x use fooX
for Bar.x use barX
etc.
Neither solution seems to fit *really* nicely into Python. Neither clashes very
badly. My personal preference is the renaming solution, because otherwise
occasionally names will grow too long to be convenient. But it doesn't seem a
difficult problem, merely a real one.
Michael Hudson wrote:
> ne...@brick.cswv.com (Neel Krishnaswami) writes:
>
> > > > 3. Multiple inheritance is unsound
> > > >
> > > > By 'unsound' I mean that a method call to a method inherited by a
> > > > subclass of two other classes can fail, even if that method call
> > > > would work on an instance of the base class.
> > >
> > > Unsure what this is about; am pretty sure nobody has complained about it
> > > before. Certainly "depth first, left to right" is an unprincipled approach,
> > > but in practice it's more than predictable enough to use.
> >
> > Here's some code:
> >
> > class Foo:
> > x = 99
> > def frob(self):
> > print "%d bottles of beer" % self.x
> >
> > class Bar:
> > x = "bar"
> > def wibble(self):
> > print "Belly up to the " + self.x
> >
> > Note that all the operations on Foo and Bar work safely for direct
> > instances of Foo and Bar. But if you define a subclass like so:
> >
> > class Baz(Bar, Foo):
> > pass
> >
Michael Hudson
> Well, in Eiffel one would need to declare the inheritance pattern to
> by used by x wouldn't you?
Fair point.
> Where Python doesn't have declarations
> of variables. I suppose that one could interpolate a rule that said
> something like:
> "The value of a variable inherited from more than one ancestor class must be
> prefixed by the name of the class from which it was inherited."
> Which shouldn't break any code that was currently safe. The other
> alternative would be to require that those variables be renamed (in
> essence, the Eiffel solution) ..
> Thus:
> for Foo.x use fooX
> for Bar.x use barX
> etc.
>
> Neither solution seems to fit *really* nicely into Python. Neither
> clashes very badly. My personal preference is the renaming
> solution, because otherwise occasionally names will grow too long to
> be convenient. But it doesn't seem a difficult problem, merely a
> real one.
It's not exactly an area I know a huge amount about, but I do know
that in Eiffel you can still get yourself in trouble along these lines
(I'm not too sure how; people who know more than I have claimed this),
and Eiffel is about the most carefully-designed-to-type-safety OO
langauge I know of, so I'd say it was a difficult problem.
Cheers,
M.
Tim Peters
> ...
> More formally, you can certainly *model* Python's scoping in Scheme,
[Bijan Parsia]
> Er..Just out of curiosity, what *can't* you model in Scheme? :)
Smalltalk <wink>.
> And-not-in-the-trivial-turing-machine-sense-ly y'rs,
> Bijan Parsia
>
> P.S. Hey! Where's my wallet?
at-the-door-just-waiting-for-you-to-find-the-exit-ly y'rs - tim
Tim Peters
> ...
> (RC has excellent locality of reference, especially when trash
> is being created and recycled at high dynamic rates).
[Vladimir Marangozov]
> That last one is a good one. So far, nobody has shown any evidence
> that LOR is good or bad when different objects are involved (except,
> I remember, some special cases like iterating over ints that happen
> to be allocated close to each other). Better yet, nobody _can_ show
> such evidence as far as one stays on top of a virtual memory manager.
Vlad, you're trying to get "deep" on what is really a "shallow" point: a
"high dynamic rate" is most likely to occur when a program is doing flat-out
computation, so type homogeneity is likely. Like ints or floats or whatever
in Python. And those are *not* "on top of a virtual memory manager" in
Python, but involve cycling & recycling via Python's type-specific internal
"free lists" (some of which you added, so you should remember this <wink>).
So long as the free lists act like LIFOs, you get to reuse the same little
chunks of memory over and over and over ... again. It's LOR in *both* time
and space, and of course that's "good": it's the difference between page
faults and, umm, not page faults <wink>.
> make-that-simply-"RC-is-faster-than-the-alternatives"-<wink>-
> -and-I'll-take-it-ly y'rs
I believe there's sufficient evidence that state-of-the-art "real gc" can be
made faster than rc. Python isn't other languages, though, and so long as
everything in Python is always "boxed", other languages' gc experiences
aren't especially relevant.
not-one-byte-in-python-is-stack-allocated-ly y'rs - tim
Neel Krishnaswami
> In article <slrn8aec0e...@brick.cswv.com>,
> ne...@alum.mit.edu wrote:
> > > >
> > > > By 'unsound' I mean that a method call to a method inherited by
> > > > a subclass of two other classes can fail, even if that method
> > > > call would work on an instance of the base class.
>
> understand it, in Dylan two classes with the same getter generic
> function ("x" in this case) are "disjoint - they can never have a common
> subclass and no object can be an instance of both classes" (from the
> Dylan Reference Manual section on Slot Inheritance). It seems like
> Dylan resolves this by simply saying you can't write code like that, not
> by making Multiple Inheritance safer. Or am I missing something?
No, not at all. Dylan makes MI safe by not letting you use it in
unsound ways. There's a moral here, but I'll leave it for the end of
the post. :)
Michael Hudson posted an Eiffel sample, and pointed out that they use
feature renaming to deal with the name conflict problem. However, this
is not something that works really cleanly in a language as dynamic as
Python -- to raise the appropriate exception when an ambiguous name is
added to a class dictionary would involve scanning the class hierarchy
both up and down for every new name. (It still might be the right
solution, depending on where you put the balance between simplicity of
implementation and safety for the user.)
So I looked at some other dynamic languages to see what they did.
o Dylan -- forbids using MI when there are multiple slots with the
same name.
o Cecil -- same as Dylan, more or less. It complains when you inherit
multiple fields with the same name.
o Smalltalk/Self/Ruby -- only allow single inheritance, so the
problem just doesn't arise.
o Oaklisp -- uses exactly the same rule as Python, and ignores the
problem of soundness just like Python does.
o PLT Scheme -- only allows single inheritance of implementation, and
MI of interface a la Java, so that ambiguities can be detected and
complained about.
o Common Lisp -- has a linearization rule that is more complicated
than Python's but still unsound.
Other languages seem to either forbid name conflicts or just suck it
up and live with an unsafe rule, which leads me to conclude that the
moral is TANSTAAFL -- there ain't no such thing as a free lunch.
Neel
Alan Daniels
(loe...@informatik.hu-berlin.de) spoke forth to us, saying...
>Indeed. I think Tim's right here (as always:); if the request for
>garbage collection is rephrased as 'reclaim cycles', everybody will be
>happy.
Okay, I'm going to go out on a limb here and demonstrate my lack of
knowledge on subtle language issues like this one, but here goes: The
debate seems to come down between reference counting (which collects
items immediately, but can lose objects due to cyclical references)
and Java-style garbage collection for lack of a better phrase (which
reclaims *all* objects, but does it in its own sweet time).
My question is: How hard would it be to keep the reference counting,
but add a built-in function, maybe called "gc()", that would walk
through all the existing objects, find the ones that have been
orphaned due to cyclical references, and reclaim them? Would something
like that be enough to make everybody happy? Possibly implemented as a
patch so that it was optional? It would still be deterministic (in
that __del__ would always get called right away, when appropriate),
but would still make sure that no objects were ever left unreclaimed
indefinitely.
Would that work, or no?
--
=======================
Alan Daniels
dan...@mindspring.com
dan...@cc.gatech.edu
Neil Schemenauer
[about a reference cycle collection scheme]
>Would that work, or no?
Basicly, yes.
Neil
Andrew Cooke
"Tim Peters" <tim...@email.msn.com> wrote:
> def traverse_post(self):
> for child in self.left, self.right:
> if child is not None:
> suspend child.traverse_post()
> suspend self.data
That finally hammered home to me just what continuations are all about.
Does anyone have something similarly elegant that shows a coroutine?
I've just had a look at the stackless python documentation and
coroutines seem to be described as something coming out of a single
detailed example. Is there a simpler definition? (I did look back
through some posts on Deja, but there was nothing recent that seemed to
explain what a coroutine actually is - sorry if I've missed something
obvious).
Also, comp.lang.lisp is currently dissing continuations. Would that be
because the alternative is to pass the code that will process the node
into the iterator as a (first class) function? Obviously, in this case,
yes, but is that true in general (are there examples where continuations
or coroutines make something possible that really is tricky to do in
other ways)?
Thanks,
Andrew
Justus Pendleton
ne...@alum.mit.edu wrote:
> No, not at all. Dylan makes MI safe by not letting you use it in
> unsound ways.
There are a lot of things you can do in Python that don't work properly
at run time. I guess I don't see how MI is a different issue from
those. It seemed like the languages you listed that do things properly
don't have anything equivalent to an eval while those that are broken
do. It seems like it would be hard for Python to give any kind of
static guarantees on this given what you can do to a class at runtime in
Python.
Tim Peters
>>> 3. Multiple inheritance is unsound
>>>
>>> By 'unsound' I mean that a method call to a method inherited by a
>>> subclass of two other classes can fail, even if that method call
>>> would work on an instance of the base class.
[Tim]
>> Unsure what this is about; am pretty sure nobody has complained
>> about it before. Certainly "depth first, left to right" is an
>> unprincipled approach, but in practice it's more than predictable
>> enough to use.
[Neel]
> Here's some code:
>
> class Foo:
> x = 99
> def frob(self):
> print "%d bottles of beer" % self.x
>
> class Bar:
> x = "bar"
> def wibble(self):
> print "Belly up to the " + self.x
>
> Note that all the operations on Foo and Bar work safely for direct
> instances of Foo and Bar. But if you define a subclass like so:
>
> class Baz(Bar, Foo):
> pass
>
> you can get type errors even though both of the superclasses have
> type-safe operations:
>
> >>> q = Baz()
> >>> q.frob()
> Traceback (innermost last):
> [...]
> TypeError: illegal argument type for built-in operation
Ah! That one. Don't worry about it <wink>.
> This is only an annoyance in day-to-day work,
It's never been an annoyance for me -- simply hasn't come up. I suppose it
may someday. Then I'll rename the ambiguous attr <0.9 wink>.
> but it becomes a big problem if you are serious about building
> automatic code analysis tools for Python (eg for CP4E). This is
> because the soundness theorems needed to write things like soft
> typing tools no longer hold. :(
The Types-SIG is wrestling with such things in a preliminary way, and it
already appears all but decided that the full range of tricks you can pull
in Python today are simply uncheckable, or even inexpressible with
reasonable effort (e.g., the canonical example of that is "spelling" the
type of Python's "map" function -- Python isn't curried, so an unboundedly
many "base" signatures get involved).
The language won't be changed to forbid such things, but if you want to use
type checking you're going to have to accept restrictions on what you can
code. I expect that, as in most other languages that take this all too
seriously <wink>, the checker will look at the code above and say "name
clash -- you're on your own; I can't help". I personally would be delighted
to get such a warning & rewrite the offending code, since I personally would
never want to rely on "depth-first left-to-right" for resolving clashes.
But somebody else may want to, and that's fine too.
Then again, overall, I think MI where implementation (beyond mere interface)
is inherited from more than one class is usually far more trouble than it's
worth (simple mixin classes are an exception), so I don't tend to get
anywhere near this kind of trouble. As you noted later, SI languages don't
have a problem here, and in practice it's (IMO) *usually* best to pretend an
MI language is an SI one.
I'll backtrack here & agree w/ your original post: this is indeed unlikely
to "get fixed". Although it's likely you *will* get an option to warn about
cases like this, at least in code where it *can* be detected at
compile-time, and provided you explicitly ask to get warned.
theoretically-absurd-yet-practically-sufficient-ly y'rs - tim
Tim Peters
> [Aahz Maruch, proves that you don't have to be a lambda-head to
> pick up the thrust of it at once -- thanks!]
[and Aahz sez]
> I have to admit that I used Icon briefly in a CS class more than a
> decade ago; it made much more of an impression on me than Lisp did,
> even though I still have my LISPcraft book and I can't seem to find
> my Icon book. (My professor was from UoA.) It was interesting
> trying to cast my memory back that far.
So it *stuck* across a decade, anyway, despite brief exposure. That's a
good sign! Python isn't as concerned about things that are obvious at first
glance, as about things that you don't forget once mastered (e.g., "points
*between* elements" indices). The Icon book is now on its 3rd edition, BTW,
and you should treat yourself to a copy <wink -- but it's among the best
intro language books ever>.
> BTW, it turns out that much of the Icon documentation is now
> on-line at
> http://www.cs.arizona.edu/icon/
Along with free distributions & source code, for people who don't know that
Icon is one of the oldest Open Source languages going (actually, Icon is
public domain -- *you* can claim the copyright <wink>).
> From my POV, Icon is a lot like Perl written by a Pascal programmer,
> with (from my more experienced viewpoint) its almost excessive emphasis
> on expressions rather than statements.
Bad Icon can make bad Perl look crystal clear, athough the Icon community is
good about being as clear as possible. Unfortunately, when "success" and
"silent failure" are fundamental notions applying to every little
expression, even in exemplary Icon it can be very difficult to figure out
whether a particular potential for failure is being ignored by mistake or
design. So while it's brilliantly innovative in many respects, in practice
I found I coudn't keep larger Icon programs working under modification.
It's still my first language of choice for tackling several kinds of complex
one-shot "researchy" problems, though.
> I think that overall Python doesn't go quite far enough in that
> direction,
I agree, but it's gotten "better" this way, mostly via little things like
the addition of dict.get() and 3-argument getattr -- not everything is
better spelled on 4 lines <wink>. Stroustrup has written that he would like
C++ to treat everything as an expression too, so that e.g. even loops could
be rvalues (as they can be in Icon -- and, irritatingly, sometimes are). If
Guido had to err here, I think he picked the right side.
> but adding generators should make up much of the balance.
Unsure it would help here, at least in the flavor I've been selling them.
*Everything* in Icon is "a generator", even the integer 42 (it generates a
singleton sequence, in theory), so the concept is fundamental & pervasive in
Icon. More fundamental even than success and failure ("failure" is simply
an empty result sequence, and "success" non-empty).
That doesn't fit with Python at all, so I'm just trying to get the most
useful & conventional & well-behaved part: explicit generation when you ask
for it, at the function level. That would add enormous bang for the buck,
and greatly aid clarity when appropriately used. So that much is Pythonic
to the core.
after-a-decade-i'm-still-trying-to-get-guido-to-understand-
what-python-is<wink>-ly y'rs - tim
Moshe Zadka
> That doesn't fit with Python at all, so I'm just trying to get the most
> useful & conventional & well-behaved part: explicit generation when you ask
> for it, at the function level. That would add enormous bang for the buck,
> and greatly aid clarity when appropriately used. So that much is Pythonic
> to the core.
I'm a little ignorant on both the bang and the buck issues. While what
Tim said certainly shows there's a big bang, I'm wondering about the buck.
Modulu Stackless, what would it take to implement this?
--
Moshe Zadka <mza...@geocities.com>.
INTERNET: Learn what you know.
Share what you don't.
Christian Tismer
Andrew Cooke wrote:
>
> In article <000001bf768e$48e40580$45a0143f@tim>,
> "Tim Peters" <tim...@email.msn.com> wrote:
> > def traverse_post(self):
> > for child in self.left, self.right:
> > if child is not None:
> > suspend child.traverse_post()
> > suspend self.data
>
> That finally hammered home to me just what continuations are all about.
> Does anyone have something similarly elegant that shows a coroutine?
> I've just had a look at the stackless python documentation and
> coroutines seem to be described as something coming out of a single
> detailed example. Is there a simpler definition? (I did look back
> through some posts on Deja, but there was nothing recent that seemed to
> explain what a coroutine actually is - sorry if I've missed something
> obvious).
What did you read, actually?
Here some pointers:
Homepage: http://www.tismer.com/research/stackless/
IPC8 paper: http://www.tismer.com/research/stackless/spcpaper.htm
Slides: http://www.tismer.com/research/stackless/spc-sheets.ppt
The latter gives you a bit of understanding what a continuation is.
Tim Peters about coroutines can be found here:
http://www.tismer.com/research/stackless/coroutines.tim.peters.html
More on coroutines by Sam Rushing:
http://www.nightmare.com/~rushing/copython/index.html
On Scheme, continuations, generators and coroutines:
http://www.cs.rice.edu/~dorai/t-y-scheme/
Revised Scheme 5 report:
http://www.swiss.ai.mit.edu/~jaffer/r5rs_toc.html
The following books are also highly recommended:
- Andrew W. Appel, Compiling with Continuations, Cambridge University
Press, 1992
- Daniel P. Friedman, Mitchell Wand, and Christopher T. Haynes,
Essentials of Programming Languages, MIT Press, 1993
- Christopher T. Haynes, Daniel P. Friedman, and Mitchell Wand,
Continuations and Coroutines, Computer Languages, 11(3/4): 143-153,
1986.
- Strachey and Wadsworth, Continuations: A mathematical semantics which
can deal with full jumps. Technical monograph PRG-11, Programming
Research Group, Oxford, 1974.
I don't think this is easy stuff at all, and you can't expect
to find a simple answer by skimming a couple of web pages.
It took me a lot of time to understand a fair part of this,
and this is also a showstopper to get this stuff to be used.
> Also, comp.lang.lisp is currently dissing continuations. Would that be
> because the alternative is to pass the code that will process the node
> into the iterator as a (first class) function? Obviously, in this case,
> yes, but is that true in general (are there examples where continuations
> or coroutines make something possible that really is tricky to do in
> other ways)?
An iterator is quite an easy thing, and it can be implemented
without continuations rather easily. Continuations cover problems
of another order of magnitude. It is due to too simple examples
that everybody thinks that coroutines and generators are the
only target. Continuations can express any kind of control flow,
and they can model iterators, coroutines and generators easily.
The opposite is not true!
I know this isn't enough to convince you, but at the time I have
no chance. I need to build some very good demo applications
which use continuations without exposing them to the user,
and this is admittedly difficult.
ciao - chris
--
Christian Tismer :^) <mailto:tis...@appliedbiometrics.com>
Applied Biometrics GmbH : Have a break! Take a ride on Python's
Düppelstr. 31 : *Starship* http://starship.python.net
12163 Berlin : PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF
we're tired of banana software - shipped green, ripens at home
Vladimir Marangozov
>
> [Tim]
> > ...
> > (RC has excellent locality of reference, especially when trash
> > is being created and recycled at high dynamic rates).
>
> [Vladimir Marangozov]
> > That last one is a good one. So far, nobody has shown any evidence
> > that LOR is good or bad when different objects are involved (except,
> > I remember, some special cases like iterating over ints that happen
> > to be allocated close to each other). Better yet, nobody _can_ show
> > such evidence as far as one stays on top of a virtual memory manager.
>
> Vlad, you're trying to get "deep" on what is really a "shallow" point:
Tim, you're trying to get "shallow" on what is really a "deep" point.
Saying that "RC has excellent LOR", even when trash is recycled at high
rates, is disturbing.
> a "high dynamic rate" is most likely to occur when a program is doing
> flat-out computation, so type homogeneity is likely. Like ints or floats
> or whatever in Python.
Nope, not "whatever". Take a flat-out computation involving a container
(say, a dict with ints) which gets resized repeatedly, and your argument
is flawed. It's likely that the dict will walk through your memory.
(and more precisely, through your virtual memory).
> And those are *not* "on top of a virtual memory manager" in Python,
> but involve cycling & recycling via Python's type-specific internal
> "free lists" (some of which you added, so you should remember this <wink>).
That first one is a good one <wink>.
> So long as the free lists act like LIFOs, you get to reuse the same little
> chunks of memory over and over and over ... again.
Here, you're absolutely right.
> It's LOR in *both* time and space, and of course that's "good":
> it's the difference between page faults and, umm, not page faults <wink>.
And here, you're cheating.
You're saying that the drops of an intense rain falling over the MIT
building will *not* wet Cambridge and Boston, without knowing:
(1) how long the rain will last, (2) the evolution of the rainy cloud and
(3) which parts of Boston/Cambridge are included in the calculus. <wink>
IOW, you're qualifying LOR without considering (1) a time slice, (2) the
nature of the objects (which may be "moving"), and most importantly,
(3) a fixed "working set" of memory pages, which gives the basis for
any further reasoning about LOR.
However, your "educated guess" <wink> is based on assumptions & observations
of popular combos. You're assuming that we have enough memory so that
the working set covers all objects involved in the intense computation
(which is far from being granted, especially for combos with limited memory),
and that these objects are "static" for a long perod of time.
This may well be granted in most of the cases (a PC with 8+ MB of RAM,
without heavy loads, i.e single-user) and for most of the Python scripts
we're running. But note that this doesn't give us *any* evidence about LOR.
And that's why I don't buy your argument. We still don't know anything about
LOR in Python, so drop it from your vocabulary <wink>. It may be bad, it may
be good, it may be acceptable.
>
> > make-that-simply-"RC-is-faster-than-the-alternatives"-<wink>-
> > -and-I'll-take-it-ly y'rs
>
> I believe there's sufficient evidence that state-of-the-art "real gc"
> can be made faster than rc. Python isn't other languages, though, and
> so long as everything in Python is always "boxed", other languages'
> gc experiences aren't especially relevant.
Agreed. With Toby/Neil's recent work, things look promising...
BTW, any comparison of RC with state-of-the-art GC is fine by me.
as-long-as-you-don't-mess-with-LOR-<wink>-ly y'rs
François Pinard
> Saying that "RC has excellent LOR", even when trash is recycled at high
> rates, is disturbing.
Sorry, I'm getting lost in acronyms, as we have a seen a lot on this topic
so far. RC is for reference counting, right? What is LOR for?
> BTW, any comparison of RC with state-of-the-art GC is fine by me.
As for GC, I figured it out :-).
--
François Pinard http://www.iro.umontreal.ca/~pinard
Alex
> What is LOR for?
I just figured that out, myself, I think. My guess is locality of
reference.
Alex.
Michael Hudson
> Vladimir Marangozov <Vladimir....@inrialpes.fr> writes:
>
> > Saying that "RC has excellent LOR", even when trash is recycled at high
> > rates, is disturbing.
>
> Sorry, I'm getting lost in acronyms, as we have a seen a lot on this topic
> so far. RC is for reference counting, right? What is LOR for?
LOR = locality of reference (I'm fairly sure).
Important for caches and things. Though I'd have to say worrying about
cache misses caused by Python's reference counting strikes me as, uh,
not especially productive, though I'd have to admit I'm not really
following every nuance of the discussion (Tim Peters and Vladimir
Marangozov being difficilt to understand in a technical discussion!
Gosh!).
I suppose VM page misses are more important...
Cheers,
MIchael
Aahz Maruch
Alan Daniels <dan...@mindspring.com> wrote:
>
>My question is: How hard would it be to keep the reference counting,
>but add a built-in function, maybe called "gc()", that would walk
>through all the existing objects, find the ones that have been
>orphaned due to cyclical references, and reclaim them? Would something
>like that be enough to make everybody happy? Possibly implemented as a
>patch so that it was optional? It would still be deterministic (in
>that __del__ would always get called right away, when appropriate),
>but would still make sure that no objects were ever left unreclaimed
>indefinitely.
That is precisely what the middle-grounders have been suggesting. It's
still not trivial, especially if you want the garbage collection to work
on memory allocated by extensions.
Aahz Maruch
Tim Peters <tim...@email.msn.com> wrote:
>[and Aahz sez about Python's lack of "functional expression"]
>>
>> but adding generators should make up much of the balance.
>
>Unsure it would help here, at least in the flavor I've been selling them.
Not directly, no, but it would extend the power of Python-style
iteration sufficiently to combat many of the complaints about the lack
of functional expression. It's essentially a completely different path
to many of the same ends, but one that fits into Python. IMO
As a side note, I'm thinking that implicit generator creation is a Bad
Idea. We should force people to do things like
b = BinTree()
g = generator b.traverse()
for i in g():
....
but I'm not at all certain of the syntax. Another option would be to
make "generator" a keyword on the same level as "def", so the "function
call" to a generator always creates a generator frame, and you then need
to call the generator reference (as above) for each value.
(As you've probably guessed, I strongly prefer "generator" to
"iterator", but I won't whine if you or Guido have the opposite
preference.)
William Tanksley
>Vladimir Marangozov <Vladimir....@inrialpes.fr> writes:
>> Saying that "RC has excellent LOR", even when trash is recycled at high
>> rates, is disturbing.
>Sorry, I'm getting lost in acronyms, as we have a seen a lot on this topic
>so far. RC is for reference counting, right? What is LOR for?
"Lord Of the Rings." Tolkien's work was allegorical on many levels. Ring
Counting (or RC), as in the books, helps to determine which objects need
to be destroyed. However, Rings are naturally in cycles, so naive RC
didn't work to actually destroy the Ring.
I'll stop now. Even though I really want to keep going. "Reference
Counting" and "Locality of Reference" (the lack of which destroys the
usefulness of caching).
>> BTW, any comparison of RC with state-of-the-art GC is fine by me.
>As for GC, I figured it out :-).
"Guido's C" conventions? Of course.
>François Pinard http://www.iro.umontreal.ca/~pinard
--
-William "Billy" Tanksley, in hoc signo hack
François Pinard
> I guess I don't see how MI is a different issue [...]
Neither do I. But for a different reason. What means MI? :-)
Remco Gerlich
> Justus Pendleton <jus...@my-deja.com> writes:
>
> > I guess I don't see how MI is a different issue [...]
>
> Neither do I. But for a different reason. What means MI? :-)
"Multiple Inheritance".
In Python a class can be based on several other classes. In some other
languages, classes can only inherit one other class.
--
Remco Gerlich, scar...@pino.selwerd.nl
9:09pm up 84 days, 3:13, 6 users, load average: 0.37, 0.34, 0.29
"This gubblick contains many nonsklarkish English flutzpahs, but the
overall pluggandisp can be glorked from context" (David Moser)
Remco Gerlich
> On 15 Feb 2000 11:50:13 -0500, François Pinard wrote:
> >Vladimir Marangozov <Vladimir....@inrialpes.fr> writes:
>
> >> Saying that "RC has excellent LOR", even when trash is recycled at high
> >> rates, is disturbing.
>
> >Sorry, I'm getting lost in acronyms, as we have a seen a lot on this topic
> >so far. RC is for reference counting, right? What is LOR for?
>
> "Lord Of the Rings." Tolkien's work was allegorical on many levels. Ring
> Counting (or RC), as in the books, helps to determine which objects need
> to be destroyed. However, Rings are naturally in cycles, so naive RC
> didn't work to actually destroy the Ring.
And how does this relate to the fact that Angband players can only use
two rings at a time? ;-)
*hides*
--
Remco Gerlich, scar...@pino.selwerd.nl
Hi! I'm a .sig virus! Join the fun and copy me into yours!
Harald Hanche-Olsen
| That finally hammered home to me just what continuations are all about.
| Does anyone have something similarly elegant that shows a coroutine?
Not me, but let me show you what I learned about it, way back in '73.
I had the pleasure of learning assembly language programming at the
feet of a master -- Ole Johan Dahl of Simula fame -- and of course he
just had to teach us about coroutines and how to do them. It was
amazingly simple and powerful.
I need to tell you a bit about the CDC3300 and the assembly language
used for it (Compass): There were no such modern amenities as a
stack. The usual way to code a subroutine was as follows:
FOO JMP *
...
JMP,I FOO
and you would call FOO with this instruction:
JSR FOO
JSR stood for "Jump, Save Return" and that is what it did: It computed
the return address (PC+1) and saved it in the address field at address
FOO (indicated by the asterisk above), then jumped to FOO+1. To
return, FOO just jumps to its own start address, which by now contains
a jump instruction correctly addressed to get back (except we usually
made the jump indirect, as shown, to save a machine instruction).
If you understand this, we're ready for the coroutine example. All it
takes is three lines of glue, then the coruotines themselves:
FOOBAR JMP FOO
BARFOO JMP BAR
JMP,I FOOBAR
FOO ... (skipping the usual JMP * glue)
...
JSR FOOBAR (call BAR)
...
JSR FOOBAR (call BAR again)
...
BAR ... (skipping the usual JMP * glue)
...
JSR BARFOO (call FOO)
...
JSR BARFOO (call FOO again)
...
To get the whole thing going, just execute, e.g.,
JMP FOO
Whenever FOO wants to invoke BAR, it does so via a JSR to the glue at
FOOBAR. The return address is stored where the address FOO used to
be, and the glue code jumps to BAR. Then BAR does some work, calls
BARFOO, which stores a new return address in the glue where BAR's
address used to be, then jumps to FOOBAR, which jumps to FOO - no,
there is a new address there now, so it jumps *into* FOO, not to the
top. And so control is transferred back and forth, and both FOO and
BAR can be written as if *it* is the main program and it's just
calling on its coroutine as if it were a mere subroutine. Mighty
handy if, as it was in the example task we were given, FOO would need
to consume a stream of data, massage it in some way, and BAR would do
further work on the resulting stream of output data from FOO.
(My code as shown assumes that the pair will go on forever, or until
one of them halts the program. To allow a controlled return, a
trivial amount of extra JMP * / JSR - type glue code will be needed.)
What a delight to be able to post ancient assembly code on c.l.py...
Geriatrically y'rs,
--
* Harald Hanche-Olsen <URL:http://www.math.ntnu.no/~hanche/>
- "There arises from a bad and unapt formation of words
a wonderful obstruction to the mind." - Francis Bacon
Evan Simpson
news:88c7hu$209$1...@nntp5.atl.mindspring.net...
> As a side note, I'm thinking that implicit generator creation is a Bad
> Idea. We should force people to do things like
>
> b = BinTree()
> g = generator b.traverse()
> for i in g():
> ....
Why attach such significance to it? Why not just...
b = BinTree()
for node in b.traverser():
...? While I'm at it, for general edification I present a somewhat
hacked-up version of a coroutine example originally posted long ago by Tim
Peters, and preserved at
http://www.tismer.com/research/stackless/coroutines.tim.peters.html
I changed it to use the 'suspend' syntax, along with a totally imaginary
Coroutines class. Basically, it shows how coroutines can be used like a
pipeline; We feed 'text' in one end, it comes out the other end processed,
and one of the middle bits decides when we're all done.
def getline(text):
'Break text into lines'
for line in string.splitfields(text, '\n'):
suspend line
def disassembler():
'Break lines into characters, add ; at ends'
while 1:
card = co.getline()
for c in card:
suspend c
suspend ';'
def squasher():
'Replace ** with ^ and squash whitespace'
while 1:
ch = co.disassembler()
if ch == '*':
ch2 = co.disassembler()
if ch2 == '*':
ch = '^'
else:
suspend ch
ch = ch2
if ch in ' \t':
while 1:
ch2 = co.disassembler()
if ch2 not in ' \t':
break
suspend ' '
ch = ch2
suspend ch
def assembler():
line = ''
while 1:
ch = co.squasher()
if ch == '\0':
break
if len(line) == 72:
suspend line
line = ''
line = line + ch
line = line + ' ' * (72 - len(line))
suspend line
co.exit()
def putline():
while 1:
print co.assembler()
co = Coroutines(getline, disassembler, squasher, assembler, putline)
co.setup.getline(text)
co.putline()
Cheers,
Evan @ digicool
Aahz Maruch
Evan Simpson <ev...@4-am.com> wrote:
>Aahz Maruch <aa...@netcom.com> wrote in message
>news:88c7hu$209$1...@nntp5.atl.mindspring.net...
>>
>> As a side note, I'm thinking that implicit generator creation is a Bad
>> Idea. We should force people to do things like
>>
>> b = BinTree()
>> g = generator b.traverse()
>> for i in g():
>> ....
>
>Why attach such significance to it? Why not just...
>
>b = BinTree()
>for node in b.traverser():
Because now you either have the for construct implicitly create a new
generator frame or you cannot do
b = BinTree()
for node in b.traverser():
for node in b.traverser():
I would much rather create the generator frames explicitly as in the
following (I'm now assuming my later idea that a call to a generator
creates a generator frame that can be used like a function)
b = BinTree()
g1 = b.traverser()
for node in g1():
g2 = b.traverser()
for node in g2():
It just seems more Pythonic to me.
Tres Seaver
Tim Peters <tim...@email.msn.com> wrote:
>[Tim]
>> ...
>> More formally, you can certainly *model* Python's scoping in Scheme,
>
>[Bijan Parsia]
>> Er..Just out of curiosity, what *can't* you model in Scheme? :)
>
>Smalltalk <wink>.
I'd be willing to venture some of Forth's wierder stuff, too.
>
>> And-not-in-the-trivial-turing-machine-sense-ly y'rs,
>> Bijan Parsia
>>
>> P.S. Hey! Where's my wallet?
>
>at-the-door-just-waiting-for-you-to-find-the-exit-ly y'rs - tim
This-way-to-the-egress-ly,
Tres.
--
---------------------------------------------------------------
Tres Seaver tse...@palladion.com 713-523-6582
Palladion Software http://www.palladion.com
Evan Simpson
> In article <sajg8pv...@corp.supernews.com>, Evan Simpson
<ev...@4-am.com> wrote:
> >Why attach such significance to it? Why not just...
> >
> >b = BinTree()
> >for node in b.traverser():
>
> Because now you either have the for construct implicitly create a new
> generator frame or you cannot do
>
> b = BinTree()
> for node in b.traverser():
> for node in b.traverser():
I think I'm missing something, probably something simple. Let me lay out my
implicit assumptions about how I think this would work; Stop me when I run
off the tracks, please :-)
BinTree is a normal class, which produces ordinary instances which implement
a binary tree. BinTree has a method 'traverser', which returns an object.
This object steps along the binary tree and returns a node each time its
__getitem__ is called, then raises an exception when it runs out of tree.
Instead of maintaining some kind of explicit state in the object, it simply
uses continuations to implement __getitem__ in a clear, natural fashion.
The only data in the object is the continuation for __getitem__ and a
reference to the BinTree instance.
Calling b.traverser() twice, as above, just creates two traversal objects
which independently walk the tree.
> I would much rather create the generator frames explicitly as in the
> following (I'm now assuming my later idea that a call to a generator
> creates a generator frame that can be used like a function)
>
> b = BinTree()
> g1 = b.traverser()
> for node in g1():
> g2 = b.traverser()
> for node in g2():
So g1 and g2 are generators, and g1() and g2() are 'generator frames', yes?
So what is a 'generator frame'? Perhaps that is what I'm not getting.
Cheers,
Evan @ 4-am
Dan Schmidt
| + Andrew Cooke <and...@andrewcooke.free-online.co.uk>:
|
| | That finally hammered home to me just what continuations are all about.
| | Does anyone have something similarly elegant that shows a coroutine?
|
| Not me, but let me show you what I learned about it, way back in '73.
| I had the pleasure of learning assembly language programming at the
| feet of a master -- Ole Johan Dahl of Simula fame -- and of course he
| just had to teach us about coroutines and how to do them. It was
| amazingly simple and powerful.
|
| [explanation snipped]
This is pretty much the same way that Knuth explains coroutines in
volume 1 of The Art of Computer Programming -- which you all have,
right? -- in section 1.4.2.
--
Dan Schmidt | http://www.dfan.org
Darrell
>
> I agree, but it's gotten "better" this way, mostly via little things like
> the addition of dict.get() and 3-argument getattr -- not everything is
> better spelled on 4 lines <wink>.
Is there a 3-argument getattr in the CVS version ?
--Darrell
Steve Holden
>
> [summarises GC position]
>
> My question is: How hard would it be to keep the reference counting,
> but add a built-in function, maybe called "gc()", that would walk
> through all the existing objects, find the ones that have been
> orphaned due to cyclical references, and reclaim them? Would something
> like that be enough to make everybody happy? Possibly implemented as a
> patch so that it was optional? It would still be deterministic (in
> that __del__ would always get called right away, when appropriate),
> but would still make sure that no objects were ever left unreclaimed
> indefinitely.
>
>
I seem to remember this was essentially the scheme used by Robert Dewar
in the Spitbol implementation of Snobol4 (I know, I'm showing my age: I
ported from the 1900 series to the DECSystem-10 as my final-year
undergraduate product). He did this precisely to be able to reclaim
memory from unreferenced cyclic structures.
Spitbol also allowed explicit calls to the GC, which would return
the amount of memort reclaimed, presumably for want of a more sensible
result.
However, this method may be slower than the existing collector.
regards
Steve
regards
Steve
--
"If computing ever stops being fun, I'll stop doing it"
Neel Krishnaswami
> [Tim]
> > ...
> > More formally, you can certainly *model* Python's scoping in Scheme,
>
> [Bijan Parsia]
> > Er..Just out of curiosity, what *can't* you model in Scheme? :)
>
> Smalltalk <wink>.
I know this is a joke, but wasn't Scheme invented because Sussman
wanted to understand how Smalltalk worked by implementing something
like it in MacLisp? I thought that the Smalltalk and Scheme designers
were in regular communication.
Neel
Michael Hudson
Well, yes, but it's been there since 1998/06/29 13:38:57 according to
`cvs log', i.e. before the release of 1.5.2 (about a month before
1.5.2a2, if I'm reading the log right).
Cheers,
M.
Fredrik Lundh
> > I agree, but it's gotten "better" this way, mostly via little things
like
> > the addition of dict.get() and 3-argument getattr -- not everything is
> > better spelled on 4 lines <wink>.
>
> Is there a 3-argument getattr in the CVS version ?
yes, unless someone has made a mistake
and removed it:
> python
Python 1.5.2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> print getattr.__doc__
getattr(object, name[, default]) -> value
Get a named attribute from an object; getattr(x, 'y') is equivalent to x.y.
When a default argument is given, it is returned when the attribute doesn't
exist; without it, an exception is raised in that case.
</F>
Andrew Cooke
Christian Tismer <tis...@tismer.com> wrote:
> Andrew Cooke wrote:
[...]
> > through some posts on Deja, but there was nothing recent that seemed
to
> > explain what a coroutine actually is - sorry if I've missed
something
> > obvious).
>
> What did you read, actually?
> Here some pointers:
> Homepage: http://www.tismer.com/research/stackless/
> IPC8 paper: http://www.tismer.com/research/stackless/spcpaper.htm
I read this. If you read it yourself, trying to remember that once you
did not know what coroutines were, I think you'll see that the text
doesn't contain anything like "A coroutine is...". Coroutines aren't
even mentioned in the contents.
In turns out (after reading Tim P's code in another pointer you gave)
that all I really wanted to see was something like "coroutines allow
continuations to be used in parallel" or something similar (assuming
I've got it right!).
> Slides: http://www.tismer.com/research/stackless/spc-sheets.ppt
> The latter gives you a bit of understanding what a continuation is.
I'm afraid these are in some strange format.
Thanks for the info - I'm sorry to waste your time but I do think that
you're so deep into this you may not have realised that
- coroutines are not explicitly defined, as far as I could see.
- noddy examples, while they may give people the wrong impression about
the power of the system, do let newbies understand what the basics are
(I am sure coroutines are not the only concept in computing that is more
powerful than a noddy example - I seem to be able to use those other
concepts in deep ways despite being exposed to equivalent examples in my
youth).
> > Also, comp.lang.lisp is currently dissing continuations. Would that
be
> > because the alternative is to pass the code that will process the
node
> > into the iterator as a (first class) function? Obviously, in this
case,
> > yes, but is that true in general (are there examples where
continuations
> > or coroutines make something possible that really is tricky to do in
> > other ways)?
Answering my own question here - there is an obvious advantage in the
iterator approach when iterating over several data structures in
parallel. That's a trivial (now) counterexample to my suggestion (it
struck me at the time that passing in a function or returning with from
a coroutine were like mirror images of each other - parallel data shows
how this breaks down).
> An iterator is quite an easy thing, and it can be implemented
> without continuations rather easily. Continuations cover problems
> of another order of magnitude. It is due to too simple examples
> that everybody thinks that coroutines and generators are the
> only target. Continuations can express any kind of control flow,
> and they can model iterators, coroutines and generators easily.
> The opposite is not true!
>
> I know this isn't enough to convince you, but at the time I have
> no chance. I need to build some very good demo applications
> which use continuations without exposing them to the user,
> and this is admittedly difficult.
Maybe my post gave the impression that I was doubtful or needed
convincing. I don't think either is true - I was simply curious.
Thanks for all the info,
Andrew
PS Another point in favour of noddy examples - Tim's iterator finally
made clear to me how a Prolog implementation worked in Lisp. Hardly a
case of dumbing down....
Sent via Deja.com http://www.deja.com/
Before you buy.
William Tanksley
>>> Er..Just out of curiosity, what *can't* you model in Scheme? :)
>>Smalltalk <wink>.
>I'd be willing to venture some of Forth's wierder stuff, too.
Hmm... Well, parsing Forth is certainly easier than parsing Scheme, but
since is _is_ possible to write a Scheme program which parses its own
source (a reader), I'd say that you can imitate the odd tricks Forth can do.
I still prefer Forth, of course. Parentheses are made for comments --
that, after all, is the original Greek meaning of the word.
>Tres.
Cliff Crawford
|
| [Bijan Parsia]
| > Er..Just out of curiosity, what *can't* you model in Scheme? :)
|
| Smalltalk <wink>.
I once wrote a Smalltalk interpreter in Logo. <g>
--
cliff crawford -><- http://www.people.cornell.edu/pages/cjc26/
I think, therefore I'm single.
Tim Peters
> Er..Just out of curiosity, what *can't* you model in Scheme? :)
[Tim]
> Smalltalk <wink>.
[Neel Krishnaswami]
> I know this is a joke,
Shhh! Don't spoil the fun for Bijan <wink>.
> but wasn't Scheme invented because Sussman wanted to understand
> how Smalltalk worked by implementing something like it in MacLisp?
> I thought that the Smalltalk and Scheme designers were in regular
> communication.
I had heard the latter before too, but really don't know. Sure sounds
plausible, even likely. Language designers are generally quite friendly
with each other! In other lines of business, this would be called
price-fixing <wink>.
"hey-larry-you-put-in-array-context-and-i'll-use-whitespace-for-
block-structure"-ly y'rs - tim, possibly quoting 1990 email
Tim Peters
> Er..Just out of curiosity, what *can't* you model in Scheme? :)
[Tim]
> Smalltalk <wink>.
[Tres Seaver]
> I'd be willing to venture some of Forth's wierder stuff, too.
'Twas but a joke, Tres (Bijan is a Smalltalk fan, and had a Perl fan asked
instead I would have answered Ruby <wink>). Scheme is exceptionally good at
modeling unusual semantics, which is why, e.g., GNU picked Scheme as the
basis of its GUILE project.
a-joke-explained-is-always-funnier<wink>-ly y'rs - tim
Tim Peters
> ...
> Because now you either have the for construct implicitly create a new
> generator frame or you cannot do
>
> b = BinTree()
> for node in b.traverser():
> for node in b.traverser():
>
> following (I'm now assuming my later idea that a call to a generator
> creates a generator frame that can be used like a function)
>
> b = BinTree()
> g1 = b.traverser()
> for node in g1():
> g2 = b.traverser()
> for node in g2():
>
It's certainly Pythonic to expose the machinery, but also Pythonic to
provide some sugar for the most common cases. This need not be an either/or
argument.
Python currently hides the generation of the "loop index" that supports the
__getitem__ protocol, and that it *didn't* expose the machinery there leads
to clumsiness like the common (& excessively novel):
for i in range(len(sequence)):
idiom. The bulk of for loops don't need the loop index, though, and we're
all delighted to use the simpler
for thing in sequence:
when we can (and that still generates the integers in range(len(sequence)),
but all under the covers). I expect generators would be very similar in
practice.
less-typing-or-more-power-is-a-nice-choice-to-offer-ly y'rs - tim
Justus Pendleton
=?ISO-8859-1?Q?Fran=E7ois_Pinard?= <pin...@iro.umontreal.ca> wrote:
> Justus Pendleton <jus...@my-deja.com> writes:
>
> > I guess I don't see how MI is a different issue [...]
>
> Neither do I. But for a different reason. What means MI? :-)
MI = Multiple Inheritance, i.e.
class Foo (Bar, Splat):
pass
instead of just
class Foo (Bar):
pass
Tim Peters
> def traverse_post(self):
> for child in self.left, self.right:
> if child is not None:
> suspend child.traverse_post()
> suspend self.data
[Andrew Cooke]
> That finally hammered home to me just what continuations are all
> about.
What's illustrated above is "a generator", which is a semi-coroutine, or,
literally, just half of a coroutine. continuations are one possible means
of implementing that, but are far more general.
> Does anyone have something similarly elegant that shows a coroutine?
cmdline pipes (like "cat somefile | sort | uniq -c | sort -nr") are usually
implemented via separate processes communicating via files, but are easy to
write in a language with coroutines within a single process (and within a
single thread!).
This really isn't a bad <wink> analogy: no command in the pipe is either
master or slave -- they're all equal partners. Each has its own state,
including its own "program counter", call stack, and local variables. They
don't "call" each other, they *communicate info* to each other. Under a
normal call, the callee *loses* its state as soon as it returns. Under a
coroutine transfer, it's completely symmetric: the transferor asks the
transferee to run for a while (either passing info to it, or desiring info
from it), and the transferee doesn't *return*, but merely transfers control
to some other coroutine. Nobody loses state. And each transfer to a
coroutine C resumes C in exactly the state it was in at the time C
transfered to some other coroutine. There's only one "thread" of control,
but control is not constrained to "stack like" resumptions: any coroutine
can transfer to any other at any time (btw, the pipeline example doesn't
illustrate that particular point).
Knuth properly complains that it's very hard to give a simple example of
general coroutines. He ends up writing a pretty hairy elevator simulation
to illustrate it. In fact, coroutines most often pop up in languages
designed for writing event simulations (like Simula-67, which is a wonderful
early one that also anticipated much of modern OO machinery!).
> ...
> Also, comp.lang.lisp is currently dissing continuations.
That's because some Lisp people are fighting with some Scheme people. It's
their version of using whitespace for indentation <wink>.
> Would that be because the alternative is to pass the code that
> will process the node into the iterator as a (first class) function?
> Obviously, in this case, yes, but is that true in general (are
> there examples where continuations or coroutines make something
> possible that really is tricky to do in other ways)?
This gets really involved really fast, and without a precise model of
computation to build on, discussing relative power is a trap. *Generally*
speaking, coroutines can be implemented via continuations, but not vice
versa (continuations are more powerful); but continuations and threads can
each be implemented via the other (but have vastly different pragmatics,
which is why Christian worked so hard on Stackless); so if threads are more
familiar, rephrase your question to ask whether having threads can make some
things much easier to do. "Of course", and the same for continuations. Raw
threads and raw continuations are exceedingly difficult to use correctly,
though, so Python offers the higher-level threading.py to make thread life
easier, and continuations in Scheme are usually used "under the covers" to
implement an easier-to-use abstraction (like coroutines, or backtracking
search, or ...).
that's-the-ten-cent-answer-and-worth-almost-half-ly y'rs - tim
Tim Peters
> I'm a little ignorant on both the bang and the buck issues. While
> what Tim said certainly shows there's a big bang, I'm wondering
> about the buck. Modulu Stackless, what would it take to implement
> this?
Discussions on this stretched over many months last year, beginning in about
May. Guido developed a nearly complete implementation plan for generators
in the "Generator details" thread of July '99, in the Python-Dev archives.
And Steven Majewski pretty much actually implemented them six years ago,
then lost interest when it couldn't be extended to continuations (the
implementation of which, as Christian proved, required redefining truth
itself <wink>).
generators-coexist-peacefully-with-all-truths-ly y'rs - tim
Tim Peters
[Tim]
>> Vlad, you're trying to get "deep" on what is really a "shallow" point:
[Vladimir Marangozov]
> Tim, you're trying to get "shallow" on what is really a "deep"
> point.
I suspect it's really a bit of both! This started as a very brief Usenet
posting, and I don't feel bad about pointing people to LOR (locality of
reference) as a way to get a first handle on why RC (reference counting) has
run faster in Python than every "real GC" attempt to replace it. To a first
approximation, LOR is the single most useful concept to get across to people
struggling with memory issues on VM (virtual memory) systems. To a second
approximation, "cache effects" is more accurate, but also much harder to get
a grip on. To a third approximation, "platform quirks" would have covered
my ass completely, yet left everyone without a clue.
So I'm definitely indulging in idealized fiction, that happens to be true in
some actual cases. My track record of *predicting* how various gc
approaches would turn out in Python ain't half bad, you know <wink>. OTOH,
your saying "it ain't necessarily so!" in six ways is six times true, but
doesn't leave people with even a sixth of a first approximation to trip
over. If they're interested enough, they'll pursue it and fill in the
painful details themselves.
> Saying that "RC has excellent LOR", even when trash is recycled at
> high rates, is disturbing.
Take a vacation <wink>.
> ...
> Nope, not "whatever". Take a flat-out computation involving a
> container (say, a dict with ints) which gets resized repeatedly,
> and your argument is flawed.
Not so much flawed here as ridiculous. So what? I haven't claimed to give
a rigorous account, and one can't even be attempted without a sufficiently
precise model of a specific platform (both HW and system SW). There's
nothing to argue about here short of that: take it as it was intended, a
comforting brief bedtime story with elements of truth.
BTW, I expect you do undervalue the usefulness of LOR arguments "in
practice, in general". Their utility is based on the simplicities that are
presumed usual rather than the pathologies that are presumed possible. The
real world often enough plays along to make it worth the minimal effort.
> ...
> You're saying that the drops of an intense rain falling over the MIT
> building will *not* wet Cambridge and Boston, without knowing:
> (1) how long the rain will last,
Forever, for practical purposes.
(2) the evolution of the rainy cloud and
Long periods of steady states, with brief periods of unpredictable change.
> (3) which parts of Boston/Cambridge are included in the calculus.
> <wink>
MIT people are quite certain Boston is a fiction <wink>.
> ...
> And that's why I don't buy your argument.
That's OK by me: you're still trying to make it too deep <wink>.
> We still don't know anything about LOR in Python, so drop it from
> your vocabulary <wink>. It may be bad, it may be good, it may be
> acceptable.
I know about cache effects on Pentiums, because I've watched them under
Intel's analysis tools. LOR looked like a darned good bet to me. This is
time-consuming & tedious work, though, and I only looked at a few of my own
programs. They happened to fit all the assumptions I made, so perhaps it's
not surprising that my conclusions appeared to fit <wink>.
>> I believe there's sufficient evidence that state-of-the-art "real gc"
>> can be made faster than rc. Python isn't other languages, though, and
>> so long as everything in Python is always "boxed", other languages'
>> gc experiences aren't especially relevant.
> Agreed. With Toby/Neil's recent work, things look promising...
> BTW, any comparison of RC with state-of-the-art GC is fine by me.
>
> as-long-as-you-don't-mess-with-LOR-<wink>-ly y'rs
ok-ok-it's-due-to-the-"cacheothonic-principle"-ly y'rs - tim
Christian Tismer
Andrew Cooke wrote:
[about coroutines]
> > What did you read, actually?
> > Here some pointers:
> > Homepage: http://www.tismer.com/research/stackless/
> > IPC8 paper: http://www.tismer.com/research/stackless/spcpaper.htm
>
> I read this. If you read it yourself, trying to remember that once you
> did not know what coroutines were, I think you'll see that the text
> doesn't contain anything like "A coroutine is...". Coroutines aren't
> even mentioned in the contents.
Oh I am sorry. You are right.
Yes I didn't define coroutines and generators.
These are well understood concepts, and the audience was supposed
to have an idea about that. The real problem appeared to be
continuations, and I tried to show that they are even much
simpler, and explained them by building a generator from them.
Sorry, I mixed words, and I admit that I'm much too deep
in this stuff to see other than my problems :-)
Time to write a very introductional text, I think.
cheers - chris
--
Christian Tismer :^) <mailto:tis...@appliedbiometrics.com>
Applied Biometrics GmbH : Have a break! Take a ride on Python's
Düppelstr. 31 : *Starship* http://starship.python.net
12163 Berlin : PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF
we're tired of banana software - shipped green, ripens at home
Ivan Van Laningham
Robin Becker wrote:
>
> In article <oqputya...@titan.progiciels-bpi.ca>, François Pinard
> <pin...@iro.umontreal.ca> writes
> >ne...@brick.cswv.com (Neel Krishnaswami) writes:
> >
> >> I thought that the Smalltalk and Scheme designers were in regular
> >> communication.
> >
> >Great minds and good people stay in regular communication. Aren't we all
> >often reading (and writing to) this mailing list? :-) :-) Keep happy!
> >
> oooooooommmmmmhhhhh oooooohhhhhmmmmmmm, ooooommmmmhhhhhh mani padme
> oooooohhhhmmmmmm :)
^^^^^^^^^^^^^^^^
s/oooooohhhhmmmmmm/huummmmmmmmmmmm/g
<and-damn-glad-to-have-my-email-back-after-three-days-i-nearly-died>-ly
y'rs,
Ivan
----------------------------------------------
Ivan Van Laningham
Callware Technologies, Inc.
iva...@callware.com
http://www.pauahtun.org
http://www.foretec.com/python/workshops/1998-11/proceedings.html
Army Signal Corps: Cu Chi, Class of '70
Author: Teach Yourself Python in 24 Hours
Peter Sommerfeld
>> Er..Just out of curiosity, what *can't* you model in Scheme? :)
>
>[Tim]
>> Smalltalk <wink>.
>
>> I know this is a joke,
>
>Shhh! Don't spoil the fun for Bijan <wink>.
>
>> but wasn't Scheme invented because Sussman wanted to understand
>> how Smalltalk worked by implementing something like it in MacLisp?
>> communication.
Scheme was originally invented when Steel & Sussmann made an experimental
Lisp implementation of Carl Hewitt's Actor model which has been has been
influenced by Smalltalk. [G.L. Steele: The Evolution of Lisp]. Actors are
objects which communicate by messages but without an explicit return path.
The continuation is determined by the receiving actor only.
-- Peter
François Pinard
> =?ISO-8859-1?Q?Fran=E7ois_Pinard?= <pin...@iro.umontreal.ca> writes:
> > Couldn't we approximate lexical scoping with classes? [...]
> It would also be nice if people got the terminology right [...]
Sorry, I gave in the current terminology mixup. I wanted to say that
my modest needs for Scheme lexical closures were easy to satisfy using
approximations based on Python classes. :-)
--
François Pinard http://www.iro.umontreal.ca/~pinard
François Pinard
> I suspect that, like me, you're not doing any GUI programming in
> Python yet.
But I much leer in that direction, should I confess. :-)
> The press for fancier lambdas and lexical closures is much stronger from
> people who do [...]
I've begun to see that lambda's are an attractive pole, indeed. Yet,
lambdas returning None, and called only for side-effects, are against some
good taste. I wonder which part of me will win over the other, but there
is some discomfort. Python would help me much better at picking the best
solution, if only it was not offering lambdas :-).
As for lexical closures, you probably mean packaging widgets into classes?
This looks sterling to me! But given you add:
> they want to attach small & simple procedures to oodles & oodles of
> widgets (your "common and frequent" indeed), and out-of-line class
> solutions are legitimately viewed as clumsier and more obscure.
I have the impression I did not understand what you meant by lexical
closures.
Vladimir Marangozov
elaborated garbage collection (GC) algorithms on locality of reference
(LOR).]
[Tim]
>
> [Vladmir & Tim have apparently lost everyone, so let's wrap it up]
You're right. Let's bring the volunteers on board!
>
> [Tim]
> >> Vlad, you're trying to get "deep" on what is really a "shallow" point:
>
> [Vladimir Marangozov]
> > Tim, you're trying to get "shallow" on what is really a "deep" point.
>
> I suspect it's really a bit of both!
And I have to admit that "deep" and "shallow" are perfectly balanced <wink>.
> This started as a very brief Usenet
> posting, and I don't feel bad about pointing people to LOR (locality of
> reference) as a way to get a first handle on why RC (reference counting) has
> run faster in Python than every "real GC" attempt to replace it.
Well, I for one don't feel good with misleading arguments <wink>.
By mentioning LOR in this highly valuable thread, you're suddenly getting
"deep" <0.9 wink>. Python was not implemented with LOR in mind, though some
of the objects have been subsequently tuned to be slightly "LOR aware".
I would have exposed lighter arguments for a first kiss between RC & GC:
"RC is fast enough and handles successfully 95% of the cases; it was easy
to implement and the GC implementations available 10 years ago, when Python
has seen the light for the first time, were unappropriate for a number of
reasons. See the FAQ for a discussion on some of them".
Moreover, we've experienced that RC is error prone and obviously it cannot
handle cyclic trash, so it's desirable to include some form of more
sophisticated garbage collection, adapted to Python's peculiarities.
However, this is a very subtle task, because we cannot get rid of RC
"just like that". It's something too fundamental in Python that penetrates
its most inner internals. Worse, a number of implementation strategies
have been adopted, specifically because we had RC. (I'll not mention them
here to avoid technical details)
Well, someone could try to get rid of RC by redefining Py_INCREF/Py_DECREF
to empty macros <wink>, but she'll see a fat core dump pretty soon, which
is not so attractive as a solution. And if we plug a "conventional" garbage
collector and disable RC, it doesn't perform well because of this chronic
Python dependency on the RC "drug". Period.
That's why, after several long debates here and there, some brave people
that I won't name (they are too precious to be exposed in public <wink>)
have had the idea of implementing a garbage collector "on top of" RC, thus
preserving Python's strong dependency on RC. At that time, I think that Tim
was with mixed feelings (check DejaNews for details), but he encouraged the
pioneers to implement their ideas, which proves BTW that he was not
one of them <wink>. Nor was I.
This is a complete, gentle and pretty fair introduction to the whole
story about RC and GC in Python, without any big words like LOR.
> To a first approximation, LOR is the single most useful concept to get
> across to people struggling with memory issues on VM (virtual memory)
> systems. To a second approximation, "cache effects" is more accurate,
> but also much harder to get a grip on. To a third approximation,
> "platform quirks" would have covered my ass completely, yet left
> everyone without a clue.
I like this much better (except that VM stands for Vladimir Marangozov
in the first place <wink>). Not sure that by saying it you've attracted
many passengers on board, though.
>
> So I'm definitely indulging in idealized fiction, that happens to be true
> in some actual cases. My track record of *predicting* how various gc
> approaches would turn out in Python ain't half bad, you know <wink>.
Only because you're half lucky when you predict wrong <wink>.
> OTOH, your saying "it ain't necessarily so!" in six ways is six
> times true, but doesn't leave people with even a sixth of a first
> approximation to trip over.
Point taken (see above), but let me reiterate that explicitly:
Don't ever think about LOR if you're a normal Python user and/or developer.
LOR is a blob that you'd eventually want to consider only for *tuning*,
not designing, a memory manager for Python. It's such a crap that if you
want some truth, you'll have to poke in kernel mode the virtual memory
manager (VMM) of your machine. If you can't poke your VMM, all bets are off.
In other words, you simply don't want to do that.
> If they're interested enough, they'll pursue it and fill in the
> painful details themselves.
Fair enough.
>>
>> And that's why I don't buy your argument.
>
> That's OK by me: you're still trying to make it too deep <wink>.
Oh, I can make it even deeper <0.5 wink>, but as far as we agreed that
it's not that shallow, I'll try to take a short vacation in some
indentation thread <wink>.
Because, let me tell you one thing -- white space affects directly the
LOR of the parser, which infects the LOR of the compiler, which infects
the builtins and the whole Python world.
I have formal proofs that any change of the indentation rules results
in 35% increase of the page faults for only 63.7% of the cache misses.
The net effect is an overall slowdown of 10%. What's the conclusion?
The conclusion is that we shouldn't bother changing the indentation
rules, because a simple printf inserted by Marc-Andre in the beginning
of eval_code2 is enough to slow down the interpreter by 5%, (Tim, do you
remember that experiment? -- only cache misses and very bad LOR <wink>).
Therefore a second printf at the end of eval_code2 should be sufficient
to achieve the same net effect of 10%.
just-an-educated-guess-with-a-conclusion-<wink>-ly y'rs
--
Vladimir MARANGOZOV | Vladimir....@inrialpes.fr
http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252
William Tanksley
>Justus Pendleton <jus...@my-deja.com> wrote:
>> ne...@alum.mit.edu wrote:
>> > > > 3. Multiple inheritance is unsound
>> > > > By 'unsound' I mean that a method call to a method inherited by
>> > > > a subclass of two other classes can fail, even if that method
>> > > > call would work on an instance of the base class.
>So I looked at some other dynamic languages to see what they did.
>o Dylan -- forbids using MI when there are multiple slots with the
> same name.
>Other languages seem to either forbid name conflicts or just suck it
>up and live with an unsafe rule, which leads me to conclude that the
>moral is TANSTAAFL -- there ain't no such thing as a free lunch.
Sather uses yet another system, which I see as very interesting -- it
totally disallows inheritance of every kind (single AND multiple). In its
place it offers two concepts: interface and importation.
Let me give a pseudo-Python example:
# the base class
class First:
def fun(hmm):
print hmm
# this is how to exactly imitate Python's current
# behavior (assuming no extra error checks occur)
class Second(First):
import First
# here we 'subclass' from First, but we don't inherit _any_
# behavior or data.
class Third(First):
def fun(yadda):
print "Whee!", yadda
# Here the compiler will print an error message, because we don't define
# any implementation of the interface we claim.
class Fourth(First):
def boring():
print "braces are cool, indentation sucks!"
This is only a single-inheritance example, of course. The rest is left as
an excercise to the reader.
>Neel
William Tanksley
>François Pinard <pin...@iro.umontreal.ca> writes:
>> very much like the current implications of reference counting. When I write:
>> for line in open(FILE).readlines():
>> I rely on the fact FILE will automatically get closed, and very soon since
>Indeed. I think Tim's right here (as always:); if the request for
>garbage collection is rephrased as 'reclaim cycles', everybody will be
>happy.
I'd like that, but I also like the idea of not having to track memory
usage when writing C extensions. I think Ruby's done a good job of
showing what's possible here.
>Martin
Paul Prescod
>
> ...
> May. Guido developed a nearly complete implementation plan for generators
> in the "Generator details" thread of July '99, in the Python-Dev archives.
One thing to consider is what of this can be easily added to JPython.
I'm a little leery about continuations because JPython uses the Java
stack, doesn't it? It does have a concept of a Python frame but the
"instruction pointer" is implicit in the JVM, not explicit in an
interpreter loop.
--
Paul Prescod - ISOGEN Consulting Engineer speaking for himself
"The calculus and the rich body of mathematical analysis to which it
gave rise made modern science possible, but it was the algorithm that
made possible the modern world."
- from "Advent of the Algorithm" David Berlinski
http://www.opengroup.com/mabooks/015/0151003386.shtml
Tim Peters
> Discussions on this stretched over many months last year,
> beginning in about May. Guido developed a nearly complete
> implementation plan for generators in the "Generator details"
> thread of July '99, in the Python-Dev archives.
[Paul Prescod]
> One thing to consider is what of this can be easily added to
> JPython. I'm a little leery about continuations because JPython
> uses the Java stack, doesn't it? It does have a concept of a
> Python frame but the "instruction pointer" is implicit in
> the JVM, not explicit in an interpreter loop.
+ All attempts I'm aware of to implement Scheme on a JVM gave up on full
continuations (anyone know of an exception?).
+ Generators can be implemented via much more conventional means than can
continuations (continuations are merely *an* implementation technique for
generators; generators can be implemented in many ways -- they're more of a
variant of "regular calls", and, in particular, generator control flow is
stack-like).
+ JPython is to CPython as Jcon is to Icon. Icon (ditto Jcon) doesn't have
continuations, but makes extremely heavy use of generators and light use of
coroutines. The Jcon port implements both. A good tech report on the Jcon
implementation can be gotten from a link near the bottom of:
http://www.cs.arizona.edu/icon/jcon/index.htm
Coroutines in Jcon are implemented on top of Java threads (my own ancient
implementation of coroutines via raw Python threads has been referenced
enough here recently <wink>). Generators are implemented directly in Jcon,
but do seem to be more of a pain than in Icon's C implementation.
not-especially-inclined-to-let-the-limitations-of-the-jvm-
limit-languages-other-than-java-ly y'rs - tim
Moshe Zadka
> As for lexical closures, you probably mean packaging widgets into classes?
> This looks sterling to me! But given you add:
>
> > they want to attach small & simple procedures to oodles & oodles of
> > widgets (your "common and frequent" indeed), and out-of-line class
> > solutions are legitimately viewed as clumsier and more obscure.
>
> I have the impression I did not understand what you meant by lexical
> closures.
Oh, but Francois, you did. I did some (though not as much as I'd like to)
Python/Tkinter programming (I even managed to get people to pay me for
that), and I strongly disagree with Tim. In Python GUIs, I learned from
Guido's IDLE, we make everything a class, and then we make the callbacks
methods. That way, they have the "context" of the class, in which to set
arguments. This is closer to the way you do it in C (passing in self
instead of some void * pointing to a hairy structure) then the way you do
it in Scheme (doing the same thing implicitly: the equivalent to "self"
would be a scheme-ish environment, if I remember my Scheme classes
correctly).
Hence, my personal opinion on lambdas: kill them. With knives. They merely
confuse Python GUI programmers.
and-i-do-hope-esr-won't-manage-to-shoot-at-me-across-the-ocean-ly y'rs, Z.
--
Moshe Zadka <mza...@geocities.com>.
INTERNET: Learn what you know.
Share what you don't.
Aahz Maruch
Evan Simpson <ev...@4-am.com> wrote:
>
>BinTree is a normal class, which produces ordinary instances which
>implement a binary tree. BinTree has a method 'traverser', which
>returns an object. This object steps along the binary tree and
>returns a node each time its __getitem__ is called, then raises an
>exception when it runs out of tree. Instead of maintaining some kind
>of explicit state in the object, it simply uses continuations to
>implement __getitem__ in a clear, natural fashion. The only data in
>the object is the continuation for __getitem__ and a reference to the
>BinTree instance.
>
>Calling b.traverser() twice, as above, just creates two traversal
>objects which independently walk the tree.
Hmmmm.... On second thought, I guess what you're suggesting would work,
as long as BinTree.traverser() returns a generator object (instead of
returning a list) and the for construct understands how to use generator
objects.
(I was still stuck in my original idea when I disagreed with you, where
I was thinking that BinTree.traverser() was an ordinary method that
required a keyword to create a generator object. I now think that's a
Bad Idea and that BinTree.traverser() should be unusable as anything
other than the creator of generator objects, which requires a keyword
when defining BinTree.traverser().)
>Aahz Maruch <aa...@netcom.com> wrote in message
>news:88cg9r$10t$1...@nntp9.atl.mindspring.net...
>>
>> I would much rather create the generator frames explicitly as in the
>> following (I'm now assuming my later idea that a call to a generator
>> creates a generator frame that can be used like a function)
>>
>> b = BinTree()
>> g1 = b.traverser()
>> for node in g1():
>> g2 = b.traverser()
>> for node in g2():
>
>So g1 and g2 are generators, and g1() and g2() are 'generator frames', yes?
Not quite. g1 and g2 are generator objects (aka "generator frames");
g1() and g2() are calls to the generator that either return a value or
raise an exception.
>So what is a 'generator frame'? Perhaps that is what I'm not getting.
"Generator frame" is functionally equivalent to "generator object"; I
prefer to use "frame" in analogy with "stack frame" to make clear that
the object has a local namespace and program counter. If nothing else,
generators will guarantee the demise of Python's "two namespace" rule.
(Sort of. Each frame will still maintain the two namespace rule.)
--
--- Aahz (Copyright 2000 by aa...@netcom.com)
Androgynous poly kinky vanilla queer het <*> http://www.rahul.net/aahz/
Hugs and backrubs -- I break Rule 6
Our society has become so fractured that the pendulum is swinging
several different directions at the same time
Aahz Maruch
Tim Peters <tim...@email.msn.com> wrote:
>
>This gets really involved really fast, and without a precise model
>of computation to build on, discussing relative power is a trap.
>*Generally* speaking, coroutines can be implemented via continuations,
>but not vice versa (continuations are more powerful); but continuations
>and threads can each be implemented via the other (but have vastly
>different pragmatics, which is why Christian worked so hard on
>Stackless); so if threads are more familiar, rephrase your question
>to ask whether having threads can make some things much easier to
>do. "Of course", and the same for continuations. Raw threads and
>raw continuations are exceedingly difficult to use correctly, though,
>so Python offers the higher-level threading.py to make thread life
>easier, and continuations in Scheme are usually used "under the
>covers" to implement an easier-to-use abstraction (like coroutines, or
>backtracking search, or ...).
Okay, now I'm starting to get this (be very, very frightened). Now, can
you (or someone) provide an equally simple explanation of the
differences between continuations and threads? Given that we already
have threads (sort of), what is the attraction of continuations? Is it
easier to write continuations than simulate threads on an unthreaded OS?
Moshe Zadka
> Okay, now I'm starting to get this (be very, very frightened). Now, can
> you (or someone) provide an equally simple explanation of the
> differences between continuations and threads? Given that we already
> have threads (sort of), what is the attraction of continuations? Is it
> easier to write continuations than simulate threads on an unthreaded OS?
Oh, continuations *cannot* be simulated by threads. (Coroutines can be
simulated by both, however). They are much more powerful then you think:
you can implement all sorts of funky control behaviour, from classic
exception handling to the types of exception where the catcher can
"correct" the situation and "retry". Very, very, cool.
be-much-more-frightened-ly y'rs, Z.
Samuel A. Falvo II
OK, that's informative. So much for the differences between threads and
continuations. But I'm still left wondering just what the heck
continuations are. :)
--
KC5TJA/6, DM13, QRP-L #1447
Samuel A. Falvo II
Oceanside, CA
Paul Prescod
>
> ...
>
> Okay, now I'm starting to get this (be very, very frightened). Now, can
> you (or someone) provide an equally simple explanation of the
> differences between continuations and threads?
I'm not sure if I believe what Tim said about implementing threads with
continuations or vice versa. A continuation is, in my mind, only vaguely
related to a thread.
Okay, imagine you are going through a Python program and you could all
of a sudden say to the interpreter: "remember this stack frame and all
variables in scope. Give it to me as a Python object. I'll do something
with it later." It's like groundhog day. You can *go back in time* to
somewhere you were before. Actually its the opposite of groundhog day
because what will have changed is globals and things relating to I/O. So
its like the little town in groundhog day is stuck in time but you can
tell that time is progressing because you can see news from the outside
world on television. So its more like the Truman show. Only in reverse.
:) That clears things up, right?
Anyhow, consider what you need to do to implement exception handling.
You need to say: "if this exception ever occurs, I want you to jump back
to this point in the program's execution." Well, that can be implemented
with continuations.
error=None
def SomeFunction():
if somethingIsNotRight():
error="Something stinks!"
jumpBackToOnError()
jumpBackToOnError=currentContinuation()
if error:
print "An error occurred", error
else:
SomeFunction()
It's ugly (low-level) but it works. You could have a hashtable mapping
exception types to continuations. There are ways to avoid all of the
global variables also.
Okay, next consider coroutines. Two flows of control that go back and
forth between each other:
out=0
def producer():
global producer_continuation, consumer_continuation, data
while 1:
producer_continuation=currentContinuation()
data = readSomeData()
consumer_continuation( )
def consumer():
global producer_continuation, consumer_continuation, data
while 1:
consumer_continuation=currentContinuation()
producer_continuation( )
print data
producer_continuation=None
consumer_continuation=None
data=None
You might want to include some exit condition!
You could combine the exception handling pattern with the coroutine
pattern so that your exception continuation jumps into the middle of a
loop:
while 1:
print error
error_count=error_count+1
next_error()
Not much more exciting than a callback but it would be if there was a
lot of local state that a callback would need to store in an object
somewhere.
You can also use continuations to implement back-tracking. You try one
path and if it fails you "jump back" and try the other. If you have
several days to expend in mind-altering time-travelling code obfuscation
you should try to puzzle out the strategy used by schelog to implement
prolog in Scheme using continuations.
http://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/prolog/impl/prolog/schelog/0.html
Ahh, for the good old days when I could spend hours on things that might
or might not ever have any job relevance.
Sigh-I-need-to-go-back-to-school-or-get-Tim-Peter's-job-ly 'yrs
Mike Fletcher
A continuation is a suspended execution context, you get to a point and say
"can't/won't finish this now, I'll store the state (frame) and let other
execution contexts call me when I should continue". When you want to resume
the context, you "call" the continuation, which resumes running the
suspended execution context at the next line after the call to suspend.
If I'm reading things right, there are methods on continuations which allow
for updating variables while not running the context, so you can pass
information into the context while it is suspended.
But that's just from me skimming through Christian's article :) . I've been
known to miss important CS concepts before ;) .
Enjoy,
Mike
-----Original Message-----
From: kc5...@garnet.armored.net [mailto:kc5...@garnet.armored.net]
Sent: Thursday, February 17, 2000 3:31 PM
To: pytho...@python.org
Subject: Re: Continuations and threads (was Re: Iterators & generators)
...
OK, that's informative. So much for the differences between threads and
continuations. But I'm still left wondering just what the heck
continuations are. :)
...
Harald Hanche-Olsen
| Not being a CS person, maybe I can confuse the issue :) ...
I'm not one either, so surely I can confuse it further!
| A continuation is a suspended execution context, you get to a point
| and say "can't/won't finish this now, I'll store the state (frame)
| and let other execution contexts call me when I should continue".
| When you want to resume the context, you "call" the continuation,
| which resumes running the suspended execution context at the next
| line after the call to suspend.
So far, this is not all that different from
coroutines/generators/threads/whatever. To my mind, one of the
mindboggling things about continuations is that you can call the same
continuation multiple times, whereas when you call a
coroutine/generator, its program counter advances, so that the next
time you call it, you are really calling a new continuation.
| If I'm reading things right, there are methods on continuations
| which allow for updating variables while not running the context, so
| you can pass information into the context while it is suspended.
Well, the global environment can always change between calls, but you
may even have two continuations sharing a lexical environment (i.e.,
they were both generated within the same procedure call), so calling
one can modify the local environment of the other.
maybe-posting-some-Scheme-code-could-obfuscate-the-issue-furhter-ly y'rs,
--
* Harald Hanche-Olsen <URL:http://www.math.ntnu.no/~hanche/>
- "There arises from a bad and unapt formation of words
a wonderful obstruction to the mind." - Francis Bacon
Neil Schemenauer
>OK, that's informative. So much for the differences between threads and
>continuations. But I'm still left wondering just what the heck
>continuations are. :)
Here is my two bit explaination (based on Scheme's call/cc):
A continuation is a function which represents the rest of the
program. Instead of returning from a function you can call this
function giving it one argument which is the return value.
These functions are first class just like everything else in
Scheme. You can bind them to variables or pass them as
arguments. You can also call them more than once passing
different "return" values.
I hope that doesn't damage the old melon too much. :)
Neil
jer...@cnri.reston.va.us
nasc...@enme.ucalgary.ca (Neil Schemenauer) wrote:
> Here is my two bit explanation (based on Scheme's call/cc):
>
> A continuation is a function which represents the rest of the
> program. Instead of returning from a function you can call this
> function giving it one argument which is the return value.
>
> These functions are first class just like everything else in
> Scheme. You can bind them to variables or pass them as
> arguments. You can also call them more than once passing
> different "return" values.
>
It may help to describe, in general terms, what you can do with a first
class continuation. Here's my two bit followup to your explanation.
Each time you call a function, there is an implicit continuation -- the
place where the function should return to. call/cc provides a way to
bind that implicit continuation to a name and keep hold of it.
If you don't do anything with the named continuation, the program will
eventually invoke it anyway. The function returns and passes control to
its continuation. Now that it's named, however, you could invoke the
continuation earlier than normal, or you could invoke it later (after it
has already been invoked by the normal control flow). In the early
case, the effect is a non-local exit. In the later case, it's more like
returning from the same function more than once.
Jeremy
Moshe Zadka
> >Oh, continuations *cannot* be simulated by threads. (Coroutines can be
>
> OK, that's informative. So much for the differences between threads and
> continuations. But I'm still left wondering just what the heck
> continuations are. :)
Well, just like the rest of us. I'll explain continuations in Scheme,
because that's the only continuatios I know of. I assume Christian will
pop up with Stackless's module "continuation" any time now.
Scheme has a function called call-with-current-continuation (usually
aliased in implementation to call/cc). It takes one function, which takes
one parameter, and calls it with an opaque object -- the continuation. The
continuation is a function accepting one parameter (OK, I'm simplifying
the truth here, but never mind). When the continuation is called with a
parameter "x", the function which called call/cc thinks call/cc returned
with the value "x". On the other hand, if the function which was given to
call/cc returns with the value "x", the function which called call/cc also
thinks call/cc returned with value "x".
You can use it like setjmp/longjmp, in things like callbacks: before
before calling the callback, set a global variable to the current
continuation, and call it. Any time the callback wants to "longjmp", it
merely calls the continuation with a pre-determined value. When the
call/cc returns that value, you know the callback jumped.
That was probably very incomprehensible, but that's quite all right.
Continuations are hard. The timbot explained to me a few times, so by now
I can fool lots of people into believing I understand them, but it still
sometimes baffles me.
Aahz Maruch
Paul Prescod <pa...@prescod.net> wrote:
>Aahz Maruch wrote:
>>
>> Assuming that I've understood everything you've said about
>> continuations, I think I also understand what Tim meant about modeling
>> threads with continuations and vice-versa. Seems to me that one could
>> easily implement continuations with a process that forks off and blocks
>> waiting for one of two signals: resume or die. (When a process forks,
>> both processes have almost exactly the same state, right?)
>
>Okay, but how is implementing threading via fork() using continuations?
>The underlying OS task switcher is doing all of the work, not the
>language's continuation feature.
No, no, I meant that one could implement continuations using fork() and
IPC. It should therefore be possible to implement continuations using
threads. And I think that's all the Timbot meant.
>> [code deleted]
>> Am I right?
>
>Well your version had some improvements but I think that they make more
>clear the fact that the while loop is not useful. The continuations will
>bounce back and forth forever by themselves. In this case they are very
>much like "goto" in basic which probably explains why Common Lisp people
>hate them.
Uh-huh! I have to admit that I almost ripped out the while loop because
it looked like it wasn't necessary, but it occurred to me that the
semantic of a reference to an object returned by currentContinuation()
could include advancing the PC in the current continuation frame. Which
means that the initial call to producer really ought to look like:
continuation producer()
Tim Peters
> I'm not sure if I believe what Tim said about implementing threads with
> continuations or vice versa.
This is going to turn into another Vladimir "locality of reference"
three-msg gloss on a toss-away parenthetical comment <wink>. Like I said,
this stuff gets very involved very fast, and without a precise model of
computation to build on is just going to thrash (another parallel to LOR
...).
I really should *not* have said that continuations can be implemented via
threads. My apologies for that. They can be <wink>, but not with the
flavor of threads most people are most likely to think of (the other thing
needed is hinted at below).
> A continuation is, in my mind, only vaguely related to a thread.
Threads can certainly be implemented via continuations, so far as the formal
denotational semantics go. But those formal semantics don't cover the
pragmatics of parallel execution, so in some practical contexts it's a
worthless observation.
It's of real value in Python, though, because Python's flavor of threads are
currently constrained, by the global lock, to act much more like coroutines
than "real threads" (the co-transfers are implicit instead of explicit, but
other than that they *are* just coroutines). A great deal of the potential
practical value of Christian's Stackless approach lies exactly here. Sam
Rushing has written eloquently about this, so visit his site and read his
papers on Medusa & friends. Short course: whenever you find yourself
building a hairy state machine to implement a protocol, suspect that there's
an *elegant* approach straining to break free of that cruft.
> Okay, imagine you are going through a Python program and you could all
> of a sudden say to the interpreter: "remember this stack frame and all
> variables in scope. Give it to me as a Python object. I'll do something
> with it later." It's like groundhog day. You can *go back in time* to
> somewhere you were before. Actually its the opposite of groundhog day
> because what will have changed is globals and things relating to I/O. So
> its like the little town in groundhog day is stuck in time but you can
> tell that time is progressing because you can see news from the outside
> world on television. So its more like the Truman show. Only in reverse.
> :) That clears things up, right?
No <wink>, but it goes part of the way in a helpful direction. What you
described there is no more than what C's setjmp/longjmp accomplishes, and
that isn't enough. *If* setjmp captured not only the current stack frame,
but *all* stack frames on the entire call stack-- and longjmp resumed
execution via a copy of the whole banana --then you'd be within spitting
distance of a real continuation implementation (albeit not a reasonably
*practical* one).
And that's probably the clearest "operational model" for C programmers. In
theory, it's much simpler, and *so* much simpler that it's very hard to
explain! "Continuations in theory" are more basic even than ordinary
subroutine calls. In fact, they were discovered during attempts to
rigorously define the semantics of "goto" stmts, and in an odd but real
sense continuations are "more basic" than gotos. In the theoretical
framework, it's pretty much a trivial (in retrospect) fiddling of functions
modeling "program state". If you take continuations as primitive,
everything else follows easily; the difficulty here is that "everyone
already knows" how subroutine calls work, so try to build continuations on
*top* of that assumed model. It's backwards, and the need to invert your
brain in order to "get it" is much of what Christian means when he said he
had to "redefine truth".
> ...
> Sigh-need-to-go-back-to-school-or-get-Tim-Peter's-job-ly 'yrs
You'll have to borrow Guido's time machine, then -- I stopped writing
compilers in '94, and am still writing c.l.py msgs based on the little I
haven't yet forgotten.
luckily-nothing-new-has-been-discovered-since-'39<wink>-ly y'rs - tim
Tim Peters
> ...
> No, no, I meant that one could implement continuations using fork()
> and IPC. It should therefore be possible to implement continuations
> using threads. And I think that's all the Timbot meant.
It wasn't, but maybe I should pretend it was <wink>. Tried to clarify this
in another msg.
The difficulty with using fork() is that continuations are entirely about
capturing control-flow state and nothing about capturing data state (except
for such "implementation data" that's stacked to record info about control
flow). A resumed continuation "sees" changes in ordinary data state made
since the time it was captured. If a continuation *didn't* see changes in
data state, then invoking it later multiple times would do exactly the same
thing each of those times (barring connection to an external source of
non-determinacy, etc). And that's pretty useless.
Suggestion: forget all this. It's at the wrong level for almost everyone
struggling to follow it. While continuations were a kind of breakthrough in
the theoretical world, in practice they're simply an *implementation
technique*. Almost nobody is ever going to want to use them directly.
What's *interesting* is what the language could provide if they were
available to a few insane souls to build on "under the covers". A Scheme
True Believer builds all control flow (from function calls thru exceptions
to pseudo-threading) on top of them. Python won't do that, and I expect
Guido is just irked at suggestions that "he should" (why the hell should he
<wink>?).
In Python the value would be for what they could provide we don't already
have: not exception-handling, but (most clearly) lightweight generators &
coroutines, and (less clearly) lightweight alternatives to threads. The
pragmatics of Stackless are such that you can have thousands of
"pseudothreads" running even on feeble platforms with no thread support; and
without the C stack interwined in the program state, it's even possible we
could whip something up to, e.g., freeze a (pure) Python program in
mid-stream, pickle it, send it across the web, and resume it where it left
off on some other architecture.
Indulge some crazy-ass fanatasies! Worrying about the mechanics of
continuations here is akin to a debate starting about what kind of numeric
types P3K should offer, that degenerates into an endless discussion about
how best to build adder circuits in .18 micron technology <0.9 wink>.
they're-a-means-not-an-end-ly y'rs - tim