Idea - alternative to lambda
Greg Ewing
Python wants to replace many of the uses of
lambda in list-mapping operations:
List comprehensions!
def zip(L1, L2):
return [(x, y) for x in L1, y in L2]
Now would that be elegant or would that
be elegant?
--
Greg Ewing, Computer Science Dept, | The address below is not spam-
University of Canterbury, | protected, so as not to waste
Christchurch, New Zealand | the time of Guido van Rossum.
gr...@cosc.canterbury.ac.nz
M.-A. Lemburg
>
> It suddenly occurred to me this morning what
> Python wants to replace many of the uses of
> lambda in list-mapping operations:
>
> List comprehensions!
>
> def zip(L1, L2):
> return [(x, y) for x in L1, y in L2]
>
> Now would that be elegant or would that
> be elegant?
Have a look at mxTools:
http://starship.skyport.net/~lemburg/mxTools.html
Especially the tuples(), lists() functions...
>>> import NewBuiltins
>>> l = range(1,10)
>>> k = range(2,11)
>>> tuples(l,k)
[(1, 2), (2, 3), (3, 4), (4, 5), (5, 6), (6, 7), (7, 8), (8, 9), (9,
10)]
With it you can write:
for a,b in tuples(list1,list2):
# ...
BTW: The inverse is also possible:
>>> ll,kk = lists(_)
>>> ll
[1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> kk
[2, 3, 4, 5, 6, 7, 8, 9, 10]
--
Marc-Andre Lemburg Y2000: 513 days left
---------------------------------------------------------------------
: Python Pages >>> http://starship.skyport.net/~lemburg/ :
---------------------------------------------------------
Paul Prescod
than just tupleize multiple lists?
Paul Prescod
Greg Ewing wrote:
>
> It suddenly occurred to me this morning what
> Python wants to replace many of the uses of
> lambda in list-mapping operations:
>
> List comprehensions!
>
> def zip(L1, L2):
> return [(x, y) for x in L1, y in L2]
>
> Now would that be elegant or would that
> be elegant?
>
> --
> Greg Ewing, Computer Science Dept, | The address below is not spam-
> University of Canterbury, | protected, so as not to waste
> Christchurch, New Zealand | the time of Guido van Rossum.
> gr...@cosc.canterbury.ac.nz
--
Paul Prescod - http://itrc.uwaterloo.ca/~papresco
http://www.capitol.state.tx.us/txconst/sections/cn000100-000400.html
"No religious test shall ever be required as a qualification to any
office, or public trust, in this State; nor shall any one be
excluded from holding office on account of his religious sentiments,
provided he acknowledge the existence of a Supreme Being."
- Texas Constitution, Article 1, Section 4
Paul Magwene
function any different than the following?
map(None,L1,L2)
That would seem to produce the same list of tuples...
--Paul Magwene
Greg Ewing
>
> Have a look at mxTools:
>
> http://starship.skyport.net/~lemburg/mxTools.html
>
> Especially the tuples(), lists() functions...
I think you've missed the point. What I wrote
was just one example of the use of an extremely
general mechanism.
Tim Peters
> It suddenly occurred to me this morning what
> Python wants to replace many of the uses of
> lambda in list-mapping operations:
>
> List comprehensions!
>
> def zip(L1, L2):
> return [(x, y) for x in L1, y in L2]
[Paul Prescod]
> I don't understand the wider implications of your proposal. Can
> it do more than just tupleize multiple lists?
List comprehensions are a std feature of modern functional languages, to
which you can look for more examples. You generally want an expression that
computes each element of the final list ("(x, y)" in the above), a
collection of iterators to bind names for use in the expression (the rest of
the above -- although I tend to read it as the cross product of L1 and L2
rather than a lockstep parallel iteration -- the syntax needs work), and an
optional filtering expression to further limit which elements get generated.
Some made-up examples:
[x**2 for x in range(10)] == [0, 1, 4, 9, ..., 81]
[f(x) for x in L] == map(f, L)
[sqrt(x) for x in range(-1,2) : x >= 0] == [0., 1.]
(reading ":" as "such that")
[x for x in L] == list(L)
[x for x in L : predicate(x)] == filter(predicate, L)
[42 for x in range(N)] = [42] * max(N, 0)
[(x, y**3) for x in range(3), for y in range(3): x < y] ==
[(0, 1), (0, 8), (1, 8)] ==
[(x, y**3) for x in range(3), for y in range(x+1, 3)]
M = [(len(s), s) for s in stringList]
M.sort()
stringList = [x[1] for x in M]
sorts a list of strings by length
[f(pi) for f in (sin, cos, tan)] == [sin(pi), cos(pi), tan(pi)]
[f(x, y) for x in L, for y in M(x): g(x, y)] ==
X after
<<save the current bindings for x and y>>
X = []
try:
for x in L:
for y in M(x):
if g(x, y):
X.append(f(x, y))
finally:
<<restore the saved bindings for x and y>>
Note: I avoided "the usual" transformation into nested lambdas, because
this much more efficient (in Python) flavor of implementation seems
*possible*: given Guido's overwhelming fondness for the functional features
that snuck into Python while he was snoozing <wink>, efficiency may be the
only grounds on which it could be sold.
OTOH, given the straightforwardness of the transformation, "really now, why
bother?" comes to mind <0.7 wink>. This kind of thing also gets much more
useful with related features, e.g. quantifier tests:
primesInL = [x for x in L: x > 1 and
each d in range(2,x) has
x % d != 0]
Curiously enough, ABC *did* have quantifier tests, but didn't incorporate
them into its notion of list-builders (which were just a little fancier than
Python's, and (IMO) not *usefully* so).
i'd-pay-good-money-to-know-what-i-really-think<wink>-ly y'rs - tim
Ken McDonald
> > It suddenly occurred to me this morning what
> > Python wants to replace many of the uses of
> > lambda in list-mapping operations:
> >
> > List comprehensions!
Woohoo! Tim, I haven't seen list comprehensions mentioned since the
late, unlamented (and unpleasant) days of my Ph.D. (attempt at) a
thesis. (Not all that long ago, come to think of it.) To think of
it--there
really _are_ more than thirteen people in the world who know something
about functional programming languages. :-)
More seriously, I do think that the functional language community
has come up with some very good ideas, and am glad to see a few of
them in Python. List comprehensions would be nice, though I'm
not sure a lot of people would be comfortable with them--they are
rather unusual. But what I'd really like to see (still!) is polymorphic
typing. Unfortunately, this wonderful system has languished in the
functional programming community due to their (in my mind) unfounded
abhorrence of side effects and perverse attraction to lazy evaluation.
It would be nice to see Hindley-Milner (or some related) typing
liberated.
In purely logical terms, the best language I've ever seen was the
functional language Concurrent Clean. (Somewhere on the
University of Nijmegen site.) Unfortunately, like most fpers,
the Concurrent Clean group never quite grasped the need to
come down to the level of the real world, which is why Python
is beating CC's pants off in terms of popularity. (Just for the
record--Python is a lot more _fun_ than Concurrent Clean.)
But man, what I wouldn't give for a language that combined the
best features of the two...
Cheers everyone,
Ken
M.-A. Lemburg
>
> M.-A. Lemburg wrote:
> >
> > Have a look at mxTools:
> >
> > http://starship.skyport.net/~lemburg/mxTools.html
> >
> > Especially the tuples(), lists() functions...
>
> I think you've missed the point. What I wrote
> was just one example of the use of an extremely
> general mechanism.
Maybe you should've made the point a little clearer then !? If all
you want is a short-cut for...
def inlined_map_lambda_on(y):
l = []
for x in y:
# do something with x giving z
l.append(z)
return l
...I don't see the advantage of those "inlined" lambdas. The above
gives you all you want (and much more) in a readable and easily
comprehensible way... and it's fast too :-)
Let's stick to MOPs -- they have much more practical use ;-)
--
Marc-Andre Lemburg Y2000: 512 days left
Paul Prescod
functions. I think that it is much more readable for people without a
functional programming background, especially since it delays introduction
to Python's partially-functional lambda. Personally, I almost always use
lambda with map and filter.
Johan Ovlinger
already proposed in this thread. However, as far an my news server knows, we
have yet to see a real description of what they are, so here is my take:
First let me point out that like so many other useful programming constructs,
comprehensions do nothing that cannot be done in normal python. Infact, they
are often translated (de-sugared) to the core language early on in the
compilation process. They do however allow you to succintly specify many
common list processing tasks, like this one:
[ j*j | j <- [1..10], (j%2) == 0 ]
which is the list containing all the even numbers from 1 to 10 squared.
The sematics are that the left hand side expression is the "inner loop
expression" of the loop formed by everything to the right of the "|". All the
results are appended to a list, which is the result of the comprehension.
so before the expression we create a variable to hold the result:
"res = []"
The right hand side binds variables (j in this case) to a sequence of values
(a generator)
so "j <- [1..10]" might be translated as
"for j in range(1,10+1):"
The right hand side can also perform tests to screen out unwanted elements
(a guard)
so "(j % 2) == 0" is translated to
"if not ((j%2)==0):
continue
else:"
All right hand side expressions are either generators or guards. You can nest
comprehensions.
and finally, the inner loop expression is evaluated and appended to the result:
"res.append(j*j)"
Thus the entire shebang thus becomes:
res = []
for j in range(1,10+1):
if not ((j%2)==0):
continue
else:
res.append(j*j)
Notice we need to "return res", which is the result of the comprehension.
Obviouisly my translation alone is not going to work. A translation that
actually works would probably have each comprehension translated to a function
and have the free varaibles passed in as arguments. Also, I'm sure that much
of the translation can be done in such a way that removes the assumption of
the result and inputs being lists.
For more info on list comprehensions have a look at any program from the
haskell web site (www.haskell.org). For more info on desugaring, read S.P.
Jones's book "The Implementation of Functional Programming Languages", or
check out Phil Wadler's publications; I think he has a paper published on this
subject.
I hope this clarified more than it confused.
Johan
Guido van Rossum
> functions. I think that it is much more readable for people without a
> functional programming background, especially since it delays introduction
> to Python's partially-functional lambda.
Better yet, it fixes the problems that Python's lambda has!
--Guido van Rossum (home page: http://www.python.org/~guido/)
sk...@calendar.com
> List comprehensions are more general than any of the specific suggestions
> already proposed in this thread. However, as far an my news server knows, we
> have yet to see a real description of what they are, so here is my take:
I appreciate the elaboration. I, for one, hadn't encountered them before.
> [ j*j | j <- [1..10], (j%2) == 0 ]
>
> which is the list containing all the even numbers from 1 to 10 squared.
If such a construct is adapted to Python, I suggest the range part be more
Pythoneque. Either use the range() function explicitly, or (if that's not
possible), have the range in the list ([x..y]) run from x to y-1.
> For more info on list comprehensions have a look at any program from the
> haskell web site (www.haskell.org).
I take it list comprehensions are a core construct in Haskell. If Guido is
thinking about implementing list comprehensions in Python, perhaps a more
complete set of concepts could be "lifted" from Haskell so that we get a
fully useful set of constructs. Then again, list comprehensions by
themselves may be sufficiently powerful to warrant inclusion without any
other features from Haskell.
--
Skip Montanaro
Musi-Cal ads: http://concerts.calendar.com/cgi-bin/genad
-----== Posted via Deja News, The Leader in Internet Discussion ==-----
http://www.dejanews.com/rg_mkgrp.xp Create Your Own Free Member Forum
Jo Meder
sk...@calendar.com writes:
> > [ j*j | j <- [1..10], (j%2) == 0 ]
> >
> > which is the list containing all the even numbers from 1 to 10 squared.
>
> If such a construct is adapted to Python, I suggest the range part be more
> Pythoneque. Either use the range() function explicitly, or (if that's not
> possible), have the range in the list ([x..y]) run from x to y-1.
Why not implement the notation [1..10] as an alternative in general to
range(1,11) (which btw. I always found to be unintuitive but
pythonesque as you call it ;-)
I try to come up with code that would break if you allowed this all
over python but can't think of any since IIRC the ".."-notation is not
used anywhere else.
Jo
--
j...@delorges.in-berlin.de --- Berlin, Germany
http://delorges.in-berlin.de/
Johann Hibschman
> More seriously, I do think that the functional language community
> has come up with some very good ideas, and am glad to see a few of
> them in Python. List comprehensions would be nice, though I'm
> not sure a lot of people would be comfortable with them--they are
> rather unusual.
What about array comprehensions? I read a little about them in the
Haskell gentle intro, but I don't know much about it. How far is it
from list comprehensions to APL or to IDL? In IDL, I'm used to doing
things like "A[where(A < 0)] = 0" to eliminate negative numbers. (I
don't actually know APL, so I hang my numericist head in shame, but it
has three letters and ends in L, so I thought I'd mention it. :-) )
This can be done in numeric python, but it's a bit awkward; there have
been a few discussions on the NumPy mailing list about generic set/get
commands. Is there a niche for "array comprehensions" here?
Basically, I don't know much about list comprehensions, but they
tickle my brain and make me wonder if any hook provided to allow them
would also allow clever array-manipulation syntax, which I would find
useful.
- Johann
Amit Patel
| [Johan]
| > List comprehensions are more general than any of the specific suggestions
| > already proposed in this thread. However, as far an my news server knows, we
| > have yet to see a real description of what they are, so here is my take:
|
| I appreciate the elaboration. I, for one, hadn't encountered them before.
|
| >
| > which is the list containing all the even numbers from 1 to 10 squared.
|
| If such a construct is adapted to Python, I suggest the range part be more
| Pythoneque. Either use the range() function explicitly, or (if that's not
| possible), have the range in the list ([x..y]) run from x to y-1.
|
I agree that the range construct should fit into Python but I suggest
instead using the slice notation:
_________
[ j*j | j in 1:11 and (j%2) == 0 ]
~~~~~~~~~
Actually I'd like to see 1:11 translated into range(1,11) everywhere. :---)
- Amit
Greg Ewing
>
> I don't understand the wider implications of your proposal. Can it do more
> than just tupleize multiple lists?
I thought the extrapolation of it would be fairly
obvious, but it seems not, so here's a more complete
description.
The expression
[<expr> for x1 in L1; x2 in L2; ... ; xn in Ln]
is equivalent to the value of result after executing
result = []
for x1 in L1:
for x2 in L2:
...
for xn in Ln:
result.append(<expr>)
If some of the "in" clauses are separated by commas instead
of semicolons, the relevant list traversals are parallel instead
of nested. (I hope you get what I mean by that - the equivalent
Python translation is a bit messy to write out in full.)
Tim Peters
> Woohoo! Tim, I haven't seen list comprehensions mentioned since
> the late, unlamented (and unpleasant) days of my Ph.D. (attempt
> at) a thesis. (Not all that long ago, come to think of it.)
Whew! You wrote an entire dissertation defending list comprehension <wink>?
> To think of it--there really _are_ more than thirteen people in
> the world who know something about functional programming
> languages. :-)
Indeed, if you track comp.lang.functional very closely, never missing a day,
I believe you can, over a one-year period, sometimes find as many as 15
distinct posters who didn't stumble in just because they hoped "functional"
meant "not broken" <wink>.
> More seriously, I do think that the functional language community
> has come up with some very good ideas,
Indeed they have. Landin was using indentation to denote block structure in
ISWIM in the 60's <wink>.
> and am glad to see a few of them in Python.
Last time I called them "minor conveniences", Guido corrected it to "minor
annoyances" -- they don't really fit all that well in Python! They look and
feel tacked on in Python -- and only partly because they were.
> List comprehensions would be nice, though I'm not sure a lot of
> people would be comfortable with them--they are rather unusual.
I've come to believe it's just the opposite: provided we stay away from the
full-blown fp approach of "explaining" them by appeal to an equivalent
schema of nested lambdas, and avoid jabbering about monads & such, they have
a natural translation into nested Python loops. Indeed, that's part of the
progress functional languages have made: they added enough sugar to make
common things *not* look like endlessly nested function applications. So,
like Paul, I actually expect it would be easier for a newbie to grasp
[x**2 for x in S]
than
map(lambda x: x**2, S)
The former looks familiarly imperative.
I'm comfortable enough with the latter, but then I'm just generally old and
in the way <wink>.
> But what I'd really like to see (still!) is polymorphic typing.
Python already has that -- you just have to put it in comments for now
<wink>.
> [about the perversions of the fp community, the wonderfulness of
> Concurrent Clean, and the ascendancy of Python over all due to
> its love of wallowing in mud]
> ...
> But man, what I wouldn't give for a language that combined the
> best features of the two...
Alas, you're far more likely to find one that combines the worst <0.5 wink>.
wondering-why-nobody-wants-to-combine-the-best-features-of-
fortran-and-javascript-ly y'rs - tim
Tim Peters
> I rather like this syntax as a replacement for the map and filter
> functions. I think that it is much more readable for people without a
> functional programming background, especially since it delays
> introduction to Python's partially-functional lambda.
Modern functional languages can be very readable; they've taken some
remarkable steps in the direction of syntactic sugaring. The bits of
"functional features" in Python that people don't like are really due to
Lisp <wink>.
[Guido]
> Better yet, it fixes the problems that Python's lambda has!
Assuming you mostly mean scoping here. The other differences in e.g.
[translate[x] for x in S]
vs
map(operator.__getitem__, [translate] * len(S), S)
should be considered too: natural syntax, and trick-free scalar broadcast.
OTOH, the latter is likely to run faster despite the gross by-hand broadcast
because the former loop runs at Python speed. OTTH, speed is probably
reversed for most any case in which the mapped expression is a lambda.
BTW, I double-checked today, and at least Haskell doesn't appear to have
grown special syntax for "lockstep parallel iteration" -- they do Greg's
"zip" example via a recursive function instead. OTOH,
[(x, y) for x, y in map(None, L1, L2)]
kinda begs the question <wink>. But
[(x[i], y[i]) for i in range(len(x))]
does seem a far sight prettier than
map(lambda i,x=x,y=y: (x[i], y[i]), range(len(x))]
Once reason comprehensions are nice in practice is that they put the
"important part" at the start of the line.
dreamily y'rs - tim
Terry Reedy
notations for defining sets:
1. Explicit (objective): list the members. IE
set = {4,16,36,64,100}
Python's list notation (which allows *anything* as a member, is a direct
tranlation/version of this. Example:
lst = [4,16,36,64,100]
When the order is arbitrary and meaningless, the list is essentially a
set.
2. Implicit (functional): generate the members.
As given by Johan Ovlinger, a 'list comprehension' such as
>[ j*j | j <- [1..10], (j%2) == 0 ]
>
>which is the list containing all the even numbers from 1 to 10 squared.
is a variation for lists of standard set builder notation:
S = {j*j | j e {1,...,10} & (j%2 - 0)} or
S = {j*j | 2<= j <= 10 & j e Even}
<'e' here is ASCII substitute for 'epsilon' indicating set membership>
The two notations provide two ways to test an object for membership:
scan the list versus apply the predicate(s).
The filter notation is more powerful since it comprehends infinite sets.
It might be interesting if Python incorporated a version of this second
standard and widely-used notation.
I was about to add that it would also be interesting if Python added an
implicit list type as a complement to the explicit list type, then
realized that xrange objects are an example of such and that the magic
instance method protocol allows users to define similar objects.
Terry J. Reedy
Paul Prescod
>
> I thought the extrapolation of it would be fairly
> obvious, but it seems not, so here's a more complete
> description.
> The expression
>
> [<expr> for x1 in L1; x2 in L2; ... ; xn in Ln]
>
Your formulation of it doesn't seem to support filtering, other than that
already provided in the expressions that create L1, L2, etc. The feature
doesn't strike me as very interesting without the filtering. It's the
combination of map and filter that makes some of the other proposals so
cool!
Tim Peters
> ...
> I take it list comprehensions are a core construct in Haskell.
> If Guido is thinking about implementing list comprehensions in
> Python, perhaps a more complete set of concepts could be
> "lifted" from Haskell so that we get a fully useful set of
> constructs.
Haskell is a pure functional language (no side-effects or "program
counter"), and like most such its core is curried lambda expressions
augmented by a type system. Its list comprehensions are light syntactic
sugar for another form of expression, which is in turn heavy syntactic sugar
for (after a few more layers) piles of lexically scoped lambdas. It hangs
together beautifully on its own terms, but hard to factor -- it's a prime
language <wink>.
The nice thing about list comps. is that their most useful forms could be
implemented directly as light sugar for ordinary Python loops, leaving
lambdas out of it entirely. You end up with a subtly different beast, but
so far it appears to be a beast that's compatible with cuddly pythons.
Haskell's type & type declaration system would be worth a gander, but that's
a different issue; essentially all progress in crafting spellable <wink>
type systems in dynamic polymorphic languages has originated in the fp
community.
necessity-is-the-mother-of-prevention-ly y'rs - tim
Marc Wachowitz
> What about array comprehensions?
[...]
If you want to read further exciting stuff about all this, look at Sisal,
a purely functional programming language supposed to be able to replace
Fortran in scientific computing with high performance requirements; see
http://www.llnl.gov/sisal/SisalHomePage.html
--
GiS - Gesellschaft fuer integrierte Systemplanung mbH
Marc Wachowitz Tel. +49-6201-503-38
Junkersstr. 2 Fax +49-6201-503-66
D-69469 Weinheim m.wac...@gis.ibfs.de
Terry Reedy
>I actually expect it would be easier for a newbie to grasp
>
> [x**2 for x in S]
>
>than
>
> map(lambda x: x**2, S)
I am well past the newbie stage and I already find the [] notation
easier to grasp: the naked [] says form a list; x**2 says puts squares
in it; the rest is details of what, in a concise form familiar because
of reuse of existing Python syntax. Very nice!
Terry J. Reedy
Tim Peters
> Sets are a basic concept in mathematics. There are two standard
> notations for defining sets:
>
> 1. Explicit (objective): list the members. IE
>
> set = {4,16,36,64,100}
>
> Python's list notation (which allows *anything* as a member, is a direct
> tranlation/version of this. Example:
>
> lst = [4,16,36,64,100]
>
>
> 2. Implicit (functional): generate the members.
>
> As given by Johan Ovlinger, a 'list comprehension' such as
>
>> [ j*j | j <- [1..10], (j%2) == 0 ]
>>
>> which is the list containing all the even numbers from
>> 1 to 10 squared.
>
> is a variation for lists of standard set builder notation:
>
> S = {j*j | j e {1,...,10} & (j%2 - 0)} or
> S = {j*j | 2<= j <= 10 & j e Even}
> 'e' here is ASCII substitute for 'epsilon' indicating
> set membership>
List comprehensions were, of course, inspired by std set-builder notation.
In the radically set-based language SETL, your examples are
{j*j: j in {1..10} | j mod 2 = 0}
{j*j: j in {2..10} | j in Even}
How to syntactically distinguish "the expression" from iterators from
filters is pretty much arbitrary, and for Python purposes I pretty much
unconsciously latched on to a trivial variant of Python's "for" syntax:
list_iterator: marcher [":" guard] # guard defaults to 1
marcher: "for" target_list "in" expression_list
guard: expression
It differs from existing "for" syntax only in that the tail is optional and,
when present, is filled with an expression rather than a suite. The problem
with "|" is that it already has a meaning, and the problem with eliding
"for" is that "x in y" does too. But *given* a leading "for" in the first
iterator, there's no need for additional gibberish to distinguish "the
expression", or even between iterators, so the whole syntax could be
list_comprehension: "[" expression list_iterator+ "]"
and the translation essentially
temp = []; append = temp.append
marcher1:
if not (guard1): continue
marcher2:
if not (guard2): continue
...
append(expression)
So those last two examples could be (applied to lists rather than sets):
[j*j for j in range(1,11): j % 2 == 0}
[j*j for j in range(2,11): j in Even]
> The two notations provide two ways to test an object for membership:
> scan the list versus apply the predicate(s).
But the latter is intractable: given (in your notation) {f(x) | pred(x)},
Python can't test y for membership by "applying the predicate" unless it has
an effective way to compute the inverse of f (i.e. to find all x such that
f(x) == y); the predicate doesn't apply to the set *members*, but to
arbitrary thingamabobs from which the set members are *computed* in
arbitrary ways.
> The filter notation is more powerful since it comprehends infinite
> sets.
I think Guido's likely to say he'll support infinite sets when computers
come with infinite memory <0.9 wink>.
> It might be interesting if Python incorporated a version of this second
> standard and widely-used notation.
The list-builder above is as powerful as SETL's set-builder (although SETL
also has tests for existential and universal quantification -- but that's
part of its boolean expression sublanguage, not specific to set-builders),
and SETL pushed all this about as far as is practical for a computer
language. Even when dealing with theoretical sets on paper, you generally
want to limit set builders to filtering from a pre-existing set (formally,
stick to the "axiom of separation"), lest you fall into paradoxes e.g.
russell = {x | x not in x}
print russell in russell
Although I, personally, would love to see Guido define the correct output
for that program <wink>.
guido-defines-languages-for-all-those-who-don't-define-languages-
for-themselves-ly y'rs - tim
Kevin Digweed
I think that this is much more accessable than the map()/filter()/lambda
stuff. It's syntactic sugar, but not for the sake of it - the intention
of the code becomes very clear, and that at least makes it Python-like.
The intention of code using map() filter() et al can easily be hidden
even in one-liners.
Specifically:
[Tim Peters]
>It differs from existing "for" syntax only in that the tail is optional and,
>when present, is filled with an expression rather than a suite. The problem
>with "|" is that it already has a meaning, and the problem with eliding
>"for" is that "x in y" does too. But *given* a leading "for" in the first
>iterator, there's no need for additional gibberish to distinguish "the
>expression", or even between iterators, so the whole syntax could be
[snip]
>So those last two examples could be (applied to lists rather than sets):
> [j*j for j in range(1,11): j % 2 == 0}
> [j*j for j in range(2,11): j in Even]
My initial thought when someone mentioned filtering was something like:
[j*j for j in range(1,11) if j % 2 == 0]
... however I like the idea that the for loop has a more consistent
syntax in your example. But, if we combine the two we could have
[j*j for j in range(1,11): if j % 2 == 0]
which doesn't look like much until you replace that "if" with
something else (say, "while" to prematurely terminate):
[j*j for j in range(1,11): while j < 5]
--> [0, 1, 4, 9, 16]
which in turn doesn't look like much until you use a less fixed example ;) :
tuple_list = [(100, 200), (300, 400), ("sentinel", 0), (99, 99)]
[(x[1], x[0]) for x in tuple_list: while x[0] != "sentinel"]
--> [(200, 100), (400, 300)]
Of course, we'd now need to be able to supply more than one "filter" part:
[(x[1], x[0]) for x in tuple_list: while x[0] != "sentinel", if x[0] != 100]
--> [(400, 300)]
... and there is scope for new "filter" parts to be added in the future.
Cheers, Kev.
Tim Peters
> I think that this is much more accessable than the map()/filter()/lambda
> stuff. It's syntactic sugar, but not for the sake of it - the intention
> of the code becomes very clear,
At least in simple examples <0.5 wink>.
> and that at least makes it Python-like. The intention of code using
> map() filter() et al can easily be hidden even in one-liners.
Yes, "the important part" ends up somewhere in the middle of the line, and
the lambda scope barrier leads to workaround trickery.
> [tim suggests]
> list_comprehension: "[" expression list_iterator+ "]"
> list_iterator: marcher [":" guard] # guard defaults to 1
> marcher: "for" target_list "in" expression_list
> guard: expression
[Kevin]
> My initial thought when someone mentioned filtering was something like:
>
> [j*j for j in range(1,11) if j % 2 == 0]
>
> ... however I like the idea that the for loop has a more consistent
> syntax in your example.
I'm not sure I do -- syntactic congruence can be misleading when semantics
differ, and in this case the colon doesn't "mean" what it usually means
after a "for" -- it's there just to let the parser distinguish the
expression_list from the guard (e.g. "for x in j + 3" would be ambiguous
otherwise: iterating over j+3 with no guard, or iterating over j with guard
+3?).
> But, if we combine the two we could have
>
> [j*j for j in range(1,11): if j % 2 == 0]
>
> which doesn't look like much until you replace that "if" with
> something else (say, "while" to prematurely terminate):
>
> [j*j for j in range(1,11): while j < 5]
>
> --> [0, 1, 4, 9, 16]
>
> which in turn doesn't look like much until you use a less fixed
> example ;) :
>
> tuple_list = [(100, 200), (300, 400), ("sentinel", 0), (99, 99)]
> [(x[1], x[0]) for x in tuple_list: while x[0] != "sentinel"]
>
> --> [(200, 100), (400, 300)]
Interesting twist! But the colon seems to me to *really* mislead here now,
and the parser doesn't need two pieces of noise to keep things straight.
[(two, one) for one, two in tuple_list while one != "sentinel"]
reads better to me, and the absence of a colon makes me less inclined to
misread the filter as a while loop.
> Of course, we'd now need to be able to supply more than one
> "filter" part:
>
> [(x[1], x[0]) for x in tuple_list: while x[0] != "sentinel", if
> x[0] != 100]
>
> --> [(400, 300)]
>
> ... and there is scope for new "filter" parts to be added in
> the future.
Noise isn't really needed between filters either, and I'm afraid commas
would be misleading if there's more than one iterator. I'd expect sane
people to use newlines to aid the eye, though:
[(two, one) for one, two in tuple_list
while one != "sentinel"
if one != 100]
OTOH, at this point there's so much stuff going on that an explicit for loop
is likely as clear overall:
a = []
for one, two in tuple_list:
if one == "sentinel":
break
if one != 100:
a.append((two, one))
listingly y'rs - tim
Dirk Heise
> If some of the "in" clauses are separated by commas instead
> of semicolons, the relevant list traversals are parallel instead
> of nested.
L = x*y for x,y in L1,L2
for the "parallel" evaluation? (BTW: Excuse me,
but personally i have the feeling that i need code like that
in 1 of 10000 lines of code so i don't think it makes the
language remarkably better. just more difficult to learn.)
Dirk
Greg Ewing
>
> that's too eror-prone, why not:
>
> L = x*y for x,y in L1,L2
>
> for the "parallel" evaluation?
I would have suggested this myself, except for the
unfortunate fact that Python already attaches a different
meaning to that in ordinary for loops:
for x,y in [1,2],['a','b']:
print x,y
prints:
1 2
a b
and not:
1 a
2 b
as one might expect at first glance!
Greg Ewing
>
> which doesn't look like much until you replace that "if" with
> something else (say, "while" to prematurely terminate):
>
> [j*j for j in range(1,11): while j < 5]
Noooooo..... this is going to far......
One of the nicest things about the idea as I
first proposed it is its declarativeness.
It describes a list of objects which satisfy
certain criteria, and you don't have to think
about how they are generated.
Putting in something like "while" gives it a
procedural flavour - you have to know things
like the fact that the elements are generated
in a certain order to know what it means.
Also it constrains the implementation to
generating them in that order.
Greg Ewing
>
> BTW, I double-checked today, and at least Haskell doesn't appear to have
> grown special syntax for "lockstep parallel iteration"
Regarding parallel iteration, I'd like to point
out that if it is incorporated into this list-builder
notation, it should *also* be added to the procedural
for-loop syntax as well!
Now, in response to the criticisms that:
a) comma versus semicolon for parallel versus nested
is too error-prone
b) filtering is needed
my current proposal for the list-builder syntax looks
like this:
[x*y+z for x in L1 for y in L2, z in L3 where x+y<z]
which translates into
result = []
for x in L1:
for y in L2, z in L3:
if x+y<z:
result.append(x*y+z)
Or perhaps it should all be turned around the other
way:
[for x in L1: for y in L2, z in L3: where x+y<z: x*y+z]
which might make the nesting structure easier to see.
On the other hand, it buries the result expression at
the end. Can't have everything, it seems...
Tim Peters
> If some of the "in" clauses are separated by commas instead
> of semicolons, the relevant list traversals are parallel instead
> of nested.
[Dirk Heise]
> that's too eror-prone, why not:
>
> L = x*y for x,y in L1,L2
>
> for the "parallel" evaluation?
I agree comma vs semicolon is too subtle, but for better or worse
for x, y in L1, L2
already has a different meaning in Python, & so it would be a Bad Idea to
make it mean something else inside [].
> (BTW: Excuse me, but personally i have the feeling that i need
> code like that in 1 of 10000 lines of code so i don't think it
> makes the language remarkably better. just more difficult to learn.)
Unclear whether you're referring to list comprehensions or to parallel
iteration (or both?). I think listcomps would be a good addition to the
language, but that parallel iteration should be supported in them if and
only if for loops in general grow a way to spell them too. Short of that,
there's always
for x, y in map(None, L1, L2)
Except it's revolting <wink>.
not-that-a-",||"-pseudo-operator-would-be-any-better-ly y'rs - tim
Tim Peters
> ...
> which doesn't look like much until you replace that "if" with
> something else (say, "while" to prematurely terminate):
>
> [j*j for j in range(1,11): while j < 5]
[Greg Ewing]
> Noooooo..... this is going to far......
I more than half agree <0.87 agree>.
> One of the nicest things about the idea as I
> first proposed it is its declarativeness.
> It describes a list of objects which satisfy
> certain criteria, and you don't have to think
> about how they are generated.
>
> Putting in something like "while" gives it a
> procedural flavour - you have to know things
> like the fact that the elements are generated
> in a certain order to know what it means.
> Also it constrains the implementation to
> generating them in that order.
But it's generating a list, and lists are ordered -- it's far less useful if
the list can end up in some random order, and, since Python is not free of
side-effects, ordering guarantees can be crucial to getting correct results
regardless of order.
All languages I've heard of that support listcomps specify the iteration
ordering rigidly; even Haskell, which has no side effects; and even set
comprehensions in SETL, despite that it's generating an unordered set (so
that you can rely on e.g. "i" being bound in (j: i in S, j in f(i)}, and
keep things straight if f has side-effects).
That's not to say anyone is forbidden from writing side-effect free
iterators and guards, or not caring about the output order. But as a clean
alternative to lambda-based map and filter (which appears to be its
strongest point, and on at least the map basis of which you originally
pitched it), it's unsellable without ordering guarantees for those who do
care.
orderedly y'rs - tim
Marc Wachowitz
> All languages I've heard of that support listcomps specify the iteration
> ordering rigidly
Well, the Scheme specification explicitly says that the order of calls with
the standard MAP function is unspecified (but it is defined with FOR-EACH,
which, on the other hand, doesn't have any specified results).
> But as a clean
> alternative to lambda-based map and filter (which appears to be its
> strongest point, and on at least the map basis of which you originally
> pitched it), it's unsellable without ordering guarantees for those who do
> care.
The published description of Python doesn't say anything about the order
in which the built-in map function calls its first argument with elements
of the remaining argument sequences (library manual, secion 2.3). The same
is true for filter and reduce.
Of course, as Python isn't sold, but comes for free, one might say that
your sentence isn't really relevant in this case ;-)
Guido van Rossum
> in which the built-in map function calls its first argument with elements
> of the remaining argument sequences (library manual, secion 2.3). The same
> is true for filter and reduce.
This mayt be literally true, but it must to iterate over the elements
from index 0 up; otherwise it won't be able to reliably detect the end
of variable-length sequences (which must be discovered by trying until
__getitem__() raises IndexError).
So Python does come with a guarantee. Because it's free, we couldn't
afford to spell it out in the docs. But it's a guarantee nevertheless.
Kevin Digweed
First off, I think you missed the point of my post. I was trying to say that
I would prefer a keyword to separate the filter from the iterators rather than
punctuation. That way, it is clean and easy to introduce new "styles" of filter
without cryptic use of alternative punctuation. Even if it's never capitalized
on, it's still better to leave the syntax "open", in my opinion.
The use of "while" was more example than proposal, but more on that later ;)
Forgive me for responding to your points out-of-order:
> Also it constrains the implementation to
> generating them in that order.
I'll assume that you mean the *mechanics* of the implementation can differ
(ie, the order of iteration over a given source list) rather than that the
result of the operation could be reversed (or otherwise "unsorted") ...
There is a problem with this if you allow the "for" loop to iterate over any
sequence that the current Python "for" loop can iterate over. If you call
anything which has a side-effect (xrange() is the obvious example,
fileinput.input() is another) then you cannot iterate over the sequence in
any order but first->last without introducing a confusing semantic difference
between the current "for" and this new one, eg:
[ l[:5] for l in fileinput.input() ]
I believe that you *do* see a symmetry between the current Python "for" and
this new one, given one of your other posts (regarding parallel iteration).
Otherwise, I'd suggest using something other than "for" ...
> Putting in something like "while" gives it a
> procedural flavour - you have to know things
> like the fact that the elements are generated
> in a certain order to know what it means.
If you agree that by allowing calls to things with side-effects the
implementation pretty much has to guarantee a first->last iteration anyway,
then this is less of an issue. Indeed, I'd say that given the following ...
[ x for x in [0, 1, 2, 3] ]
... the "default" expectation (that which appears natural or obvious) of the
casual user would be that the list is iterated over first->last. Therefore
if the list could be iterated over in reverse order, *that* would be
something you would "have to know" (and would likely become a newbie gotcha).
Besides, there's no harm in exploring all the possibilities, no matter how evil
they seem (you never know, the path to the perfect solution may be on the other
side of a discussion about such abominations as these).
> Noooooo..... this is going to far......
"No" would have done ...
> One of the nicest things about the idea as I
> first proposed it is its declarativeness.
I agree that your proposal is a much nicer way to map() and filter(), I was
just tossing something else into the pot. It might be a nicer way of doing
some other stuff, too.
Better to consider 50 different possibilites and chuck out 45 than to consider
5 and chuck out 4 - assuming all the ones you keep are worth keeping, of
course ;)
Cheers, Kev.
Kevin Digweed
>> My initial thought when someone mentioned filtering was something like:
>>
>> [j*j for j in range(1,11) if j % 2 == 0]
>>
>> ... however I like the idea that the for loop has a more consistent
>> syntax in your example.
[Tim]
>I'm not sure I do -- syntactic congruence can be misleading when semantics
>differ, and in this case the colon doesn't "mean" what it usually means
>after a "for" -- it's there just to let the parser distinguish the
>expression_list from the guard
OK - my point was that I thought a keyword was better than punctuation. Taking
the colon out puts it back to what I preferred, so I won't complain ;)
>OTOH, at this point there's so much stuff going on that an explicit for loop
>is likely as clear overall:
I pretty much agree with that, as things stand. Maybe that's a good enough
reason to not bother with multiple filters (at least initially, and especially
if nobody can think of any good ones ;)).
Cheers, Kev.
BTW: The subject line is misleading. This isn't an alternative to lambda,
it's an alternative to map() and filter() which reduces the need for a
lambda ...
Greg Ewing
>
> but it must to iterate over the elements
> from index 0 up; otherwise it won't be able to reliably detect the end
> of variable-length sequences
It could do a binary search between 0 and maxint to
locate the end point, then treat elements within
the range in arbitrary order.
Guido van Rossum
> > but it must to iterate over the elements
> > from index 0 up; otherwise it won't be able to reliably detect the end
> > of variable-length sequences
>
> It could do a binary search between 0 and maxint to
> locate the end point, then treat elements within
> the range in arbitrary order.
You didn't quote me completely. I added "Python comes with a
guarantee," and added an explanation why the guarantee wasn't
documented. If anyone else had said that, you'd be right to question
the reasoning. But since I said it, you can believe the guarantee.
Tim Peters
> OK - my point was that I thought a keyword was better than
> punctuation. Taking the colon out puts it back to what I preferred,
> so I won't complain ;)
Then I will, and bitterly! Ah, OK, no I won't -- we're agreeing. That's no
fun.
> ...
> Maybe that's a good enough reason to not bother with multiple
> filters (at least initially, and especially if nobody can think
> of any good ones ;)).
I do at least want to retain a (optional) filter on each iterator, though,
as opposed to allowing for only one at the very end. That's more
Haskell-like than SETL-like, and in the absence of a gonzo optimizer allows
for greater efficiency. E.g.,
heavy_waldorf_salad_dressing_cross_product = \
[(s, d) for s in salads if s.is_category("Waldorf")
for d in s.dressings() if d.is_heavy()]
is likely to run much quicker than
[(s, d) for s in salads for d in s.dressings()
if s.is_category("Waldorf") and d.is_heavy()]
in a world with far too many kinds of salad.
> BTW: The subject line is misleading. This isn't an alternative to lambda,
> it's an alternative to map() and filter() which reduces the need for a
> lambda ...
It's more than that, though, as heavy_waldorf_salad_dressing_cross_product
is a strain to compute with lambdas, maps *or* filters.
OTOH, for the Scheme folk *everything* is an alternative to lambda, so we
may have hit on a perfect all-purpose subject line <wink>.
prunes-you-say?-oh-yes-they're-just-wrinkled-purple-lambdas-ly y'rs - tim
Tim Peters
> All languages I've heard of that support listcomps specify the
> iteration ordering rigidly
[Marc Wachowitz]
> Well, the Scheme specification explicitly says that the order of
> calls with the standard MAP function is unspecified
1. map != list comprehension, though; the latter introduces names (usually
local, although that point is lost in Python <wink>) whose bindings change
within the construct; specifying the iteration ordering is useful because of
this; Scheme's map doesn't have to deal with that at all (*Python's* map
does, though, because of Python's index-based __getitem__ protocol and that
map applies to any sequences).
2. Although Scheme's map doesn't specify the order of calls, it does require
strict "left to right" output order. The obvious way to implement that is
via strict left to right iteration -- although I suppose there may be a
research project in Karlsruhe that's still trying to recover their
investment in a discontinued parallel machine <wink>.
So I don't see Scheme's map as being relevant here, and, if it is, I see no
reason to cripple Python too <wink>. The only reasons for *not* specifying
the order are:
1) To ease life for implementations exploiting fine-grained parallelism (if
order doesn't matter, "all at once" is permissible). If that ever becomes
important to Python, I'll write a parallelized pmap function for free -- and
it will be the only part of the language that does get parallelized <0.7
wink>.
2) To punish impure users for employing side-effects.
#2 is, of course, most attractive <wink>.
> ...
> The published description of Python doesn't say anything about the
> order in which the built-in map function calls its first argument
> with elements of the remaining argument sequences (library manual,
> secion 2.3). The same is true for filter and reduce.
Guido didn't get around to writing the usual "order of evaluation"
boilerplate anywhere (e.g., I don't the think the order in [f(), g(), h()]
is specified, or even in f() + g()]. The implementation is generally
left-to-right, though, and most users would be happy to bet your life it
will stay that way <wink>. Within the last month I noted (on the newsgroup)
that
{makekey(): makevalue()}
calls makevalue before makekey, and Guido indicated he would probably change
that someday.
Since all signs point to the lack of order-of-eval specs being an oversight
due to lack of time and/or native language-lawyer inclination on Guido's
part, I think specifying it up front for listcomps is doing everyone a
favor.
might-even-get-the-general-boilerplate-written-as-a-side-effect-ly y'rs -
tim
Tim Peters
> ...
> but it must to iterate over the elements from index 0 up; otherwise
> it won't be able to reliably detect the end of variable-length
> sequences
[Greg Ewing]
> It could do a binary search between 0 and maxint to
> locate the end point, then treat elements within
> the range in arbitrary order.
What purpose does this serve? Besides sheer goofiness, it would break any
number of existing sequence implementations, which rely on the protocol in
use today to such an extent that they don't even look at the index passed to
them. See fileinput.py in the std library for a popular example of a
sequence that looks at the index only long enough to puke if it's not
exactly "the next one". Sequences are often used to implement what other
languages sometimes call streams, where neither backing up nor peeking ahead
are allowed.
and-that's-the-news-from-earth<wink>-ly y'rs - tim
PS: Even in logic-chopping-for-perversity's-sake mode, no rule says a
sequence must be finite in extent, let alone bounded by sys.maxint. The way
the for/__getitem__ protocol works today, at least in CPython if the index
exceeds the size of a native C signed long, it simply wraps around
to -sys.maxint-1 and keeps going. I know this because I have __getitem__
code that's been called that often in a single run, and had to make sure it
wouldn't break.
Those who don't want to wait that long <wink> may enjoy running this
monstrosity, which should amuse on a 32-bit platform:
class MySeq:
def __init__(self, todo=20):
self.todo = todo
def __getitem__(self, i):
self.todo = self.todo - 1
if self.todo < 0:
raise IndexError
else:
return i
def doit():
for i in MySeq():
print i
c = doit.func_code
import string
attrs = string.split("""argcount nlocals stacksize flags
code consts names varnames filename name
firstlineno lnotab""")
args = []
for m in attrs:
args.append(getattr(c, "co_" + m))
args[5] = (None, int((1L << 31) - 10))
import new
c2 = apply(new.code, tuple(args))
exec c2
Kevin Digweed
>I do at least want to retain a (optional) filter on each iterator, though,
>as opposed to allowing for only one at the very end.
Yes, that looks useful ...
>heavy_waldorf_salad_dressing_cross_product = \
> [(s, d) for s in salads if s.is_category("Waldorf")
> for d in s.dressings() if d.is_heavy()]
>is likely to run much quicker than
> [(s, d) for s in salads for d in s.dressings()
> if s.is_category("Waldorf") and d.is_heavy()]
... but that's only likely if there are also guarantees on the order of
evaluation - ie, if the first "for" is guaranteed to be the outermost loop,
you can make sure that it contains what you expect to be the main input pruner.
I don't think this was touched upon in the other discussion on ordering - that
was about the ordering of iteration within a source list.
In addition, it would need to be decided what happens during parallel iteration
with unequal length source lists - map() None-pads the shorter ones, which I
guess should be brought forward (by default, at least), but will that place too
much overhead on the now-Python-speed loop if it has to start catching
exceptions ? Maybe some clever code generation would minimise this, if
neccessary.
Cheers, Kev.
Marc Wachowitz
> This mayt be literally true, but it must to iterate over the elements
> from index 0 up;
Well, strictly speaking, it could still create a new standard list of all the
elements, then replace each element of that list with the result of calling
the mapping function in an arbitrary order, and return that list. However,
I admit that I wouldn't expect you to implement something like this without
extremely good reasons. Let's say Python comes with a design sanity guarrantee
which is documented on the net ;-)
Guido van Rossum
> which is documented on the net ;-)
I like this idea ;-)
Amit Patel
|
| I do at least want to retain a (optional) filter on each iterator, though,
| Haskell-like than SETL-like, and in the absence of a gonzo optimizer allows
| for greater efficiency. E.g.,
|
| [(s, d) for s in salads if s.is_category("Waldorf")
| for d in s.dressings() if d.is_heavy()]
|
| is likely to run much quicker than
|
| [(s, d) for s in salads for d in s.dressings()
| if s.is_category("Waldorf") and d.is_heavy()]
|
|
Can this [] construct be nested? If so, you might be able to use the
end-filter to achieve the same result:
[(s, d) for s in [s for s in salads if s.is_category("Waldorf")]
for d in [d for d in s.dressings() if d.is_heavy()]]
- Amit
Amit Patel
[snip - Scheme doesn't specify the order of calls in 'map']
| So I don't see Scheme's map as being relevant here, and, if it is, I see no
| reason to cripple Python too <wink>. The only reasons for *not* specifying
| the order are:
|
| 1) To ease life for implementations exploiting fine-grained parallelism (if
| order doesn't matter, "all at once" is permissible). If that ever becomes
| important to Python, I'll write a parallelized pmap function for free -- and
| it will be the only part of the language that does get parallelized <0.7
| wink>.
|
| 2) To punish impure users for employing side-effects.
|
| #2 is, of course, most attractive <wink>.
It also happens to punish users for using exceptions and
continuations. Oops. :)
- Amit
Tim Peters
> Well, strictly speaking, it could still create a new standard
> list of all the elements, then replace each element of that
> list with the result of calling the mapping function in an
> arbitrary order, and return that list. However, I admit that I
> wouldn't expect you to implement something like this without
> extremely good reasons.
But that's exactly what Guido did! It's just that he picked the identity
permutation at the end-- on the grounds that it's as arbitrary as
anything --and then optimized the implementation by not bothering to
generate the whole list first <wink>.
whenever-you-have-a-free-choice-of-f-pick-f(x)=x-ly y'rs - tim
Tim Peters
>> I do at least want to retain a (optional) filter on each
>> iterator, though, as opposed to allowing for only one at
>> the very end.
[Kevin Digweed]
> Yes, that looks useful ...
>> heavy_waldorf_salad_dressing_cross_product = \
>> [(s, d) for s in salads if s.is_category("Waldorf")
>> for d in s.dressings() if d.is_heavy()]
>>
>> is likely to run much quicker than
>>
>> [(s, d) for s in salads for d in s.dressings()
>> if s.is_category("Waldorf") and d.is_heavy()]
> ... but that's only likely if there are also guarantees on the
> order of evaluation - ie, if the first "for" is guaranteed to be
> the outermost loop, you can make sure that it contains what you
> expect to be the main input pruner. I don't think this was
> touched upon in the other discussion on ordering - that
> was about the ordering of iteration within a source list.
I've posted my proposed translation into today's Python a few times already,
with the intent that it be taken literally, not suggestively. This nests
"for" loops with the leftmost outermost, so nails everything about the
ordering semantics at all levels. Why that's become a debating point at all
levels is beyond me <wink -- but there simply is no credible argument in
favor of leaving anything about it undefined>.
In particular, inner iterators need to be able to refer to bindings
established by outer iterators ("for d in s.dressings()"), so ordering of
loops is a must, and it's in this sense that "map" has no equivalent.
> In addition, it would need to be decided what happens during parallel
> iteration with unequal length source lists -
My flavor of listcomp doesn't support parallel iteration, except that in the
one-iterator case, parallel is the same as cross-product iteration. So for
more than one iterator, this flavor of listcomp is not a substitute for map
short of pulling the
[x+y+z for x, y, z in map(None, L1, L2, L3)]
trick. However, if Python grows a cleaner way to spell "parallel iteration"
in regular "for" loops, this flavor of listcomp would certainly play along.
> map() None-pads the shorter ones, which I guess should be brought
> forward (by default, at least), but will that place too much
> overhead on the now-Python-speed loop if it has to start catching
> exceptions ? Maybe some clever code generation would minimise this, if
> neccessary.
Presumably if Python grew a parallel iteration syntax on regular "for"
loops, a supporting function in ceval would be written to do "a step" at C
speed. The problem with parallel iteration has always been a mix of scant
demand and lack of good syntax (e.g. "for x, y in X, Y" and "for x in X, y
in Y" are both out because they both mean something already -- but that
doesn't stop them from getting suggested every time this comes up <0.7
wink>).
If listcomps are to be a general alternative to map, though, some form of
parallel iteration is needed. But I've got nothing against map myself, so
I'm not likely to push for that <0.9 wink>.
BTW, map None-padding short sequences sucks (almost never helps and tends to
delay errors), but it may not be possible to change that anymore.
none-is-a-four-letter-word-ly y'rs - tim
Tim Peters
> I do at least want to retain a (optional) filter on each
> iterator, though, as opposed to allowing for only one at
>
> heavy_waldorf_salad_dressing_cross_product = \
> [(s, d) for s in salads if s.is_category("Waldorf")
> for d in s.dressings() if d.is_heavy()]
[Amit Patel for Amit Patel in [Amit Patel]]
> Can this [] construct be nested?
It must, and for the same reason functions and classes nest in Python: it's
too much of a pain to change the grammar to disallow it <0.99 wink>.
> If so, you might be able to use the end-filter to achieve the
> same result:
>
> [(s, d) for s in [s for s in salads if s.is_category("Waldorf")]
> for d in [d for d in s.dressings() if d.is_heavy()]]
Yes, and you could simplify the second iterator to what it was in the
original example.
Then, looking at both, which would *you* prefer to write? Which is likely
to run faster? Which is clearer?
If you're unaccountably inclined to give the wrong answer <wink>, move on to
abstract = [x+y+z for x in unit for y in unit if x**2+y**2 < 1
for z in unit if z < abs(x-y)]
and answer again after replacing the first two iterators with
for x, y in [(x, y) for x in unit
for y in unit if x**2+y**2 < 1]
int-literals-other-than-1-are-an-extravagance-ly y'rs - tim
Kevin....@dial.pipex.com
"Tim Peters" <tim...@email.msn.com> wrote:
> Presumably if Python grew a parallel iteration syntax on regular "for"
> loops, a supporting function in ceval would be written to do "a step" at C
> speed.
> The problem with parallel iteration has always been a mix of scant
> demand and lack of good syntax (e.g. "for x, y in X, Y" and "for x in X, y
> in Y" are both out because they both mean something already
Hmmmm. In that case, I'll suggest (tentatively, 'cause I didn't think about it
for very long) "and" :
for x in L1 and y in L2: print (y, x)
[(x, y, z) for x in L1 and y in L2 if x+y < 10
for z in L3 if z > 0]
alternatively, "for x in L1 and for y in L2:", but I don't think that's
neccessary (and too wordy).
A speedy scan of the docs leads me to believe that this wouldn't be ambiguous,
and it's better than punctuation IMHO.
As for the demand, I promise to use it even when I don't need to ;)
> BTW, map None-padding short sequences sucks (almost never helps and tends to
> delay errors), but it may not be possible to change that anymore.
I've never taken advantage of it, so can't comment. I just guessed that the
"backwards compatible" option would have to be the default regardless of how
it would have been preferred. But having said that, this is a new operation
and doesn't *have* to be backwards compatible with anything. It would just
mean that you couldn't blindly change any map()s over (so leave them alone -
map()'s not going anywhere).
Cheers, Kev.
-----== Posted via Deja News, The Leader in Internet Discussion ==-----
http://www.dejanews.com/rg_mkgrp.xp Create Your Own Free Member Forum
Tim Peters
> The problem with parallel iteration has always been a mix of scant
> demand and lack of good syntax (e.g. "for x, y in X, Y" and
> "for x in X, y in Y" are both out because they both mean something
> alternative to map, it's needed anyway]
> Hmmmm. In that case, I'll suggest (tentatively, 'cause I didn't
> think about it for very long) "and" :
>
> for x in L1 and y in L2: print (y, x)
Ah, you're making life too hard! There's no need to think <0.5 wink>. Plug
that line into Python today, and you don't get a syntax error. Which means
Python already ascribes a meaning to it -- you may not know *what* meaning
offhand <0.9 wink>, but for purposes of syntax suggestions it's enough to
know that it's dead on arrival.
The best I ever dreamed up is
for x in L1 || y in L2:
print (y, x)
where the "parallel lines" at least suggest parallel iteration. Since it
raises a syntax error today, it's feasible. Maybe "&&" would be more
suggestive. OTOH, && and || have "do this *then* maybe that" and "do this
*or* maybe that" meanings in C and assorted shells, so it doesn't have much
to recommend it beyond being illegal today.
> alternatively, "for x in L1 and for y in L2:", but I don't think
> that's neccessary (and too wordy). A speedy scan of the docs leads
> me to believe that this wouldn't be ambiguous,
It does pass the "blows up today" test. Maybe we should replace "and for"
with "lambda", though <wink>.
> and it's better than punctuation IMHO.
"also" would be a nice connector here,
for x in L1 also y in L2:
and especially with the other fabled lost "for" extension
for i indexing x in L1 also y in L2:
But introducing new keywords is effectively impossible; although given their
possible uses in "for" loops no ambiguity is possible even if they weren't
keywords; e.g. the user vrbl named "also" is referenced 8 times here
for also, also in also, also also also, also in also, also:
and while that's stupid it's easily parseable (maybe in a parser other than
Python's, though <0.7 wink>).
> As for the demand, I promise to use it even when I don't need to ;)
I'd use it if it were there, but that doesn't count for much because I even
use things that aren't there <wink>.
> [about map doing None-padding]
> ...
> But having said that, this is a new operation and doesn't *have* to
> be backwards compatible with anything. It would just mean that you
> couldn't blindly change any map()s over (so leave them alone -
> map()'s not going anywhere).
So what *should* it do with a short sequence? That (however it's spelled)
for x1 in L1 also x2 in L2 also ... also xn in Ln:
stops as soon as some Li raises IndexError is clear enough. If i > 1,
should it raise some "sequence length mismatch" exception? And if i == 1,
should it go on to probe L2 thru Ln to make sure they all raise IndexError
too? Or should it just plain stop at Li? That e.g.
for i in xrange(1, sys.maxint) also thing in Producer():
print "thing number", i, "is", thing
isn't obviously insane makes me leery about griping. OTOH, once the loop
ends "i" would be bound to an int one larger than the number of things
produced, and that may be surprising.
sometimes-tough-luck-is-the-only-luck-there-is-ly y'rs - tim
Kevin Digweed
[on parallel iteration]
>>> [but if list comprehensions are to be a general alternative to map, it's
>>> needed anyway]
>> for x in L1 and y in L2: print (y, x)
>Plug that line into Python today, and you don't get a syntax error.
Of course ... get Python to read the docs for me! ;)
> for x in L1 || y in L2:
> print (y, x)
To me, that has too much baggage associated with it. Coming from a C/C++
background, I can cope with "or" meaning || and "and" meaning &&, but if you
now make || mean something different, even if it's only within a for loop
declaration, things start feeling a little uncomfortable. I have to
double-check the docs nearly every time I use REXX because of the meanings
of ||, |, &, =, ==, ~=, ~== etc - even for the ones which are basically the
same as C, simply because I remember that *some* of them aren't.
I remember I get bitten by it, so I always have to get the manual out and
check - and that counts as another bite.
Maybe that's just because I don't use REXX enough <shudder!> ;)
>> alternatively, "for x in L1 and for y in L2:"
>It does pass the "blows up today" test. Maybe we should replace "and for"
>with "lambda", though <wink>.
I'm not sure I really like that one though, 'cause the extra "for" makes it
look a little too much like nested loops (especially in the proposed list
comprehensions where consensus seems to be that there should be no trailing
colon):
[ (x, y) for x in L1 for y in L2 ]
[ (x, y) for x in L1 and for y in L2]
OK, probably my parting shot here is the semi-colon:
[ (x, y) for x in L1; y in L2 ]
for x in L1; y in L2: print x, y
It gives an error today and implies connectivity (both in written English
and Python syntax). I'm interested to hear the reason why that one can't be
used, either ;)
>So what *should* it do with a short sequence? That (however it's spelled)
> for x1 in L1 also x2 in L2 also ... also xn in Ln:
>stops as soon as some Li raises IndexError is clear enough. If i > 1,
>should it raise some "sequence length mismatch" exception? And if i == 1,
>should it go on to probe L2 thru Ln to make sure they all raise IndexError
>too? Or should it just plain stop at Li? That e.g.
> for i in xrange(1, sys.maxint) also thing in Producer():
> print "thing number", i, "is", thing
>isn't obviously insane makes me leery about griping.
If it were to just stop at Li, you found your lost indexing "for":
for i in xrange(0, sys.maxint); x in L1; y in L2:
> OTOH, once the loop ends "i" would be bound to an int one larger than the
> number of things produced, and that may be surprising.
Or useful:
unique = 0
for Ln in (L1, L2, L3):
for unique in xrange(unique, sys.maxint); element in Ln:
element.SetID(unique)
Of course,
for thing in Producer(); i in xrange(1, sys.maxint):
print "thing number", i, "is", thing
... would not have that side-effect. I'm warming to the "plain stop at Li"
semantic as being the better of the two, though not perfect, as it's easier
to code the other behaviour than it would be the other way around.
If you really need the lists to be of the same length, then either check
their lengths before entering the loop or, in the case of a generator, do
the xrange() indexing trick and probe the next index on the way out. Doesn't
help the [] version though, 'cause there's no scope for putting additional
code in the loops - in which case, hand-rolling the loop and building the
list by hand isn't so hard if the regular style "for" supports parallel
iteration, anyway.
Cheers, Kev.
Greg Ewing
Nope, that already has a meaning too ("L1 and y in L2"
is a syntactically valid expression).
> alternatively, "for x in L1 and for y in L2:", but I don't think that's
> neccessary (and too wordy).
It has the advantage of being non-ambiguous, though!
Wait, I think I feel a solution coming on... (screws
up eyes and thinks very hard...)
How about this:
for x and y in L1, L2:
The full syntax is:
'for' target ('and' target)* 'in' expr ':' suite
When the 'and' form is used, the expr must evaluate
to a tuple of sequences, and the iteration is performed
in parallel over those sequences.
> it would have been preferred. But having said that, this is a new operation
> and doesn't *have* to be backwards compatible with anything.
Indeed. I'd vote for making it an error if the lists
are of different lengths. If you want some particular
behaviour with unequal length lists, you adjust them
appropriately beforehand - pad, truncate, whatever.
Greg Ewing
>
> Can this [] construct be nested?
It's an expression, so yes, but...
> If so, you might be able to use the
> end-filter to achieve the same result:
>
> [(s, d) for s in [s for s in salads if s.is_category("Waldorf")]
> for d in [d for d in s.dressings() if d.is_heavy()]]
Not much point in using nesting for that - obfuscates
the expression and makes it less efficient (building
intermediate lists) unless some fancy optimisation is
done.
Although my last example only had a guard at the end,
I didn't mean to imply that only an end guard would
be allowed. Order of nesting should certainly be
specified, and iterators and guards should be freely
interminglable (what a fun word!)
Putting it all together:
list_builder_expr ::= '[' expr (iterator | guard)+ ']'
iterator ::= 'for' target ('and' target)* 'in' expr
guard ::= 'if' expr
Greg Ewing
>
> If listcomps are to be a general alternative to map, though, some form of
> parallel iteration is needed. But I've got nothing against map myself, so
> I'm not likely to push for that <0.9 wink>.
My original idea was that they should be usable
as a replacement for map-with-lambda. I've got
nothing against map itself, either, but combining
it with lambda seems to lead to hard-to-read
code, to my eyes. I'd be disappointed if this
feature were introduced without making it as
capable as map.
Detlef Lannert
: Amit Patel wrote:
:>
:> Can this [] construct be nested?
: It's an expression, so yes, but...
:> If so, you might be able to use the
:> end-filter to achieve the same result:
:>
:> [(s, d) for s in [s for s in salads if s.is_category("Waldorf")]
:> for d in [d for d in s.dressings() if d.is_heavy()]]
: Not much point in using nesting for that - obfuscates
: the expression and makes it less efficient (building
: intermediate lists) unless some fancy optimisation is
: done.
: Although my last example only had a guard at the end,
: I didn't mean to imply that only an end guard would
: be allowed. Order of nesting should certainly be
: specified, and iterators and guards should be freely
: interminglable (what a fun word!)
: Putting it all together:
: list_builder_expr ::= '[' expr (iterator | guard)+ ']'
: iterator ::= 'for' target ('and' target)* 'in' expr
: guard ::= 'if' expr
Even if the feature is nice, I'd dislike it if it introduces too many
new syntactic elements. Statements with lots of '[]()'... (for simple
things) are getting close to the P-language ;-).
Has this already been proposed?
for s in sizes, for t in toppings:
price = s * t
...
If yes, I'd vote for it ... [Syntactically it should work because "for"
is a reserved word and currently illegal after the ','.]
For my taste the "and" has too much of a "boolean smell" to it.
Sorry, I haven't grasped the use of the "guards" yet, but I've missed a
few postings. If I want to select certain elements of one of the lists,
I can use filter(); yes, it may need a lambda, but it's also not very
useful here anyway because the lists would get out of sync. If I want to
select tuples, a simple
if t > s: continue
within the loop does the job, doesn't it? (Apologies again, I don't want
to rehash some earlier argument.)
Detlef
Chris Wright
loop macro of Common Lisp. That follows the general approach of CL (do
EVERYTHING, EVERYWAY - I mean, output format conversion to Roman numerals is
part of the language!). I'm not sure that's the stylistic path we want to go
down (cop out, not being really sure if we're talking semantics or syntax
here :-))
An example
(loop for i from 1 to (compute-top-value)
while (not (unacceptable i))
collect (square i)
do (format t "Working on ~D now" i)
when (evenp i)
do (format t "~D is a non-odd number" i)
finally (format t "About to exit!"))
(From Common Lisp, the Language - Steele )
which, by the way, has the best index entry I have seen in a 971 page
book:
under K
........
........
kludges 1-971
........
........
Tim Peters
> It seems as if some of these examples have been getting very
> close to the loop macro of Common Lisp.
No -- we're trying to squash everything onto one line without new
keywords -- Common Lisp is readable <0.94 wink>.
Seriously, in designing a new feature you have to worry about how it will
work in extreme cases -- even if they appall you. You can bet your
patients' lives Guido worried about whether crap like
a[a[a[a[i]+a[i+a[i]]]]+a[a[a[i]+a[a[i]+i]]]]
would work (hint: it does <wink>). An odd thing about implementing
languages is that picturing the most convoluted cases usually suggests an
approach that makes the simple cases very reliable. And when's the last
time you saw a code-generation error in Python? I can't think of any ever.
Constrast with Perl, where the most convoluted cases are beyond human
comprehension <0.7 wink>.
suspect-that-with-ivan-around-we're-gonna-get-mayan-formats-before-
roman-numerals-ly y'rs - tim
Tim Peters
> My original idea was that [list comprehensions] should be usable
> as a replacement for map-with-lambda. I've got nothing against
> map itself, either, but combining it with lambda seems to lead
> to hard-to-read code, to my eyes.
Oh yes -- to my eyes too!
> I'd be disappointed if this feature were introduced without
> making it as capable as map.
Same here, but not as much. grepping through reams of Python turned up many
uses of map, but few with more than one sequence argument -- and half of the
latter were of the dreadful (but fast)
map(operator.__getitem__, [mapper]*len(indices), indices)
form, handled naturally by the one-sequence
[mapper[i] for i in indices]
flavor of listcomp. So if we could only get one flavor of multiple
iteration, I'd vote for cross-product iteration (which opens up pleasant
ways to spell all sorts of things that today's map can't handle naturally at
all). Parallel iteration would be nice too, but, frankly, all syntax
suggestions so far-- short of a new keyword (or pseudo-keyword in
context) --look strained to me.
yet-guido-works-in-mysterious-ways-ly y'rs - tim
Greg Ewing
> pseudo-keyword in context) --look strained to me.
Yeah. I'd actually be in favour of a non-backward-compatible
change so that "for x,y in L1,L2" actually does what you
expect. How much code do you think there is out there that
relies on the current interpretation of that?
Suspecting-non-parenthesised-tuple-constructor-expressions-were-a-big-mistake,
Greg
Rainer Joswig
<c...@belloc.med.monash.edu.au> wrote:
> It seems as if some of these examples have been getting very close to the
> loop macro of Common Lisp.
How about this gem of Common Lisp:
? (setf xs '(1 2 3 4 5 6 7 8))
? [x (x <- xs) (oddp x)]
-> (1 3 5 7)
Or define QSORT:
(defun qsort (ax)
(and ax
(let ((a (car ax))
(x (cdr ax)))
(append (qsort [y (y <- x) (< y a)])
(list a)
(qsort [y (y <- x) (>= y a)])))))
? (qsort '(6 2 7 3 5 1 98 44 22 31 555))
-> (1 2 3 5 6 7 22 31 44 98 555)
The implementation ;-) :
;------------------
(defmacro comp ((e &rest qs) l2)
(if (null qs)
`(cons ,e ,l2)
(let ((q1 (car qs))
(q (cdr qs)))
(if (not (eq (cadr q1) '<-))
`(if ,q1
(comp (,e . ,q) ,l2) ,l2)
(let ((v (car q1))
(l1 (third q1))
(h (gensym "H-"))
(us (gensym "US-"))
(us1 (gensym "US1-")))
`(labels ((,h (,us)
(if (null ,us)
,l2
(let ((,v (car ,us))
(,us1 (cdr ,us)))
(comp (,e . ,q) (,h ,us1))))))
(,h ,l1)))))))
(defun open-bracket (stream ch)
(do ((l nil)
(c (read stream t nil t) (read stream t nil t)))
((eq c '|]|)
`(comp ,(reverse l) ()))
(push c l)))
(defun closing-bracket (stream ch)
'|]|)
(set-macro-character #\[ #'open-bracket)
(set-macro-character #\] #'closing-bracket)
;------------------
Or if you want miranda pattern matching
style function definitions:
(require 'select-match)
;;; wer's braucht:
(defmacro miranda (function-name (arg) &body forms)
`(defun ,function-name (,arg)
(select-match ,arg . ,forms)))
(miranda perms (x)
() => '(())
(_ . _) => [(cons a p)
(a <- x)
(p <- (perms (remove a x :count 1)))])
? (perms '(2 3 4 5))
-> ((2 3 4 5) (2 3 5 4) (2 4 3 5) (2 4 5 3) (2 5 3 4) (2 5 4 3) (3 2 4 5)
(3 2 5 4) (3 4 2 5) (3 4 5 2) (3 5 2 4) (3 5 4 2) (4 2 3 5) (4 2 5 3) (4 3
2 5) (4 3 5 2) (4 5 2 3) (4 5 3 2) (5 2 3 4) (5 2 4 3) (5 3 2 4) (5 3 4 2)
(5 4 2 3) (5 4 3 2))
M.-A. Lemburg
>
> In article <6retop$mtl$1...@towncrier.cc.monash.edu.au>, "Chris Wright"
> <c...@belloc.med.monash.edu.au> wrote:
>
> > It seems as if some of these examples have been getting very close to the
> > loop macro of Common Lisp.
>
> How about this gem of Common Lisp:
If there really is such a strong need to write these things with
a new syntax, why doesn't anyone show some examples of interpreting
functions, that is, functions which compile and execute given
strings (using some imaginary syntax) in the caller's namespace ?
It's not too hard to do...
Then you can write
lc('[mapper[i] for i in indices]')
and it will do whatever you want it to. And if you really care for these
constructs in Python, why not add an import wrapper or a source
filtering hook that'll do the necessary compilation to normal Python
code on-the-fly ?
...or maybe we just need a CL-binding ;-)
--
Marc-Andre Lemburg Y2000: 497 days left
---------------------------------------------------------------------
: Python Pages >>> http://starship.skyport.net/~lemburg/ :
---------------------------------------------------------
M.-A. Lemburg
>
> > all syntax suggestions so far-- short of a new keyword (or
> > pseudo-keyword in context) --look strained to me.
>
> Yeah. I'd actually be in favour of a non-backward-compatible
> change so that "for x,y in L1,L2" actually does what you
> expect. How much code do you think there is out there that
> relies on the current interpretation of that?
I think
for i in 1,2:
#xxx
is not uncommon. It certainly is used in some of my code :-)
BTW: What's so bad about constructing a new crossproduct list first ?
def parallel(*args):
return apply(map,(None,)+args)
for x,y in parallel (l1,l2):
#xxx
Tom Culliton
M.-A. Lemburg <m...@lemburg.com> wrote:
>Greg Ewing wrote:
>>
>> > all syntax suggestions so far-- short of a new keyword (or
>> > pseudo-keyword in context) --look strained to me.
>>
>> Yeah. I'd actually be in favour of a non-backward-compatible
>> change so that "for x,y in L1,L2" actually does what you
>> expect. How much code do you think there is out there that
>> relies on the current interpretation of that?
TONS. Think about natural ways to use getopts:
import getopt
opts, files = getopt.getopt(args, "s:r:S:R:")
for opt, arg in opts:
if opt == "-s":
# do stuff...
This IS what I expect. I'd like to see a simple way to do parallel
iteration but not at the expense of the existing capabilities!
>I think
>
>for i in 1,2:
> #xxx
>
>is not uncommon. It certainly is used in some of my code :-)
>
>BTW: What's so bad about constructing a new crossproduct list first ?
>
>def parallel(*args):
> return apply(map,(None,)+args)
>
>for x,y in parallel (l1,l2):
> #xxx
Cute...
--
Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my permission.
Violation of my privacy with advertising or SPAM will result in a suit for a
MINIMUM of $500 damages/incident, $1500 for repeats.
Ivan Van Laningham
The Tim Peters wrote:
>
[snippety snip snip snip]
> suspect-that-with-ivan-around-we're-gonna-get-mayan-formats-before-
> roman-numerals-ly y'rs - tim
Ooooh, I made the signature!!! ;-) ;-)
My first compiler project was (drum roll) a Roman Numeral compiler. It
wasn't until a year or so later that I woke up one morning wondering
what day it was in the Mayan calendar (it took 5 years for me to find
out). I still have all that code, too. And I even incorporated the
Roman Numeral stuff into the shell I wrote about the other day--just
like ksh has a math mode (let and (( ), lsh would recognize numbers
beginning with 'Rx' as Roman. I never did incorporate Mayan Numbers
into the shell, though--would've led to some interesting stuff:
Q: What is Rx(((I))) + .|| @ @?
A: <g>
Hmmm. Maybe I should resurrect that shell;-)
<they're-already-there-tim-you-just-haven't-looked-in-the-right-place>-ly
y'rs
Ivan
----------------------------------------------
Ivan Van Laningham
Callware Technologies, Inc.
iva...@callware.com
http://www.pauahtun.org
----------------------------------------------
Graham Clemo
On Fri, 21 Aug 1998, Rainer Joswig wrote:
> In article <6retop$mtl$1...@towncrier.cc.monash.edu.au>, "Chris Wright"
> <c...@belloc.med.monash.edu.au> wrote:
>
> > It seems as if some of these examples have been getting very close to the
> > loop macro of Common Lisp.
>
[...]
> Or define QSORT:
>
> (defun qsort (ax)
> (and ax
> (let ((a (car ax))
> (x (cdr ax)))
> (append (qsort [y (y <- x) (< y a)])
> (list a)
> (qsort [y (y <- x) (>= y a)])))))
>
> ? (qsort '(6 2 7 3 5 1 98 44 22 31 555))
> -> (1 2 3 5 6 7 22 31 44 98 555)
In Python with the list comprehensions, this would be something like
def qsort(l): return list(l) and qsort([x for x in l[1:] if x<=l[0]]) + \
[l[0]] + qsort([x for x in l[1:] if x>l[0]])
But without list comprehensions,
def qsort(l):
return list(l) and qsort(filter(lambda x,z=l[0]:x<=z, l[1:])) \
+ [l[0]] + qsort(filter(lambda x,z=l[0]:x>z, l[1:]))
So I think list comprehensions in this case helps a bit, especially
eliminating those default arguments.
I think the Python list-comprehension version is easier to read than that
Lisp version, assuming you understand the 'list(l) and' bit at the start,
despite the "non-Pythonic" functional nature of the algorithm.
Of course experienced Python programmers would just use list.sort(),
but that's too easy...
I'm not sure that use of [] to mean three different things is a good idea
- it already means list-creation and sequence indexing. Although
admittedly list-comprehension is a form of list creation.
I once wrote this horrible piece of Python code to filter duplicates
from a list (OK I was being deliberately obfuscative at the time)
def unique(z):
z=list(z); z.sort(); b=[]
for c in z: [b,[]][b[-1:]==[c]][-1:]=b[-1:]+[c]
return b
So there is a possible moral there against overuse of [] ... :-)
-- Graham
Nick Leaton
http://emr.cs.uiuc.edu/home/reingold/calendar-book/index.shtml
--
Nick
Doug Landauer
> Check out this for Mayan and others.
>
> http://emr.cs.uiuc.edu/home/reingold/calendar-book/index.shtml
You might also want to seek out Claus Tondering's excellent
"frequently asked questions about calendars". A dejanews
search for that phrase should turn up a copy for you.
He posts it occasionally to sci.astro and soc.history, it appears.
--
Doug Landauer land...@apple.com (work)
land...@scruznet.com (not-work)
Andrew Kuchling
>I once wrote this horrible piece of Python code to filter duplicates
>from a list (OK I was being deliberately obfuscative at the time)
>
>def unique(z):
> z=list(z); z.sort(); b=[]
> for c in z: [b,[]][b[-1:]==[c]][-1:]=b[-1:]+[c]
> return b
Very impressive... <applause>
--
A.M. Kuchling http://starship.skyport.net/crew/amk/
A world without corruption would be a strange world indeed -- and a damned bad
world for lawyers, let me say.
-- Robertson Davies, _The Rebel Angels_
Tim Peters
> Yeah. I'd actually be in favour of a non-backward-compatible
> change so that "for x,y in L1,L2" actually does what you
> expect. How much code do you think there is out there that
> relies on the current interpretation of that?
I count more than 200 uses of a compound "for" target in the std
distribution alone. Only about 5% do
for (x, y) in ...
instead of
for x, y in ...
But only one used an expression list as opposed to an expression, i.e.
for x, y in thing1, thing2, ...:
as opposed to e.g.
for x, y in some_func(some_args):
or
for x, y in (thing1, thing2, ...)
or
for x, y in [thing1, thing2, ...]
So despite the formal syntax, people seem to feel that "x, y" is OK in the
target position but shy away from it in the expression position. Oh ya: I
noted with perverse glee that the sole instance of
for x, y in thing1, thing2, ...:
was in code I originally wrote <wink>.
> Suspecting-non-parenthesised-tuple-constructor-expressions-were-a-
> big-mistake,
It certainly hinges on fine points of the grammar, and confused me often
when I started with Python. E.g., that
for x in 1, 2, 3:
was OK but
if x in 1, 2, 3:
was a syntax error; or that
x = 1, 2
was OK but
if x == 1, 2:
was a syntax error. The rules for expression lists were probably introduced
to make parallel assignment pleasant:
if x < y:
x, y = y, x
Writing
(x, y) = (y, x)
instead would have been irksome.
Since a for loop's target list follows exactly the same rules as parallel
assignment stmts,
for x, y in whatever:
followed naturally. Allowing "whatever" to be an expression list seems much
more open to question. OTOH, while
for x, y in m, n:
is very rare, an expression list is *not* rare when the target is
non-compound:
for x in m, n:
So all the variations are in use today.
My favorite was
for type in (['b', 'h', 'i', 'l', 'f', 'd']):
where the author was so afraid of what might happen they wrapped the
expression list in useless square brackets *and* a useless set of parens
<wink>. For that matter,
for type in "bhilfd":
would have worked too ...
We haven't tried
for x,,y in seq1,,seq2:
yet, right?
parallel-commas-are-a-sure-sign-of-desperation-ly y'rs - tim
Tim Peters
> [...]
> Or define QSORT:
>
> (defun qsort (ax)
> (and ax
> (let ((a (car ax))
> (x (cdr ax)))
> (append (qsort [y (y <- x) (< y a)])
> (list a)
> (qsort [y (y <- x) (>= y a)])))))
>
> ? (qsort '(6 2 7 3 5 1 98 44 22 31 555))
> -> (1 2 3 5 6 7 22 31 44 98 555)
[Graham Clemo]
> In Python with the list comprehensions, this would be something like
>
> def qsort(l): return list(l) and qsort([x for x in l[1:] if x<=l[0]]) + \
> [l[0]] + qsort([x for x in l[1:] if x>l[0]])
I don't think the point of listcomps is to enable even more strained
imitations of Lisp <0.5 wink>. Leaving aside that this is a ridiculously
inefficient implementation of quicksort (in CL or Python), the Python
version could at least be made a little Pythonic:
def qsort(l):
if not l:
return l
# if we're going to pay this much for a quicksort, might as
# well make it stable, & avoid quadratic time on presorted or
# mostly-equal or reverse-sorted input
pivot = random.choice(l)
lt = [x for x in l if x < pivot]
eq = [x for x in l if x == pivot]
gt = [x for x in l if x > pivot]
return qsort(lt) + eq + qsort(gt)
> But without list comprehensions,
> [even uglier]
> ...
> So I think list comprehensions in this case helps a bit, especially
> eliminating those default arguments.
Given that both are a godawful mess <wink>, constrasting the above with
lt = filter(lambda x, pivot=pivot: x < pivot, l)
eq = filter(lambda x, pivot=pivot: x == pivot, l)
gt = filter(lambda x, pivot=pivot: x > pivot, l)
may make that point more clearly.
> ...
> I'm not sure that use of [] to mean three different things is a good
> idea - it already means list-creation and sequence indexing. Although
> admittedly list-comprehension is a form of list creation.
[] also means mapping (dict) lookup, sequence slicing, and via __getitem__
anything whatsoever <wink> -- but an expression that *begins* with "[" means
"build a list out of the guts" with or without listcomps. Worry about "["
having other meanings in other contexts is like worry that ":" is used both
to introduce a block and to separate keys from values in dict literals.
> I once wrote this horrible piece of Python code to filter duplicates
> from a list (OK I was being deliberately obfuscative at the time)
>
> def unique(z):
> z=list(z); z.sort(); b=[]
> for c in z: [b,[]][b[-1:]==[c]][-1:]=b[-1:]+[c]
> return b
Wicked nasty, Graham! I like it.
> So there is a possible moral there against overuse of [] ... :-)
Indeed, get rid of some of the []s and it runs a lot faster <wink>:
def unique2(z):
z=list(z); z.sort(); b=[]; k=len(z)
for c in z: b[k:]=[c][:b[-1:]!=[c]]
return b
transparently y'rs - tim
Rainer Joswig
<tim...@email.msn.com> wrote:
> I don't think the point of listcomps is to enable even more strained
> imitations of Lisp <0.5 wink>. Leaving aside that this is a ridiculously
> inefficient implementation of quicksort (in CL or Python)
Shouldn't one use mergesort to sort lists?
How about a Python version of this (destructive) CL example:
(defun merge-1 (a b)
(let* ((c (list nil))
(z c))
(cond ((null a) b)
((null b) a)
(t (loop when (<= (first a) (first b))
do (setf (rest c) a
c a
a (rest a))
else
do (setf (rest c) b
c b
b (rest b))
until (null (rest c)))
(setf (rest c) (or a b))
(rest z)))))
(defun mergesort (list)
(if (or (null list)
(null (rest list)))
list
(let ((a list)
(b (rest (rest list))))
(loop while b
do (setf list (rest list)
b (rest (rest list))))
(setf b (rest list)
(rest list) nil)
(merge-1 (mergesort a)
(mergesort b)))))
Tim Peters
> Shouldn't one use mergesort to sort lists?
>
> How about a Python version of this (destructive) CL example:
I would *certainly* use a mergesort in Lisp, but Python's lists aren't cons
cells, they're contiguous vectors. So stuff like finding the middle of a
list by sending one pointer down it at twice the speed of another is, while
clever and elegant in Lisp, plain goofy in Python -- that kind of stuff is
much easier as
halflen = len(a) / 2 # constant time
half1, half2 = a[:halflen], a[halflen:] # easy but slow
Since "a" isn't a linked list to begin with in Python, the slicing
operations in the second line build physical copies of the list slices.
For contiguous lists, it's actually hard to write a mergesort that runs as
fast as a good quicksort on modern architectures, due to the greater data
copying required by the former (a good median-of-3 quicksort does about C/5
swaps, where C is the number of comparisons done; a copying mergesort does a
copy for *each* comparison it does, and the # of compares in mergesort is
only about 20% smaller so the increase in data movement swamps that
advantage without great care).
a-fan-of-writing-lisp-in-lisp-but-python-in-python-ly y'rs - tim
Greg Ewing
>
> for (x, y) in ...
>
> instead of
>
> for x, y in ...
>
> But only one used an expression list as opposed to an expression, i.e.
>
> for x, y in thing1, thing2, ...:
Hmmm... so the interpretation of the target might have to
be dependent on whether there is a single expression or
an expression list.
> We haven't tried
>
> for x,,y in seq1,,seq2:
>
> yet, right?
And I hope we never do...
How about:
for {x, y} in L1, L2:
Greg
Tim Peters
> ...
> Hmmm... so the interpretation of the target might have to
> be dependent on whether there is a single expression or
> an expression list.
> How about:
>
> for {x, y} in L1, L2:
I strongly doubt Guido would change the meaning of *any* syntax accepted
today. You don't like this either, you know <0.9 wink>.
>> We haven't tried
>>
>> for x,,y in seq1,,seq2:
>>
>> yet, right?
> And I hope we never do...
'Twas but a joke. I simply don't have a good idea short of introducing a
new keyword-in-context. Burned out on it too <0.3 wink> -- if Guido leaps
up and says *he'd* like to address this, I'll buy some more gas, though.
Perhaps
pfor x, y in L1, L2:
banking on "pfor" being darned unlikely to conflict with anyone's existing
vrbl name.
the-p-is-silent<sssh'ed-wink>-ly y'rs - tim
Nick Leaton
>
> In article <35DD9B53...@calfp.co.uk>, nickle@pauillac wrote:
>
> > Check out this for Mayan and others.
> >
> > http://emr.cs.uiuc.edu/home/reingold/calendar-book/index.shtml
>
> You might also want to seek out Claus Tondering's excellent
> "frequently asked questions about calendars". A dejanews
> search for that phrase should turn up a copy for you.
>
> He posts it occasionally to sci.astro and soc.history, it appears.
There is a url for this
http://www.pip.dknet.dk/~c-t/calendar.html
--
Nick
Ken Manheimer
> [Greg Ewing]
> > ...
> > Hmmm... so the interpretation of the target might have to
> > be dependent on whether there is a single expression or
> > an expression list.
> > ...
> > How about:
> >
> > for {x, y} in L1, L2:
>
> I strongly doubt Guido would change the meaning of *any* syntax accepted
> today. You don't like this either, you know <0.9 wink>.
There is currently no defintion for sequence ** sequence, so how about:
>>> (1, 2, 3) ** (4, 5, 6)
((1, 4), (2, 5), (3, 6))
Is this too arbitrary? I must say, i think it would be useful. It
would follow the precedent that '+' on sequences sets, denoting
structural combination of the sequences, rather than element-wise math.
(Hmm. What would 'seq * seq' and 'seq / seq' be? Maybe the above
should use '*' rather than '**'?)
Anyway, then the 'for' clause doesn't need to be changed:
for x, y in L1 ** L2:
would do what you want - except the parallelising of the lists would
happen at once, before the iteration over the loop...
Ken Manheimer k...@python.org 703 620-8990 x268
(orporation for National Research |nitiatives
# If you appreciate Python, consider joining the PSA! #
# <http://www.python.org/psa/>. #
Guido van Rossum
>
> >>> (1, 2, 3) ** (4, 5, 6)
> ((1, 4), (2, 5), (3, 6))
This is bad because ** elsewhere doesn't mean anything remotely like
that. The elegance of overloading + and * for sequence concatenation
and repetition is that these are "close enough" to the normal
(arithmetic) meaning for + and *, which helps remembering their
meaning for sequences. I believe some branches of math use the same
conventions. I must admit that string % variable doesn't have the
same closeness, but it's easy to remember for another reason.
I think that redefining
for x1, x2, ... in seq1, seq2, ...:
is impossible at this point because
for x in seq1, seq2, ...:
is fairly common; looking at the form of the target is unprecedented
in Python (and in fact would be useful at times).
My personal favorite (and easy to parse) would be
for x1 in seq1; x2 in seq2; ...:
This is fairly obvious and easy to add to the current parser.
I don't recall how this was received though.
--Guido van Rossum (home page: http://www.python.org/~guido/)
Tim Peters
> There is currently no defintion for sequence ** sequence, so how
> about:
>
> >>> (1, 2, 3) ** (4, 5, 6)
> ((1, 4), (2, 5), (3, 6))
>
tuple ** tuple -> tuple of 2-tuples, from the example.
string ** user-defined-__getitem__-sequence -> ???
*Something's* gonna be arbitrary besides the "**" <wink>. Not to mention
tuple ** tuple ** tuple
Creates a tuple of two-tuples where the *second* element of each is also a
two-tuple (don't forget that ** is right-associative, and this can't be
hidden if a user-defined sequence is in the mix hence triggers __pow__
and/or __rpow__)?
> I must say, i think it would be useful.
Parallel iteration in general, some form of list comprehension, or an infix
operator to compute sequence zips?
> ...
> (Hmm. What would 'seq * seq' and 'seq / seq' be? Maybe the above
> should use '*' rather than '**'?)
For all its suggestive power, it may as well use 'divmod' <0.9 wink>.
> Anyway, then the 'for' clause doesn't need to be changed:
>
> for x, y in L1 ** L2:
>
> would do what you want - except the parallelising of the lists would
> happen at once, before the iteration over the loop...
Oh, but I really, really don't want that! The thing that kills the
ever-suggested
def zip2(s1, s2):
return map(None, s1, s2)
for x, y in zip2(s1, s2):
...
in practice is the memory overhead (and sometimes the delay in getting the
first output inside the loop) if the sequences are long, or one or more are
unbounded. That plus map's infernal None-padding of short sequences.
Python is computing the loop index under the covers, the nature of the
for/__getitem__ protocol pretty much guarantees it will *always* compute a
loop index under the covers, and this is just another case of finding a way
to spell "you got that, so use that, before I complain most unpleasantly".
not-that-anyone-here-would-ever-complain-mind-you-ly y'rs - tim
Tim Peters
> ...
> I think that redefining
>
> for x1, x2, ... in seq1, seq2, ...:
>
> is impossible at this point because
>
> for x in seq1, seq2, ...:
>
> is fairly common;
Plus test_long.py contains
for base, mapper in (8, oct), (10, str), (16, hex):
and in order to rewrite it we'd have to figure out what it means <wink>.
> ...
> My personal favorite (and easy to parse) would be
>
> for x1 in seq1; x2 in seq2; ...:
>
> This is fairly obvious and easy to add to the current parser.
>
> I don't recall how this was received though.
It was received poorly, *but* in the different form "use comma to separate
parallel iterations and semicolons to separate cross-product iterations" (&
all in the context of list comprehensions).
Your suggestion clearly *works*, and I doubt I'll get much argument if I
claim nobody posted a better idea.
Think this might actually happen <wink>?
If so, at least Kevin Digweed & I convinced each other (via an exchange of
examples) that parallel iteration would be most useful if it simply quit
upon hitting the first IndexError, specifically avoiding both of (1)
map-like null-padding of short sequences, and (2) trying to verify that all
sequences end on the same loop trip.
summarizingly y'rs - tim
Guido van Rossum
> > My personal favorite (and easy to parse) would be
> >
> > for x1 in seq1; x2 in seq2; ...:
> >
> > This is fairly obvious and easy to add to the current parser.
> >
> > I don't recall how this was received though.
[Tim]
> It was received poorly, *but* in the different form "use comma to separate
> parallel iterations and semicolons to separate cross-product iterations" (&
> all in the context of list comprehensions).
>
> Your suggestion clearly *works*, and I doubt I'll get much argument if I
> claim nobody posted a better idea.
>
> Think this might actually happen <wink>?
Perhaps...
> If so, at least Kevin Digweed & I convinced each other (via an exchange of
> examples) that parallel iteration would be most useful if it simply quit
> upon hitting the first IndexError, specifically avoiding both of (1)
> map-like null-padding of short sequences, and (2) trying to verify that all
> sequences end on the same loop trip.
Yes, this is exactly what I would do (you convinced me long ago that
map() does the wrong thing...).
kevin_...@my-dejanews.com
> > My personal favorite (and easy to parse) would be
> >
> > for x1 in seq1; x2 in seq2; ...:
> >
> > This is fairly obvious and easy to add to the current parser.
> >
> > I don't recall how this was received though.
If you're referring to my suggestion posted on the 18th August, it wasn't
received at all! Nobody commented and the discussion branch died :(
[Tim]
> It was received poorly, *but* in the different form "use comma to separate
> parallel iterations and semicolons to separate cross-product iterations" (&
> all in the context of list comprehensions).
That was something else (one of Greg's early suggestions, I believe).
> Your suggestion clearly *works*, and I doubt I'll get much argument if I
> claim nobody posted a better idea.
Am I allowed to argue that I posted an idea exactly as good ?? ;)
Cheers, Kev. <-- Sulking because nobody was interested when *he* suggested
it. ;) ;)
-----== Posted via Deja News, The Leader in Internet Discussion ==-----
http://www.dejanews.com/rg_mkgrp.xp Create Your Own Free Member Forum
Guido van Rossum
> received at all! Nobody commented and the discussion branch died :(
You actually posted it on August 13. Here's part of that post:
> Subject: RE: Idea - alternative to lambda
> From: "Kevin Digweed" <Kevin....@dial.pipex.com>
> To: pytho...@cwi.nl
> Date: 13 Aug 98 09:35:45 +0000
.
.
.
> OK, probably my parting shot here is the semi-colon:
>
> [ (x, y) for x in L1; y in L2 ]
> for x in L1; y in L2: print x, y
>
> It gives an error today and implies connectivity (both in written English
> and Python syntax). I'm interested to hear the reason why that one can't be
> used, either ;)
I liked it then, but was already on my way out so I didn't comment.
I particularly like it because of the parallel with written English
that you note. The parallel with existing Python syntax is weaker
(';' is *only* used to separate whole statements).
Terry Reedy
>My personal favorite (and easy to parse) would be
> for x1 in seq1; x2 in seq2; ...:
>This is fairly obvious and easy to add to the current parser.
I like this a lot. Even a beginner should quickly overcome any
confusion with ; as statement separator. Actually, if one interprets
the above as
While there is no IndexError, start each loop iteration with ...
x1 = next(seq1); x2 = next(seq2); ...
then the paralellism with current meaning is almost exact.
One possible trap: replacing
>>> for i in len(a): a[i] = a[i]+b[i]
with
>>> for ai in a; bi in b: ai = ai+bi
would not work, but
>>> for i in len(a); ai in a; bi in b: a[i] = ai+bi
presumably would, and might run faster.
>I don't recall how this was received though.
Previous reception from me (none), and -- I suspect -- others, was
tempered by (1) the volume of proposals and variations and (2)
uncertainty as to the possibility of having any effect on inducing an
upgrade.
Terry J. Reedy
Tim Peters
> ...
> One possible trap: replacing
> >>> for i in len(a): a[i] = a[i]+b[i]
> with
> >>> for ai in a; bi in b: ai = ai+bi
> would not work, but
> >>> for i in len(a); ai in a; bi in b: a[i] = ai+bi
> presumably would, and might run faster.
You need to make that
for i in range(len(a)); ... # or xrange
instead. The damnable thing is that Python is already computing a loop
index under the covers, so that range(len(a)) wouldn't be necessary if we
could only get *at* that index. See the long & impassioned thread on the
"indexing" extension from last year for more than you can stomach on that.
While Guido certainly doesn't want to see *that* thread regurgitated
<intestinal wink -- but neither do I>, a syntax trick to enable it in the
context of parallel iterations is worth some thought. Say
for i in *; ai in a; bi in b:
a[i] = ai + bi
The rationale being that in ASCII English, "*" makes you look for a
footnote, and the footnote says "(*) i ranges over the iteration's indices
in the obvious way" <wink>.
a-sequence-without-an-index-is-like-a-camel-without-a-water-bag-ly y'rs -
tim
Tim Peters
> Think this might actually happen <wink>?
[Guido]
> Perhaps...
Ah! Newcomers should be told that this is Guido's subtle way of saying he's
in doubt about scraping together September rent money. Enough said.
emailing-cash-and-lots-of-it-ly y'rs - tim
Tim Peters
> [Guido's] suggestion clearly *works*, and I doubt I'll get much
> argument if I claim nobody posted a better idea.
[Kevin Digweed, in his own words "sulking" <wink>]
> Am I allowed to argue that I posted an idea exactly as good ?? ;)
Yes -- for all the good it will do you. The rule is that if there's a good
idea posted that Guido likes, and it turns out to work well in practice,
history records that it was Guido's idea from the start. If you protest,
history goes on to record that Guido bravely implemented it over your
foolish objections.
If it works out horribly in practice (say, topples some third-world economy,
or kills someone who wasn't a suspected Perl user), history records that it
was indeed entirely your idea, and that you snuck the patch containing it
into the distribution without Guido's knowledge, and despite his brave
objections to your foolishness.
If it turns out it's so obscure nobody ever uses it, I get the credit -- but
nobody notices.
in-"benevolent-dictator"-don't-misidentify-the-beneficiary<wink>-ly y'rs -
tim
kevin_...@my-dejanews.com
> You actually posted it on August 13.
"Dejanews lied to me"))
> > [ (x, y) for x in L1; y in L2 ]
> > for x in L1; y in L2: print x, y
[snip]
> I particularly like it because of the parallel with written English
> that you note. The parallel with existing Python syntax is weaker
> (';' is *only* used to separate whole statements).
Interesting. I guess it's the way you look at it. Given the line ...
if x: x = x / 10; y = y * x
..., as the alternative is to physically separate each statement and put it
on it's own line, I tend to think of it as "enabling me to join several
statements onto a single line" rather than "separating statements on a
multi-statement line" (which is, of course, how the parser understands it).
The "connectivity" implied is that the statements as a whole form a suite and
are therefore related. I agree that the relationship is weaker than the "for"
example though, as a suite is at a higher level than a statement.
Having said that, I'll use separate lines 99.9% of the time anyway, for
clarity.
Cheers, Kev.
kevin_...@my-dejanews.com
Sounds fair ;)
> in-"benevolent-dictator"-don't-misidentify-the-beneficiary<wink>-ly y'rs -
True - it doesn't matter how few people saw it, as long as the one with all
the votes did ;)
Cheers,
Kev. <-- Hardly even a pout anymore.
Marc Wachowitz
> The damnable thing is that Python is already computing a loop
> index under the covers, so that range(len(a)) wouldn't be necessary if we
> could only get *at* that index.
For that purpose, I'd suggest a new built-in - call it integers() for the
example - which produces lots of integers increasing from zero; i.e. something
which works just like this (which can be used until the former is created):
class integers:
def __getitem__(self, index): return index
# end class integers
except that it should be faster than interpreted Python. Once the various
ideas about known-or-promised-not-to-be-redefined-built-in-bindings come
to fruition, the next logical step would be to have the compiler recognize
this specially, and generate an appropriate opcode which delivers the internal
loop counter. However, I'd expect such a simple built-in to be already faster
than range(len(a)) even without this optimization, and it might also be useful
in other contexts (e.g. if you need a counting loop with an internal break).
for i in integers(); ai in a; bi in b:
a[i] = ai + bi
Doesn't look too bad, or does it?
--
GiS - Gesellschaft fuer integrierte Systemplanung mbH
Marc Wachowitz Tel. +49-6201-503-38
Junkersstr. 2 Fax +49-6201-503-66
D-69469 Weinheim m.wac...@gis.ibfs.de
Greg Ewing
>
> >>> for i in len(a); ai in a; bi in b: a[i] = ai+bi
> presumably would, and might run faster.
Here's a *really* wild idea:
for i:
a[i] = a[i] + b[i]
which translates to:
i = 0
try:
while 1:
a[i] = a[i] + b[i]
i = i + 1
except IndexError:
pass
The motivations for this are:
1) It looks really cool
2) Symmetry: It frees you from having to make an arbitrary
choice of which sequence to base the iteration on
(do you write "for i in xrange(0, len(a)):" or
"for i in xrange(0, len(b)):"? Decisions, decisions...)
3) It does the right thing if a and b are not the same length
(according to Guido's latest pronouncement on what he
thinks the right thing is).
4) It looks really cool (did I mention that?)
[Note: In the above translation, "IndexError" should
really be "IndexError_caused_by_the_use_of_i_as_an_index",
but I don't know how to spell that in Python...]
--
Greg Ewing, Computer Science Dept, | The address below is not spam-
University of Canterbury, | protected, so as not to waste
Christchurch, New Zealand | the time of Guido van Rossum.
gr...@cosc.canterbury.ac.nz
Guido van Rossum
>
> for i:
> a[i] = a[i] + b[i]
>
> which translates to:
>
> i = 0
> try:
> while 1:
> a[i] = a[i] + b[i]
> i = i + 1
> except IndexError:
> pass
Cute, but it loses because it would mask errors: suppose I write
for i: foo(a[i])
where a bug inside foo() raises an IndexError. This would be masked,
and the programmer would be scratching her head about why the loop
terminated early.
Andrew Robinson
>While Guido certainly doesn't want to see *that* thread regurgitated
><intestinal wink -- but neither do I>, a syntax trick to enable it in the
>context of parallel iterations is worth some thought. Say
>
> for i in *; ai in a; bi in b:
> a[i] = ai + bi
>
YES PLEASE (and I don't care about the syntax). There are loads of
times when I'm not sure whether to use an index loop or a simple one
over a sequence.
(Parallel: Oracle introduced a magic 'row ID' variable some time ago,
which some say violates the set-theoretic purity of SQL, but loads of
people find useful)
Andy Robinson
Robinson Analytics Ltd.
Greg Ewing
That's why I added the note at the bottom. The implementation
would have to make sure that it only caught exceptions raised
by the indexing operations.
Greg
Guido van Rossum
Not knowing how to spell something has stopped me from considering
lesser suggestions.
However, there's certainly some charm to Tim's proposal
for i in *; ai in a; bi in b:
a[i] = ai + bi
Perhaps * could be spelled 'indexing'? :-)
Tim Peters
> ...
> True - it doesn't matter how few people saw it, as long as the
> one with all the votes did ;)
>
> Cheers,
> Kev. <-- Hardly even a pout anymore.
An exhaustive review of the posts on this topic reminded me why I didn't
think of your ";" suggestion when I should have: someone else suggested it
in pvt email before your post:
13 Aug 98 10:49:51 +0200 origination time of their post
13 Aug 98 09:35:45 +0000 origination time of your post
So, by an almost an hour, and until Guido rewrites history to make it his
own, the idea really belongs to ... (drum roll, please) Guido's brother, Mr.
Just! Sometimes it *does* feel like a rigged game <wink>.
then-again-if-history-were-important-god-wouldn't-have-hid-
it-in-the-past-ly y'rs - tim
Tim Peters
> ...
> However, there's certainly some charm to Tim's proposal
>
> for i in *; ai in a; bi in b:
> a[i] = ai + bi
>
> Perhaps * could be spelled 'indexing'? :-)
You just won't let that idea die in peace, will you <snort>?
As long as we're pursuing charm, how about:
for i in /; ai in a; ...: # index as float
for i in //; ai in a; ...: # index as int
for i in //L; ai in a; ...: # index as long int
for i in /j; ai in a; ...: # index as complex
for i in /""; ai in a; ...: # index as string
for i in //+=; ai in a; ...: # index as triangular int
for i in //////: ai in a; ...:# index as int ratio
for i in ///; ai in a; ...: # syntax error
Frankly, though, I'd trade away the whole lot 'of em for "i in *" <wink>.
BTW, the thing I like least about this is enabling crypticisms like
for i in *; a[i] in b: pass # a[:len(b)] = b
OTOH, it's really the pseudo-parallelism that's the enabler, not the "i in
*" gimmick:
for j in ii; x[j] in y: pass
is probably the same as
for i in range(len(y)):
x[ii[i]] = y[i]
Both examples point out a mildly ugly surprise (to me), too: an IndexError
can be generated by a target as well as by an iteratee. However, if neither
of us points this out, nobody will stumble into it <0.9 wink>.
abusively y'rs - tim
kevin_...@my-dejanews.com
> [Guido]
> > ...
> > However, there's certainly some charm to Tim's proposal
> >
> > for i in *; ai in a; bi in b:
> > a[i] = ai + bi
> >
> > Perhaps * could be spelled 'indexing'? :-)
[Tim]
> You just won't let that idea die in peace, will you <snort>?
[snip Tim's alternatives to "for i in *", impressive as they are ;)]
Well, if we're looking for alternatives, how about these ...
Stealing from Greg's earlier post:
for i; element in sourcelist: print "Element %d is" % i, element
Or (and I prefer this one, FWIW) :
for i in pass; element in sourcelist: print "Element %d is" % i, element
The rationale being that when presented with a for loop, the interpreter makes
zero or more passes over the body of the loop.
If a trick like this is used to get at the for loop index (whatever the
syntax) there needs to be a definition as to what happens when it's the
*only* iterator, assuming that is allowed at all.
It makes sense to me that:
for i in pass:
<stuff>
is shorthand for (and would be faster than):
i = -1
while 1:
i = i + 1
<stuff>
I`m not sure how the Python internal loop index wraps, but assume that there
is an appropriate try/except clause in the while loop addition if neccessary.
Cheers, Kev.
Guido van Rossum
>
I thought you'd appreciate the reference :-)
> BTW, the thing I like least about this is enabling crypticisms like
>
> for i in *; a[i] in b: pass # a[:len(b)] = b
>
> OTOH, it's really the pseudo-parallelism that's the enabler, not the "i in
> *" gimmick:
>
> for j in ii; x[j] in y: pass
>
> is probably the same as
>
> for i in range(len(y)):
> x[ii[i]] = y[i]
>
> Both examples point out a mildly ugly surprise (to me), too: an IndexError
> can be generated by a target as well as by an iteratee. However, if neither
> of us points this out, nobody will stumble into it <0.9 wink>.
But an IndexError in the target assignment would not be interpreted as
"end of loop"; it would be propagated as a real IndexError, just like
today. Or are you after something else? (Like Greg Ewing's proposal?
I don't even want to think about how to describe that exactly -- and
neither does Greg :-)
Ken Manheimer
> Well, if we're looking for alternatives, how about these ...
>
> Stealing from Greg's earlier post:
> for i; element in sourcelist: print "Element %d is" % i, element
I go "Bingo!" for this one. I see the "for i;" as the inductive "for
all i such that". Eg, for something like:
for i; a in x; b in y:
i can read:
for all i such that
i is an element of range(len(x))
and i is an element of range(len(y)) # ie, intersection
then a gets x[i] and b gets y[i] over the course of the loop.
Of course, the lengths are actually gotten by virtue of hitting
IndexError when indexing the iteration subjects, x and y, within the
loop. I *think* i understand about the problem tim and guido aren't
supposed to mention <wink>, where an index target var (a and b) might
itself be a sequence that can be exhausted - and raise the terminating
IndexError - before the indexing subjects (x and y). Eg, for:
for i; a[i] in x; b in y:
What should happen when a is shorter than x and y? It seems clear to me
that the "in" clauses explicitly are being applied to the index
subjects, the sequences x and y, so use of a too short index target (a)
would just be an error.