Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

List Comprehension Syntax

2 views
Skip to first unread message

Moosebumps

unread,
Jul 10, 2004, 3:46:23 AM7/10/04
to
Does anyone here find the list comprehension syntax awkward?

I like it because it is an expression rather than a series of statements,
but it is a little harder to maintain it seems.

e.g. you could do:

result = []
for element in list:
if element[0:4] == 'blah':
result.append( element.replace( 'blah', 'moof' ) )

or just:

result = [ element.replace( 'blah', 'moof' ) for element in list if
element[0:4] == 'blah' ]

The second looks cleaner in some cases, but it is less maintainable. It
tends to promote long lines. It all seems to run together and such. And
often you would need to add another condition or modify it, it seems better
to use the first way even though it has the ugly extra init (and which would
cause more allocs than the list comprehension? because it wouldn't know the
size off the bat?)

I am in favor of short lines like I think Guido said in the style guide. I
like each line to be so simple as to not require any thinking reading it,
e.g.

I prefer:

x = c( d, e )
y = f( g, h )
z = b( x, y )
w = a( z )

to stuff like this:

w = a( b( c( d, e ), f( g, h ) ) )

It is more maintainable, when you need to make a change, just insert a line,
rather than having to decode an expression.

Along the same lines, it seems more maintainable to split things up.

You could do:

result = [
element.replace( 'blah', 'moof' )
for element in list
if element[0:4] == 'blah' ]

I guess, but that seems awkward to me. Looks too much like a for loop and
an if, and then the value is at the top, which reads funny to me.
(Strangely putting it on one line doesn't read as funny, but it is less
readable.) Maybe I just have to get used to it. Which do you prefer?
Comments?

MB


Ville Vainio

unread,
Jul 10, 2004, 6:39:57 AM7/10/04
to
>>>>> "Moosebumps" == Moosebumps <cr...@crap.crap> writes:

Moosebumps> Does anyone here find the list comprehension syntax awkward?

...

Moosebumps> I am in favor of short lines like I think Guido said
Moosebumps> in the style guide. I like each line to be so simple
Moosebumps> as to not require any thinking reading it,

You need to think of the total complexity involved with having several
lines. When you see a list comprehension, you know what to expect -
transformation and/or filtering applied to a list. Therefore, you can
easily read and write out the beast.

LC's also encourage the list transformation/filtering approach to
problems, which I find absolutely splendid. Appending elements to a
list manually is tedious, especially if the problem really is a
stereotypical problem solved by a LC (and interestingly, most problems
are ;-).

Moosebumps> You could do:

Moosebumps> result = [
Moosebumps> element.replace( 'blah', 'moof' )
Moosebumps> for element in list
Moosebumps> if element[0:4] == 'blah' ]

Moosebumps> I guess, but that seems awkward to me. Looks too much
Moosebumps> like a for loop and an if, and then the value is at
Moosebumps> the top, which reads funny to me. (Strangely putting
Moosebumps> it on one line doesn't read as funny, but it is less
Moosebumps> readable.) Maybe I just have to get used to it.
Moosebumps> Which do you prefer?

It's just a matter of getting used to it. Admittedly LC's are
sometimes confusing for newbies, but they are an example of such a
feature where the tradeoff between newbie and non-newbie friendliness
has really paid off.

Now that genexps are coming around, you'll be facing even bigger
payoffs. So just keep using them, even if they might not feel as
maintanable at the moment. LC's (and genexps even to a bigger extent)
are pretty much what defines the "pythonic" way of doing things for me
these days.

--
Ville Vainio http://tinyurl.com/2prnb

Eric S. Johansson

unread,
Jul 10, 2004, 8:53:29 AM7/10/04
to pytho...@python.org
Ville Vainio wrote:

as someone approaching 25 years experience with programming languages
and their implications, I find list comprehension's incomprehensible.
Personally I think my learning is inhibited by the documentation which
seems driven by the syntax of list comprehensions rather than the
execution model and how to recognize when it's appropriate to apply that
model.

I find multiple short lines actually easier to comprehend because it
translates into a common mental model of mine. The current syntax for
those comprehension's reminds me of APL which was a wonderful language
if you thought in nothing but vectors.

I will admit this discussion has goaded me into trying to learn list
comprehensions again as I am trying to solve a problem which is
filtering a dictionary of data elements based on predicates selecting
individual elements and/or predicates on the offset from the start of
the previous predicate list.

By the way, similarly hampered-by-the-documentation are generators,
iterators, and profiling time bases. I'm having a lot of difficulty
pulling out model abstractions, and as I said above, understanding where
to apply them in problem spaces.

>
> Now that genexps are coming around, you'll be facing even bigger
> payoffs. So just keep using them, even if they might not feel as
> maintanable at the moment. LC's (and genexps even to a bigger extent)
> are pretty much what defines the "pythonic" way of doing things for me
> these days.

this is not meant to be picking on you in any way shape or form but my
experience has been that any time you find yourself having to "thinking
in the language", you are not really solving the right problem and are
more likely using a collection of magic tricks to confound and amaze
others and possibly insure job security.

if you have models that can be implemented independent of the language
and you can express a problem in terms that are natural to the problem,
you invariably have a better solution for the people following you as
well as the machine.

generalize, don't pythonize.

---eric

Ville Vainio

unread,
Jul 10, 2004, 9:41:16 AM7/10/04
to
>>>>> "Eric" == Eric S Johansson <e...@harvee.org> writes:

Eric> as someone approaching 25 years experience with programming
Eric> languages and their implications, I find list
Eric> comprehension's incomprehensible. Personally I think my
Eric> learning is inhibited by the

I found LC's a bit odd too at first. Previous programming experience
probably doesn't matter too much with them, because they are quite
different.

Eric> documentation which seems driven by the syntax of list
Eric> comprehensions rather than the execution model and how to
Eric> recognize when it's appropriate to apply that model.

Probably. It's all too easy to dismiss if the documentation doesn't
sell it well (this is fixable, luckily). I kinda ignored LC's too, but
persistent ramblings on c.l.py (by Alex Martelli and others) changed
that, for which I'm grateful and feel honor-bound to continue the
pseudo-oral tradition :-).

The main thing to realize about list comprehensions is that
they simply provide a more elegant way to do 'map' and 'filter' when a
function to be applied is not something trivial like str or
int. Having LC's handy urges one to go ahead with map/filter like
approaches to problems where implementing new functions (or calling
old ones via lambda) seems like an unnecessary hassle.

Eric> I find multiple short lines actually easier to comprehend
Eric> because it translates into a common mental model of mine.

It helps if your mental model involves manipulating lots of
lists. I've found that the list manipulation model works great for me,
allowing me to solve most problems quickly and (I think) elegantly. I
guess it depends a lot on what you are doing - my python use is mostly
just scripting these days (for reasons not in my control, of course
;-).

Eric> I will admit this discussion has goaded me into trying to
Eric> learn list comprehensions again as I am trying to solve a
Eric> problem which is filtering a dictionary of data elements
Eric> based on predicates selecting individual elements and/or
Eric> predicates on the offset from the start of the previous
Eric> predicate list.

Good for you. Do it with map and filter (and in-scope funcs using
closures) and go LC only afterwards if that feels easier. I believe
people still feel more comfortable with LCs than nested scopes,
because the LC is "visually" more in the same scope. The part after
"and/or" seemed too mysterious to give the solution now, but it seems
you'll need to implement a function in addition to the LC to keep the
solution clean.

Eric> By the way, similarly hampered-by-the-documentation are
Eric> generators, iterators, and profiling time bases. I'm having
Eric> a lot of difficulty pulling out model abstractions, and as I
Eric> said above, understanding where to apply them in problem
Eric> spaces.

Do you feel it's the offical docs that are lacking, or have you tried
reading some Python books? I've probably been in the "enthusiast"
crowd that gets a kick from reading those "what's new" docs, PEPs and
such?

Eric> experience has been that any time you find yourself having
Eric> to "thinking in the language", you are not really solving
Eric> the right problem and are more likely using a collection of
Eric> magic tricks to confound and amaze others and possibly
Eric> insure job security.

Eric> if you have models that can be implemented independent of
Eric> the language and you can express a problem in terms that are
Eric> natural to the problem, you invariably have a better
Eric> solution for the people following you as well as the
Eric> machine.

Luckily, the underlying model of LCs is very well understood (map &
filter), and solving things the list-processing way is a time-honed
practice. In the "amaze your friends" front, LCs are more in the "look
how elegant and concise this can be" genre, not in the "check this
out, all recursion and no variables, I bet you can't get it even after
staring it for 5 minutes" genre loved by some academics.

I think you'll find that you don't need to sacrifice any of your old
models or even aesthetic preferences to appreciate LCs.

Dave Brueck

unread,
Jul 10, 2004, 10:29:21 AM7/10/04
to pytho...@python.org
Eric S. Johansson wrote:

> Ville Vainio wrote:
>> It's just a matter of getting used to it. Admittedly LC's are
>> sometimes confusing for newbies, but they are an example of such a
>> feature where the tradeoff between newbie and non-newbie friendliness
>> has really paid off.
>
> as someone approaching 25 years experience with programming languages
> and their implications, I find list comprehension's incomprehensible.
> Personally I think my learning is inhibited by the documentation which
> seems driven by the syntax of list comprehensions rather than the
> execution model and how to recognize when it's appropriate to apply that
> model.

What I like about LCs is that they made sense to me before reading the
documentation - they just came across as very expressive of what was
happening. I use the simplest forms, i.e.

[op(x) for x in y]
[x for x in y if z]
(as well as some slight variations)

quite a bit, but rarely do I use the more complex forms (e.g. multiple
for's) because (1) the meaning doesn't jump right out as quickly (to me)
and (2) the more complex they are, the more they seem like a fancy trick
rather than the right thing to do.

But most any time you're using map or filter or basically doing:

L = []
for item in L2:
L.append(op(item))

the LC form is often more desirable because it screams "I'm taking a
sequence and using it to create a list" - the intent of the code is very
clear. And by extension, if you're using an LC to do something obtuse,
you deserve a slap on the wrist (e.g. you write [f() for f in x] and
throw away the resulting list).

>> Now that genexps are coming around, you'll be facing even bigger
>> payoffs. So just keep using them, even if they might not feel as
>> maintanable at the moment. LC's (and genexps even to a bigger extent)
>> are pretty much what defines the "pythonic" way of doing things for me
>> these days.
>

> this is not meant to be picking on you in any way shape or form but my

> experience has been that any time you find yourself having to "thinking
> in the language", you are not really solving the right problem and are
> more likely using a collection of magic tricks to confound and amaze
> others and possibly insure job security.

That's not what I understood by Ville's comment. I think he just meant
that LC's (and genexps) are powerful tools in the Python toolbox, and
useful enough that the OP should continue working to become familiar
with them. They're not obscure magic tricks but "first class" features
of the language.

I've seen many comments on c.l.py to the effect of "LCs seem bad because
they can abused", citing bizarre made-up examples with 4 loops and as
many if statements. Those *are* magic tricks and should be avoided, but
then again any feature can be abused.

> if you have models that can be implemented independent of the language
> and you can express a problem in terms that are natural to the problem,
> you invariably have a better solution for the people following you as
> well as the machine.
>
> generalize, don't pythonize.

I'm not so sure. Why program to the lowest common language denominator?
I don't suggest going to the extreme to use obscure language quirks just
because you can, but it doesn't make sense to avoid using a feature at
your disposal because it's unique to a language (or, in this case, a
small set of languages).

Half the reason you use one language over another is because of the
toolset it gives you. In the case of list comprehensions, they are
usually chosen for the very reason that they *do* allow you to express a
problem in natural terms.

-Dave

Eric S. Johansson

unread,
Jul 10, 2004, 10:42:04 AM7/10/04
to pytho...@python.org
Ville Vainio wrote:

> Eric> documentation which seems driven by the syntax of list
> Eric> comprehensions rather than the execution model and how to
> Eric> recognize when it's appropriate to apply that model.
>
> Probably. It's all too easy to dismiss if the documentation doesn't
> sell it well (this is fixable, luckily). I kinda ignored LC's too, but
> persistent ramblings on c.l.py (by Alex Martelli and others) changed
> that, for which I'm grateful and feel honor-bound to continue the
> pseudo-oral tradition :-).

I understand, having been the creator of more than a couple oral
traditions myself.

> The main thing to realize about list comprehensions is that
> they simply provide a more elegant way to do 'map' and 'filter' when a
> function to be applied is not something trivial like str or
> int. Having LC's handy urges one to go ahead with map/filter like
> approaches to problems where implementing new functions (or calling
> old ones via lambda) seems like an unnecessary hassle.

maybe that's the problem. I've never seen anyplace where it's easier to
use map and filter.

> It helps if your mental model involves manipulating lots of
> lists. I've found that the list manipulation model works great for me,
> allowing me to solve most problems quickly and (I think) elegantly. I
> guess it depends a lot on what you are doing - my python use is mostly
> just scripting these days (for reasons not in my control, of course
> ;-).

well, I do manipulate a few lists but more often, I manipulate
dictionaries. I tend to think more in terms of sets, bags and queues.

if you want to take on a bigger (open source) project, I have one you
can work on. ;-)

> Good for you. Do it with map and filter (and in-scope funcs using
> closures) and go LC only afterwards if that feels easier. I believe
> people still feel more comfortable with LCs than nested scopes,
> because the LC is "visually" more in the same scope. The part after
> "and/or" seemed too mysterious to give the solution now, but it seems
> you'll need to implement a function in addition to the LC to keep the
> solution clean.

I will show you what I create the easiest which I can guarantee you will
be a iterative solution. then maybe we can work through the process of
converting it to a list comprehension.


>
> Eric> By the way, similarly hampered-by-the-documentation are
> Eric> generators, iterators, and profiling time bases. I'm having
> Eric> a lot of difficulty pulling out model abstractions, and as I
> Eric> said above, understanding where to apply them in problem
> Eric> spaces.
>
> Do you feel it's the offical docs that are lacking, or have you tried
> reading some Python books? I've probably been in the "enthusiast"
> crowd that gets a kick from reading those "what's new" docs, PEPs and
> such?

I've tried reading a few sources. and I'm not trying to throw bricks
because I do know how hard it is to write user understandable
documentation especially if you have been totally immersed in a project
for a while.

> Luckily, the underlying model of LCs is very well understood (map &
> filter), and solving things the list-processing way is a time-honed
> practice. In the "amaze your friends" front, LCs are more in the "look
> how elegant and concise this can be" genre, not in the "check this
> out, all recursion and no variables, I bet you can't get it even after
> staring it for 5 minutes" genre loved by some academics.

but even good tools can be used for the purposes of evil... ;-)

---eric


Eric S. Johansson

unread,
Jul 10, 2004, 11:38:52 AM7/10/04
to pytho...@python.org
Dave Brueck wrote:

> What I like about LCs is that they made sense to me before reading the
> documentation - they just came across as very expressive of what was
> happening. I use the simplest forms, i.e.

...


> throw away the resulting list).

good advice. Thank you for making this clear.

> I'm not so sure. Why program to the lowest common language denominator?
> I don't suggest going to the extreme to use obscure language quirks just
> because you can, but it doesn't make sense to avoid using a feature at
> your disposal because it's unique to a language (or, in this case, a
> small set of languages).
>
> Half the reason you use one language over another is because of the
> toolset it gives you. In the case of list comprehensions, they are
> usually chosen for the very reason that they *do* allow you to express a
> problem in natural terms.

it all depends on your definition of natural terms. ;-)

For me, the abstractions I use tend to be higher level than what most
languages support and is always a loss of clarity in the translation to
implementation. Python minimizes the translation distance for me.

Sometimes I find myself avoiding language quirks and features because
I'm trying to get a job done and I don't want to go through the mental
puzzle of mapping the abstract form into the implementation form using a
particular feature. So for example, I use for loops in preference to
list comprehensions just because it's faster to implement and get the
job done.

Most customers don't pay you for pretty code, they pay you to accomplish
something quickly and make it understandable to the less skilled people
following you. which probably explains why so much software is crap but
that's a whole different discussion.

your observation about reasons for choosing languages are certainly
accurate for most people but for me, it's quite different. If I can't
write code using speech recognition without doing a job on my throat, I
won't use the language. List comprehension syntax is approaching dammed
ugly for speech recognition users.

but thank you again for your clear example of how you use list
comprehensions.

---eric

Ville Vainio

unread,
Jul 10, 2004, 11:50:40 AM7/10/04
to
>>>>> "Eric" == Eric S Johansson <e...@harvee.org> writes:

>> It helps if your mental model involves manipulating lots of
>> lists. I've found that the list manipulation model works great
>> for me,

Eric> well, I do manipulate a few lists but more often, I
Eric> manipulate dictionaries. I tend to think more in terms of
Eric> sets, bags and queues.

These work as sequences (by lists, I meant sequences) as
well. Especially as we get genexps, LCish operations can be performed
for free without space penalty.

Eric> if you want to take on a bigger (open source) project, I
Eric> have one you can work on. ;-)

No doubt ;-).

Eric> I will show you what I create the easiest which I can guarantee you
Eric> will be a iterative solution. then maybe we can work through the
Eric> process of converting it to a list comprehension.

Sounds like a plan.

Ville Vainio

unread,
Jul 10, 2004, 12:35:15 PM7/10/04
to
>>>>> "Eric" == Eric S Johansson <e...@harvee.org> writes:

Eric> it all depends on your definition of natural terms. ;-)

Eric> For me, the abstractions I use tend to be higher level than
Eric> what most languages support and is always a loss of clarity
Eric> in the translation to implementation. Python minimizes the
Eric> translation distance for me.

That's good news, because LCs are higher level than explicit loops.

Eric> a particular feature. So for example, I use for loops in
Eric> preference to list comprehensions just because it's faster
Eric> to implement and get the job done.

That's mostly a result of you not a comfortable grasp of LCs yet. As
with most things in Python, learning them is going to be worth the
time used for learning.


Eric> your observation about reasons for choosing languages are
Eric> certainly accurate for most people but for me, it's quite
Eric> different. If I can't write code using speech recognition
Eric> without doing a job on my throat, I won't use the language.
Eric> List comprehension syntax is approaching dammed ugly for
Eric> speech recognition users.

How? The only special thing about LCs are the surrounding ['s. It's
also a lot less speaking, which is probably a good thing.

Admittedly I have never used speech recognition software. How do you
speak out the following equivalent snippets:

----

files = [f.lower() for f in allfiles if f.endswith(".txt")]

----


files = []
for f in allfiles:
if f.endswith(".txt"):
files.append(f.lower())

----


If it has something to do with line breaking, the following is
obviously ok too (and in no way inferior to the one-line approach):

files = [f.lower()
for f in allfiles
if f.endswith(".txt")]

Eric S. Johansson

unread,
Jul 10, 2004, 12:55:24 PM7/10/04
to pytho...@python.org
Ville Vainio wrote:

>
> Admittedly I have never used speech recognition software. How do you
> speak out the following equivalent snippets:
>
> ----
>
> files = [f.lower() for f in allfiles if f.endswith(".txt")]

files equal sign between brackets foxtrot dot lower matched parens for
foxtrot in all no space files if foxtrot dot ends no space with matched
parens between quotes dot tango x-ray tango

is a reasonably close approximation not counting misrecognitions.
personally, I would never ever use single character variables unless I'm
typing them and I would never use merged words. I would always join
them with _ because it's easier and more accurate than saying no-space.
and mixed case is right out because you have to state every time you
shift case unless NaturallySpeaking just happens to know the word in the
right case.

and the matched brackets etc. macros are of my own creation

> ----
>
>
> files = []

files equal sign matched brackets

> for f in allfiles:

for foxtrot in all no space files :

> if f.endswith(".txt"):

if foxtrot dot ends no space with between parens between quotes dot
tango x-ray tango end colon

> files.append(f.lower())

files dot append between parens foxtrot dot lower matched parens


and it's not fun. take a look at camram if you want to see a fair
amount of Python written probably 80 or 90 percent by voice. Its
somewhere over 5000 lines of code if I'm not counting improperly.


> If it has something to do with line breaking, the following is
> obviously ok too (and in no way inferior to the one-line approach):

it's easier with speech recognition to say small things and correct
especially when coding. unfortunately, NaturallySpeaking does not work
extremely well with applications that do not use a very limited set of
edit controls.

---eric

Mitja

unread,
Jul 10, 2004, 2:24:34 PM7/10/04
to
Moosebumps <cr...@crap.crap>
(news:j9NHc.8596$mJ3....@newssvr25.news.prodigy.com) wrote:

> You could do:
>
> result = [
> element.replace( 'blah', 'moof' )
> for element in list
> if element[0:4] == 'blah' ]
>
> I guess, but that seems awkward to me. Looks too much
> like a for loop and an if, and then the value is at the
> top, which reads funny to me. (Strangely putting it on
> one line doesn't read as funny, but it is less readable.)
> Maybe I just have to get used to it. Which do you
> prefer? Comments?

I usually do


result = [
element.replace( 'blah', 'moof' )
for element in list
if element[0:4] == 'blah'
]

It seems clean and logical enough to me - like e.g. defining big dicts.


Andrea Griffini

unread,
Jul 10, 2004, 3:42:34 PM7/10/04
to
On 10 Jul 2004 16:41:16 +0300, Ville Vainio <vi...@spammers.com>
wrote:

>I think you'll find that you don't need to sacrifice any of your old
>models or even aesthetic preferences to appreciate LCs.

I'm quite new to python, and I found LC quite simple to understand
and powerful. The only "suprising" part for me was that the looping
variable is not local... I slipped on that a couple of times.
A list comphrension is very "local" in my mind ... I would say that
[x*x for x in xrange(10)] is just a list of perfect squares, but
instead it's not... it's that AND the assignment to x of the value 9.

Andrea

Moosebumps

unread,
Jul 10, 2004, 3:42:36 PM7/10/04
to

> What I like about LCs is that they made sense to me before reading the
> documentation - they just came across as very expressive of what was
> happening. I use the simplest forms, i.e.
>
> [op(x) for x in y]
> [x for x in y if z]
> (as well as some slight variations)

That's true, I tend to use the simplest forms too -- but then that is not as
maintainable.

We all know that things don't always stay so simple. There is always
something you need to add. IMO with that in mind, it is easier to keep
everything consistent, then to try to "sneak in" the LC for the simpler
cases. You would end up flipping back and forth too much when you need to
add a condition, or add an expression.

Moosebumps

unread,
Jul 10, 2004, 3:46:25 PM7/10/04
to

"Ville Vainio" <vi...@spammers.com> wrote in message
news:du7n028...@mozart.cc.tut.fi...

> >>>>> "Moosebumps" == Moosebumps <cr...@crap.crap> writes:
>
> Moosebumps> Does anyone here find the list comprehension syntax
awkward?
>
> ...
>
> Moosebumps> I am in favor of short lines like I think Guido said
> Moosebumps> in the style guide. I like each line to be so simple
> Moosebumps> as to not require any thinking reading it,
>
> You need to think of the total complexity involved with having several
> lines. When you see a list comprehension, you know what to expect -
> transformation and/or filtering applied to a list. Therefore, you can
> easily read and write out the beast.

Yes, I agree completely! Note that I have no problem with the idea of LC, I
love the idea. I am math person, I am very used to the set syntax:

S = { A(x) | Vx in 2^T }

etc., where V is the "for all", 2^ is powerset, whatever.

I am just wondering about the syntax, that's all. Once you try to come up
with anything a little more complicated than the basic forms, they get to be
too unreadable and unformattable.

I agree it is far superior to use an expression when that is all you need,
and to only use statements when you need them!

MB


Moosebumps

unread,
Jul 10, 2004, 3:52:36 PM7/10/04
to

> I usually do
> result = [
> element.replace( 'blah', 'moof' )
> for element in list
> if element[0:4] == 'blah'
> ]
> It seems clean and logical enough to me - like e.g. defining big dicts.
>
>

But what if you have multiple for and if's? Seems like you would want to
indent them then.

I think I might just get used to the multiline syntax since LCs are very
useful... it just seems awkward for some reason.

result = [ x for x in blah if f(x) ]

seems more elegant than:

result = [
x
for x in blah
if f(x) ]

but I like to keep things consistent and have maintainability, so then the
second one wins. i.e. the second one "scales better" to more complicated
expressions!

MB

Dave Brueck

unread,
Jul 10, 2004, 4:20:57 PM7/10/04
to pytho...@python.org
Moosebumps wrote:

>>What I like about LCs is that they made sense to me before reading the
>>documentation - they just came across as very expressive of what was
>>happening. I use the simplest forms, i.e.
>>
>>[op(x) for x in y]
>>[x for x in y if z]
>>(as well as some slight variations)
>
>
> That's true, I tend to use the simplest forms too -- but then that is not as
> maintainable.
>
> We all know that things don't always stay so simple. There is always
> something you need to add. IMO with that in mind, it is easier to keep
> everything consistent, then to try to "sneak in" the LC for the simpler
> cases. You would end up flipping back and forth too much when you need to
> add a condition, or add an expression.

I can see the potential for a maintenance problem, but I haven't seen it
actually be a problem in practice.

Perhaps it's due in part to the fact that an LC normally wouldn't "grow"
more complex in isolation - the change would likely be tied to changes
in surrounding code as well - and so maybe the entire code block would
be refactored. Dunno... but compared to the number of times LCs are used
in code, the number of times they are later "unrolled" to the for-loop
form has been too low to worry about.

-Dave

Mitja

unread,
Jul 10, 2004, 5:00:45 PM7/10/04
to
Moosebumps <cr...@crap.crap>

(news:8OXHc.13877$4y7....@newssvr27.news.prodigy.com) wrote:
>> I usually do
>> result = [
>> element.replace( 'blah', 'moof' )
>> for element in list
>> if element[0:4] == 'blah'
>> ]
>> It seems clean and logical enough to me - like e.g.
>> defining big dicts.
>>
>>
>
> But what if you have multiple for and if's? Seems like
> you would want to indent them then.

Hm... I never used a complicated enough case :)
I still think I wouldn't indent them - it is clear enough how they corelate.
If anything, I'd do
foo = [
a*b
for a in range(10)
for b in range(10)
for c in ['foo','bar']
if 42==True
]

Can you have multiple ifs at all?

Reinhold Birkenfeld

unread,
Jul 11, 2004, 2:47:55 AM7/11/04
to
Mitja wrote:
> Moosebumps <cr...@crap.crap>
> (news:8OXHc.13877$4y7....@newssvr27.news.prodigy.com) wrote:
>>> I usually do
>>> result = [
>>> element.replace( 'blah', 'moof' )
>>> for element in list
>>> if element[0:4] == 'blah'
>>> ]
>>> It seems clean and logical enough to me - like e.g.
>>> defining big dicts.
>>>
>>>
>>
>> But what if you have multiple for and if's? Seems like
>> you would want to indent them then.
>
> Hm... I never used a complicated enough case :)
> I still think I wouldn't indent them - it is clear enough how they corelate.
> If anything, I'd do
> foo = [
> a*b
> for a in range(10)
> for b in range(10)
> for c in ['foo','bar']
> if 42==True
> ]
>
> Can you have multiple ifs at all?

No, but you can have if (...) and (...) or (...)

Reinhold

--
Wenn eine Linuxdistribution so wenig brauchbare Software wie Windows
mitbrächte, wäre das bedauerlich. Was bei Windows der Umfang eines
"kompletten Betriebssystems" ist, nennt man bei Linux eine Rescuedisk.
-- David Kastrup in de.comp.os.unix.linux.misc

Moosebumps

unread,
Jul 11, 2004, 5:25:59 AM7/11/04
to
> > Can you have multiple ifs at all?
>
> No, but you can have if (...) and (...) or (...)

Sure you can:

>>> result = [x for x in range(10) if x % 2 == 0 if x % 3 == 0]
>>> result
[0, 6]
>>> result = [ x*y for x in range(10) if x%2 == 0 for y in range(10) if y %
3 == 0]
>>> result
[0, 0, 0, 0, 0, 6, 12, 18, 0, 12, 24, 36, 0, 18, 36, 54, 0, 24, 48, 72]
>>>


It appears that you can have as many if's and for's as you like.

Just curious -- anyone care to tell me how they would format the above? (or
maybe you wouldn't write it all)

MB


Reinhold Birkenfeld

unread,
Jul 11, 2004, 8:25:22 AM7/11/04
to
Moosebumps wrote:
>> > Can you have multiple ifs at all?
>>
>> No, but you can have if (...) and (...) or (...)
>
> Sure you can:

OK, thanks.

Peter Hansen

unread,
Jul 11, 2004, 8:35:35 AM7/11/04
to
Moosebumps wrote:

> Just curious -- anyone care to tell me how they would format the above? (or
> maybe you wouldn't write it all)

result = [x for x in range(10)


if x % 2 == 0
if x % 3 == 0
]

result = [x * y


for x in range(10) if x % 2 == 0

for y in range(10) if y % 3 == 0
]

Probably something like that?

-Peter

Ville Vainio

unread,
Jul 11, 2004, 10:22:44 AM7/11/04
to

Moosebumps wrote:
> Just curious -- anyone care to tell me how they would format the above? (or
> maybe you wouldn't write it all)

Peter indents:
---------------


result = [x for x in range(10)
if x % 2 == 0
if x % 3 == 0
]

result = [x * y
for x in range(10) if x % 2 == 0
for y in range(10) if y % 3 == 0
]

-----------------

Nope, the only correct way is


result = [x for x in range(10)
if x % 2 == 0
if x % 3 == 0
]

result = [x * y
for x in range(10) if x % 2 == 0
for y in range(10) if y % 3 == 0
]

Because that's the way python-mode.el does it in emacs :-).

James Moughan

unread,
Jul 11, 2004, 12:40:29 PM7/11/04
to
Andrea Griffini <agr...@tin.it> wrote in message news:<f5h0f0d4l7a9qhj0k...@4ax.com>...

Yup, scoping rules are usually the one confusing part of Python. I was
especially surprised by the result of:

l = [lambda y: x*y for x in range(10)]

which doesn't do remotely what you'd expect. Especially if you
learned about list comprehensions in Haskell, with no variable
assignments... :-)

Matthew Scott

unread,
Jul 11, 2004, 3:32:54 PM7/11/04
to
Ville Vainio wrote:
> Nope, the only correct way is
>
>
> result = [x for x in range(10)
> if x % 2 == 0
> if x % 3 == 0
> ]
>
> result = [x * y
> for x in range(10) if x % 2 == 0
> for y in range(10) if y % 3 == 0
> ]
>
> Because that's the way python-mode.el does it in emacs :-).

Unless, of course, you leave the opening bracket as the last character
on the line, in which case python-mode will prefer you to do this:

result = [
x for x in range(10)
if x % 2 == 0
if x % 3 == 0
]

result = [
x * y
for x in range(10) if x % 2 == 0
for y in range(10) if y % 3 == 0
]

:)

Peter Hansen

unread,
Jul 11, 2004, 10:01:37 PM7/11/04
to
Ville Vainio wrote:

> Nope, the only correct way is
>
> result = [x for x in range(10)
> if x % 2 == 0
> if x % 3 == 0
> ]
>
> result = [x * y
> for x in range(10) if x % 2 == 0
> for y in range(10) if y % 3 == 0
> ]
>
> Because that's the way python-mode.el does it in emacs :-).

Unfortunately, python-mode.el is wrong, since it results
in code that is harder to maintain (at least with another
editor).

Indentation that has to line up with random items on
the previous line, as opposed to the standard four spaces,
is clearly wrong.

But if you let emacs define the limits of your world, then
it's probably okay to do it that way. ;-)

François Pinard

unread,
Jul 12, 2004, 9:32:14 AM7/12/04
to Peter Hansen, pytho...@python.org
[Peter Hansen]

> Unfortunately, python-mode.el is wrong, since it results in code that
> is harder to maintain (at least with another editor).

I'm not arguing here if `python-mode.el' is right or wrong.

However, for me at least, the greatest virtue of Python is that one can
write it legibly, and maintainability mainly results from legibility.

Being easier or harder to maintain with this or that editor is rather
irrelevant, if legibility is at stake. A lot of people are using lesser
editors anyway, and do all their alignment "by hand".

> But if you let emacs define the limits of your world, then it's
> probably okay to do it that way. ;-)

I do see the smiley. Yet, if people use an editor which is more helpful
on the side of legibility, it's not limitative, but plain good for them!

--
François Pinard http://www.iro.umontreal.ca/~pinard

Peter Hansen

unread,
Jul 12, 2004, 10:19:14 AM7/12/04
to
François Pinard wrote:

> ... for me at least, the greatest virtue of Python is that one can


> write it legibly, and maintainability mainly results from legibility.

I don't find the legibility of the carefully aligned version of
that code to be higher than the simpler "indent the standard amount"
version of the code. Sometimes consistency is not foolish.

As for maintainability, the problem is not that other people
use an editor which makes it harder to produce the same level
of legibility (since I find the legibility the same or worse
with the "artistic" alignment). The problem is that other people
who use another editor (and it should be very clear that there
are such people and will always be) will find it much harder to
maintain alignment which, without editor support, requires
careful manual alignment.

> Being easier or harder to maintain with this or that editor is rather
> irrelevant, if legibility is at stake. A lot of people are using lesser
> editors anyway, and do all their alignment "by hand".

If they do the alignment that way, making things carefully line
up with the parentheses, and manually, they are definitely wasting
their time, IMHO, because there is NO increase in legibility. I
used to do that... same as how I used to waste time lining up the
equals signs in initialization sections of my code. (Now the
only place I do such things is in sections where series of related
constants are assigned, since only there does it really produce
increased readability).

In a lot of ways, this is probably the same argument as that against
using TABs. People who like TABs most often seem to like them
because they feel TABs make it easier to tune things to their own
particular (more aesthetically pleasing) indentation standards.

In fact, since they are gaining arguably nothing (for example,
there are arguments that could be made about four spaces being
the optimal size, if we wanted to go there), but are making
maintenance harder for others, it's a net negative in the greater
scheme of things, though perhaps easier on themselves at
least in the short term.

(Wow... a simultaneous segue into both the tab war and the editor
wars! I think I'd better leave now...)

-Peter

Ville Vainio

unread,
Jul 12, 2004, 11:06:23 AM7/12/04
to
>>>>> "Peter" == Peter Hansen <pe...@engcorp.com> writes:

Peter> But if you let emacs define the limits of your world, then
Peter> it's probably okay to do it that way. ;-)

It's just the path of least resistance - any other indentation
requires manual work, and won't reindent properly when pressing <tab>
on the line.

Anyway, the point is mostly moot because

blah = [
i + 1
for i in range(10)
if i != 3
]

indents the way you want it to, and that's the way I usually do it
anyway. I just M-x indent-region'ed your code snippet :).

Perhaps this is something that could be fixed in python-mode.el? I
mean making the indentation on the next line be previous_line + 4 in
bracket-opening situations... Or is the current behaviour better in
some generally agreed way?

François Pinard

unread,
Jul 12, 2004, 10:59:45 AM7/12/04
to Peter Hansen, pytho...@python.org
[Peter Hansen]
> François Pinard wrote:

> >... for me at least, the greatest virtue of Python is that one can
> >write it legibly, and maintainability mainly results from legibility.

> I don't find the legibility of the carefully aligned version of
> that code to be higher than the simpler "indent the standard amount"
> version of the code.

That's the important point. Write for legibility! :-)

> [...] same as how I used to waste time lining up the equals signs in


> initialization sections of my code.

Yes, this is often abused, with not much gain. Agreed!

> (Wow... a simultaneous segue into both the tab war and the editor
> wars! I think I'd better leave now...)

We'll surely meet somewhere else, anyway! :-)

LittleD...@yahoo.com

unread,
Jul 12, 2004, 11:57:16 AM7/12/04
to
>Luckily, the underlying model of LCs is very well understood (map &
>filter), and solving things the list-processing way is a time-honed
>practice. In the "amaze your friends" front, LCs are more in the "look
>how elegant and concise this can be" genre, not in the "check this
>out, all recursion and no variables, I bet you can't get it even after
>staring it for 5 minutes" genre loved by some academics.

I'd like to point out that list comprehensions were invented by people
who you'd probably put in that genre of academics, namely the creators
of Haskell. I don't really list comprehensions because the syntax is
too specific. In Kogut, a functional language that you'd probably
dismiss as being made by academics because it's functional, the syntax
for the original list comprehension at the beginning of the thread
would be
let result = Filter list ?elem {Part elem 0 4 == "blah"}
->Map (ReplacePart _ "moof" 0 4)
This syntax is very conducive to splitting up into multiple lines and
doesn't require much more explanation than list comprehensions do. I
really hate it when people say that academics are making programming
languages that are really hard to read; it reflects on the speaker's
stupidity more than anything else.

Daniel Ehrenberg

Isaac To

unread,
Jul 12, 2004, 11:36:16 AM7/12/04
to
>>>>> "Peter" == Peter Hansen <pe...@engcorp.com> writes:

Peter> If they do the alignment that way, making things carefully
Peter> line up with the parentheses, and manually, they are
Peter> definitely wasting their time, IMHO, because there is NO
Peter> increase in legibility.

Except that if you write things like

a=[[[1, 2],
[3, 4],
[5, 6]],
[[7, 8],
[5, 6]],
[9]]

as

a=[[[1, 2],
[3, 4],
[5, 6]],
[[7, 8],
[5, 6]],
[9]]

and say there is no lose of legibility, then I'll seriously question
your taste.

BTW, for somebody with python-mode installed and is using Emacs, it is
actually *harder* and a *bigger* waste of time to write the code as
you proposed, because by default whenever you press enter the cursor
is already at exactly the location in the former style, and you got to
count spaces in order to go to the place you proposed.

Regards,
Isaac.

Ville Vainio

unread,
Jul 12, 2004, 12:53:55 PM7/12/04
to
>>>>> "Danny" == LittleDanEhren <LittleD...@yahoo.com> writes:

Danny> I'd like to point out that list comprehensions were
Danny> invented by people who you'd probably put in that genre of
Danny> academics, namely the creators

Of course, and lots of other stuff too. Academics have great ideas,
and bad ideas. The stuff that flies is adopted in the industry, stuff
that doesn't is not. I'm by no means dismissing all academic research
on software as worthless (quite on the contrary) - especially when
it's research of new stuff. MIT and UCB have given us some great
things.

Danny> comprehensions do. I really hate it when people say that
Danny> academics are making programming languages that are really
Danny> hard to read; it reflects on the speaker's stupidity more
Danny> than anything else.

Indeed. 80% of software engineers should be suffocated silently in the
night, resulting in a master race of programmers (after 3-4
generations of breeding) that could reimplement all the software in
existence today with purely recursive algorithms. I bet the software
available then would rock.

Peter Hansen

unread,
Jul 12, 2004, 1:12:49 PM7/12/04
to
Ville Vainio wrote:

>>>>>>"Peter" == Peter Hansen <pe...@engcorp.com> writes:
>
>
> Peter> But if you let emacs define the limits of your world, then
> Peter> it's probably okay to do it that way. ;-)
>
> It's just the path of least resistance - any other indentation
> requires manual work, and won't reindent properly when pressing <tab>
> on the line.

Not sure what you mean here. The indentation approach I'm
referring to (i.e. the "standard" way, as opposed to what I
was calling the "artistic" way) is just to hit tab in precisely
the same way you would if you were indenting an "if" block.

> Anyway, the point is mostly moot because
>
> blah = [
> i + 1
> for i in range(10)
> if i != 3
> ]
>
> indents the way you want it to, and that's the way I usually do it
> anyway. I just M-x indent-region'ed your code snippet :).

Not sure which code-snippet you refer to, but the ones that I
typed were indented the same way, except perhaps that I put
the first expression the same line as the opening bracket.

> Perhaps this is something that could be fixed in python-mode.el? I
> mean making the indentation on the next line be previous_line + 4 in
> bracket-opening situations... Or is the current behaviour better in
> some generally agreed way?

Nope, definitely not in a generally agreed way. I don't think anyone
thinks this issue is on par with the tab war though, so until that
particular war is won, we're best off leaving this issue to personal
preference. (That is to say, "forever". :-)

-Peter

Peter Hansen

unread,
Jul 12, 2004, 1:18:18 PM7/12/04
to
Isaac To wrote:

> Except that if you write things like
>
> a=[[[1, 2],
> [3, 4],
> [5, 6]],
> [[7, 8],
> [5, 6]],
> [9]]
>
> as
>
> a=[[[1, 2],
> [3, 4],
> [5, 6]],
> [[7, 8],
> [5, 6]],
> [9]]
>
> and say there is no lose of legibility, then I'll seriously question
> your taste.

Ugh. But nobody should write something like the first one, either.
(And I won't say how it should be written without a real example,
since otherwise we can each contrive all kinds of interesting and
meaningless ways to format something that nobody would actually
write.)

> BTW, for somebody with python-mode installed and is using Emacs, it is
> actually *harder* and a *bigger* waste of time to write the code as
> you proposed, because by default whenever you press enter the cursor
> is already at exactly the location in the former style, and you got to
> count spaces in order to go to the place you proposed.

Not sure which one you think I proposed, but you definitely don't
have to count any spaces in order to do what I suggested. You
hit ENTER and TAB (once), exactly as you would with any other
block. Note that this (hitting TAB once) of course relies on
an editor which auto-indents to the same level as the preceding
line of code by default. Any editor which does not do at least
this, these days, is broken. But counting spaces would be silly.
What editor are you using that would force you to do that? vi
without a Python configuration?

-Peter

Tim Peters

unread,
Jul 12, 2004, 1:45:15 PM7/12/04
to pytho...@python.org
[littled...@yahoo.com]

> I'd like to point out that list comprehensions were invented by people
> who you'd probably put in that genre of academics, namely the creators
> of Haskell.

Na, Haskell's creators adapted them from the older SETL language, who
in turn were inspired by set theory's so-called Axiom of Comprehension
(which is why they're called "comprehensions" to begin with).

Of course that doesn't detract from your point that the OP would class
them as academics <wink>.

Larry Bates

unread,
Jul 12, 2004, 3:03:58 PM7/12/04
to
I don't agree that the second is "less maintainable",
but do find it "wordy" for little reason. Try:

result=[x.replace('blah','moof') for x in L if x.startswith('blah')]

or if you prefer short lines:

result=[x.replace('blah','moof') \
for x in L \
if x.startswith('blah')]

Note: BAD idea to call variable 'list' as it kills the
list function for remainder of Python session (I changed
to L). For same reason never call variable 'dict', 'str',
etc. Using startswith also takes care of not having to
get the slice correct (which is easier to maintain should
you change 'blah' to something else, you don't have to
mess with the slice).

Larry Bates
Syscon, Inc.

"Moosebumps" <cr...@crap.crap> wrote in message
news:j9NHc.8596$mJ3....@newssvr25.news.prodigy.com...


> Does anyone here find the list comprehension syntax awkward?
>

> I like it because it is an expression rather than a series of statements,
> but it is a little harder to maintain it seems.
>
> e.g. you could do:
>
> result = []
> for element in list:
> if element[0:4] == 'blah':
> result.append( element.replace( 'blah', 'moof' ) )
>
> or just:


>
> result = [ element.replace( 'blah', 'moof' ) for element in list if
> element[0:4] == 'blah' ]
>

> The second looks cleaner in some cases, but it is less maintainable. It
> tends to promote long lines. It all seems to run together and such. And
> often you would need to add another condition or modify it, it seems
better
> to use the first way even though it has the ugly extra init (and which
would
> cause more allocs than the list comprehension? because it wouldn't know
the
> size off the bat?)
>
> I am in favor of short lines like I think Guido said in the style guide.
I
> like each line to be so simple as to not require any thinking reading it,
> e.g.
>
> I prefer:
>
> x = c( d, e )
> y = f( g, h )
> z = b( x, y )
> w = a( z )
>
> to stuff like this:
>
> w = a( b( c( d, e ), f( g, h ) ) )
>
> It is more maintainable, when you need to make a change, just insert a
line,
> rather than having to decode an expression.
>
> Along the same lines, it seems more maintainable to split things up.
>
> You could do:


>
> result = [
> element.replace( 'blah', 'moof' )
> for element in list
> if element[0:4] == 'blah' ]
>

> I guess, but that seems awkward to me. Looks too much like a for loop and
> an if, and then the value is at the top, which reads funny to me.
> (Strangely putting it on one line doesn't read as funny, but it is less
> readable.) Maybe I just have to get used to it. Which do you prefer?
> Comments?
>

> MB
>
>


Ville Vainio

unread,
Jul 12, 2004, 4:05:34 PM7/12/04
to
>>>>> "Peter" == Peter Hansen <pe...@engcorp.com> writes:

Peter> Not sure what you mean here. The indentation approach I'm
Peter> referring to (i.e. the "standard" way, as opposed to what I
Peter> was calling the "artistic" way) is just to hit tab in
Peter> precisely the same way you would if you were indenting an
Peter> "if" block.

In emacs, tab is 'indent-line', which gives the line "correct"
indentation. Occasionally one needs to dedent a line manually to close
a block. Pressing <enter> puts the cursor in the right place
immediately (indenting on ':', dedenting on 'return').

if 1:
hui();

If I press tab on line with hui(), the result is

if 1:
hui();


>> Anyway, the point is mostly moot because
>> blah = [
>> i + 1
>> for i in range(10)
>> if i != 3
>> ]
>> indents the way you want it to, and that's the way I usually do it
>> anyway. I just M-x indent-region'ed your code snippet :).

Peter> Not sure which code-snippet you refer to, but the ones that
Peter> I typed were indented the same way, except perhaps that I
Peter> put the first expression the same line as the opening
Peter> bracket.

Yes, and that made the difference.

Aahz

unread,
Jul 22, 2004, 10:09:39 AM7/22/04
to
In article <HI7Ic.17372$1Z7....@newssvr27.news.prodigy.com>,

Moosebumps <cr...@crap.crap> wrote:
>
>>>> result = [x for x in range(10) if x % 2 == 0 if x % 3 == 0]
>>>> result
>[0, 6]
>>>> result = [ x*y for x in range(10) if x%2 == 0 for y in range(10) if y %
>3 == 0]
>>>> result
>[0, 0, 0, 0, 0, 6, 12, 18, 0, 12, 24, 36, 0, 18, 36, 54, 0, 24, 48, 72]
>
>Just curious -- anyone care to tell me how they would format the above? (or
>maybe you wouldn't write it all)

Wouldn't use either.
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/

Barbara Boxer speaks for me:
http://buffaloreport.com/2004/040713.boxer.marriage.html

David Eppstein

unread,
Jul 22, 2004, 1:45:51 PM7/22/04
to
In article <cdohr3$680$1...@panix2.panix.com>, aa...@pythoncraft.com (Aahz)
wrote:

> In article <HI7Ic.17372$1Z7....@newssvr27.news.prodigy.com>,
> Moosebumps <cr...@crap.crap> wrote:
> >
> >>>> result = [x for x in range(10) if x % 2 == 0 if x % 3 == 0]
> >>>> result
> >[0, 6]
> >>>> result = [ x*y for x in range(10) if x%2 == 0 for y in range(10) if y %
> >3 == 0]
> >>>> result
> >[0, 0, 0, 0, 0, 6, 12, 18, 0, 12, 24, 36, 0, 18, 36, 54, 0, 24, 48, 72]
> >
> >Just curious -- anyone care to tell me how they would format the above? (or
> >maybe you wouldn't write it all)
>
> Wouldn't use either.

I would probably use instead something like

>>> [x for x in range(0,10,6)]
[0, 6]
>>> [x*y for x in range(0,10,2) for y in range(0,10,3)]


[0, 0, 0, 0, 0, 6, 12, 18, 0, 12, 24, 36, 0, 18, 36, 54, 0, 24, 48, 72]

I think two for or if clauses in a comprehension is the most I'd want to
use on a regular basis. I certainly wouldn't use two ifs in a row as in
your first example, I'd use and instead.

--
David Eppstein
Computer Science Dept., Univ. of California, Irvine
http://www.ics.uci.edu/~eppstein/

0 new messages