a break for comprehensions
Kevin Lacker
answer = []
for x in my_list:
if not is_good(x):
break
answer.append(process(x))
with a list comprehension somehow? I'm starting to dislike explicit for
loops just for the purposes of constructing another list from a current one.
Would this work in the future python with iterators...
def done(): raise StopIteration
answer = [process(x) for x in my_list if is_good(x) or done()]
If I used it a lot it might not be a pain to define done(). I think it would
be cool to be able to reuse the while keyword inside comprehensions like
answer = [process(x) for x in my_list while is_good(x)]
Makes sense to me... it's like being able to use a sentinel with iterators
but more flexible.
Tom Jenkins
> Can you do this:
>
> answer = []
> for x in my_list:
> if not is_good(x):
> break
> answer.append(process(x))
>
> with a list comprehension somehow? I'm starting to dislike explicit for
> loops just for the purposes of constructing another list from a current one.
do you mean like this?
>>> def even(x):
... return int(divmod(x,2)[1]) == 0
...
>>> a = [1, 2, 3, 4, 5, 6, 7]
>>> b = [z for z in a if not even(z)]
>>> print b
[1, 3, 5, 7]
>>> c = [z for z in a if even(z)]
>>> print c
[2, 4, 6]
>>>
oops sorry just saw that you had process(x) in there... lets try it:
>>> def process(x):
... return str(x)
...
>>> process(1)
'1'
>>> c = [process(z) for z in a if even(z)]
>>> print c
['2', '4', '6']
>>>
hmmm, seems to work for me (python 2.1)
Marcin 'Qrczak' Kowalczyk
> Can you do this:
>
> answer = []
> for x in my_list:
> if not is_good(x):
> break
> answer.append(process(x))
>
> with a list comprehension somehow?
The pattern can be wrapped in a function. It's quite nice in Python 2.2:
def take_while(p, l):
for x in l:
if not p(x): break
yield x
answer = [process(x) for x in take_while(is_good, my_list)]
> I think it would be cool to be able to reuse the while keyword
> inside comprehensions like
>
> answer = [process(x) for x in my_list while is_good(x)]
Looks nice for me.
--
__("< Marcin Kowalczyk * qrc...@knm.org.pl http://qrczak.ids.net.pl/
\__/
^^ SYGNATURA ZASTĘPCZA
QRCZAK
Nick Perkins
"Tom Jenkins" <tjen...@nospiced.ham.devis.com> wrote in message
news:3B6031C3...@nospiced.ham.devis.com...
This misses the point of the original post.
Notice the 'break' in the original for-loop version.
The idea is to return consecutive elements,
up to, but not including, the first element which
does not meet the condition.
ie.
lst = [0,2,4,5,6]
evens = [0,2,4]
I don't think list comprehensions can do this.
Here's a generator approach to the problem:
from __future__ import generators
def conditional_iter(lst,cond):
for item in lst:
if cond(item):
yield item
else:
raise StopIteration
mylist = [0,2,4,5,6]
def even(num): return not (num%2)
for num in conditional_iter(mylist, even)
print num
Tom Jenkins
>
> This misses the point of the original post.
> Notice the 'break' in the original for-loop version.
> The idea is to return consecutive elements,
> up to, but not including, the first element which
> does not meet the condition.
>
> ie.
> lst = [0,2,4,5,6]
> evens = [0,2,4]
>
*sigh* yep missed that break
now-if-my-newsgroup-reader-only-had-python-syntax-highlighting-ly yours
Tom
Terry Reedy
"Tom Jenkins" <tjen...@nospiced.ham.devis.com> wrote in message
news:3B6031C3...@nospiced.ham.devis.com...
>
> > Can you do this:
> >
> > answer = []
> > for x in my_list:
> > if not is_good(x):
> > break
> > answer.append(process(x))
> >
> > with a list comprehension somehow? I'm starting to dislike
explicit for
> > loops just for the purposes of constructing another list from a
current one.
>
>
> do you mean like this?
[filter the list]
No, 'break' stops the for loop. Good things after non-good x do not
get processed and appended.
Terry J. Reedy
Tom Good
You could simulate a "while...break" within comprehensions using a
generator, like this:
#---- begin code
def itrWhile(obj, func):
itr = iter(obj)
while 1:
x = itr.next()
if func(x):
yield x
else:
return
#---- end code
Then:
>>> def isGood(n):
... return n < 4
...
>>> a = [1,2,3,4,5,6,7]
>>> [x for x in itrWhile(a, isGood)]
[1, 2, 3]
>>>
Notice the important difference between this and "[x for x in a if
isGood(x)]"
>>> def isGood(n):
... print "comparing", n
... return n < 4
...
>>> [x for x in a if isGood(x)]
comparing 1
comparing 2
comparing 3
comparing 4
comparing 5
comparing 6
comparing 7
[1, 2, 3]
>>> [x for x in itrWhile(a, isGood)]
comparing 1
comparing 2
comparing 3
comparing 4
[1, 2, 3]
>>>
The "itrWhile" version breaks out after the first unsuccessful
comparison. The version using "if" tests every element.
Tom
Steven D. Majewski
>>> [ x for x in iter( iter([1,2,3,4,5,6,7,8,9]).next, 6 ) ]
[1, 2, 3, 4, 5]
>>> print iter.__doc__
iter(collection) -> iterator
iter(callable, sentinel) -> iterator
Get an iterator from an object. In the first form, the argument must
supply its own iterator, or be a sequence.
In the second form, the callable is called until it returns the sentinel.
>>>
iter(list).next gives you a callable object that sequentially
returns each value -- which gets fed to the 2 arg variant
of iter that takes a callable and a sentinel. When it hits
the sentinal value, it stops.
-- Steve Majewski
Terry Reedy
"Nick Perkins" <nper...@home.com> wrote in message
news:xYX77.563869$eK2.11...@news4.rdc1.on.home.com...
> from __future__ import generators
>
> def conditional_iter(lst,cond):
> for item in lst:
> if cond(item):
> yield item
> else:
> raise StopIteration
>
>
> mylist = [0,2,4,5,6]
> def even(num): return not (num%2)
>
> for num in conditional_iter(mylist, even)
> print num
Will the generator also work in a comprehension (and answer the
original question) as in
list2 = [i for i in conditional_iter(mylist, even)]
?
phy...@mailcity.com
of parallelism. The original idea of list comprehensions does _not_ include
the order in which the elements are evaluated. Because of that they can all
be evaluated in parallel - a compiler/interpreter can decide to do that.
Then, with side effects possible inside function calls I guess it may not
matter that much in Python.
"Marcin 'Qrczak' Kowalczyk" <qrc...@knm.org.pl> wrote in message
news:slrn.pl.9m0e...@qrnik.zagroda...
Peter Hansen
>
> Can you do this:
>
> answer = []
> for x in my_list:
> if not is_good(x):
> break
> answer.append(process(x))
>
> with a list comprehension somehow? I'm starting to dislike explicit for
> loops just for the purposes of constructing another list from a current one.
I can clearly understand what the above code is doing. Doing this
as a list comprehension would be less readable for me (I'm not a
heavy user of list comprehensions, yet.)
Are you sure you really don't like the explicit, readable code
above, and would prefer a more compressed, (admittedly explicit,
I suppose), more cryptic version instead? There are other
languages which could do the above in one tiny line, no doubt.
(No names. :)
--
----------------------
Peter Hansen, P.Eng.
pe...@engcorp.com
Kevin Lacker
> > ...
> > with a list comprehension somehow? I'm starting to dislike explicit for
> > loops just for the purposes of constructing another list from a current
one.
>
> I can clearly understand what the above code is doing. Doing this
> as a list comprehension would be less readable for me (I'm not a
> heavy user of list comprehensions, yet.)
>
> Are you sure you really don't like the explicit, readable code
> above, and would prefer a more compressed, (admittedly explicit,
> I suppose), more cryptic version instead? There are other
> languages which could do the above in one tiny line, no doubt.
> (No names. :)
How cryptic is
[process(x) for x in my_list while is_good(x)]
? Not really any more cryptic than
[process(x) for x in my_list if is_good(x)]
which seems to be agreeably a good thing to have. I certainly like it.
I don't really know much Lisp, but in the back of my mind I half remember
someone claiming they had more flexible iteration abilities that still
looked functional and noncryptic...
Peter Hansen
>
> How cryptic is
>
> [process(x) for x in my_list while is_good(x)]
>
> ? Not really any more cryptic than
>
> [process(x) for x in my_list if is_good(x)]
>
> which seems to be agreeably a good thing to have. I certainly like it.
Assuming you can read the one, the other is at least as
readable.
In my post I was attempting to point out that I find list
comprehensions less readable than the equivalent, longer but
more explicit while loop version, not that one list comprehension
is more or less cryptic than another.
(I'm quite sure in time list comprehensions will be quite
readable to me. Eventually even Perl can become readable
to someone. That doesn't mean the meaning of either is
obvious to a newcomer, which is what bothers me about both.)
Paul Svensson
>Kevin Lacker wrote:
>>
>> How cryptic is
>>
>> [process(x) for x in my_list while is_good(x)]
>>
>> ? Not really any more cryptic than
>>
>> [process(x) for x in my_list if is_good(x)]
>>
>> which seems to be agreeably a good thing to have. I certainly like it.
>
>Assuming you can read the one, the other is at least as
>readable.
>
>In my post I was attempting to point out that I find list
>comprehensions less readable than the equivalent, longer but
>more explicit while loop version, not that one list comprehension
>is more or less cryptic than another.
In my mind, the most important difference between a list comprehension
and the explicit loop replacement, is that the latter distracts my attention
away from the data going into the list, focusing too much on the process
of putting it there.
/Paul
phaw...@connact.com
PH> In my post I was attempting to point out that I find list
PH> comprehensions less readable than the equivalent, longer but
PH> more explicit while loop version, not that one list comprehension
PH> is more or less cryptic than another.
Personally, as soon as I saw list comprehensions, I found them
splendidly, instantly readable -- so long as they aren't nested, I can
read them at a glance, where it takes me just a little longer to
comprehend all of a loop. This is certainly true for the examples
we've been looking at. So I think this just has to do with how one's
mind works.
and-no-I-don't-find-perl-at-all-readable-ly yr's
Patricia
--
Patricia J. Hawkins
Hawkins Internet Applications, LLC
Tom Jenkins
>>>>>>"PH" == Peter Hansen <pe...@engcorp.com> writes:
>>>>>>
>
> PH> In my post I was attempting to point out that I find list
> PH> comprehensions less readable than the equivalent, longer but
> PH> more explicit while loop version, not that one list comprehension
> PH> is more or less cryptic than another.
>
> Personally, as soon as I saw list comprehensions, I found them
> splendidly, instantly readable -- so long as they aren't nested, I can
> read them at a glance, where it takes me just a little longer to
> comprehend all of a loop. This is certainly true for the examples
> we've been looking at. So I think this just has to do with how one's
> mind works.
I couldn't read list comprehensions until I actually started playing
around with them. After experimenting and writing (oh about 3 or 4) I
discovered I could read them ;)
>
> and-no-I-don't-find-perl-at-all-readable-ly yr's
> Patricia
>
and-i-don't-find-lisp-readable-either-although-with-list-comprehensions-i'm-starting-to-ly
yours
Tom
Peter Hansen
>
> >>>>>>"PH" == Peter Hansen <pe...@engcorp.com> writes:
> > PH> In my post I was attempting to point out that I find list
> > PH> comprehensions less readable than the equivalent, longer but
> > PH> more explicit while loop version, not that one list comprehension
> > PH> is more or less cryptic than another.
> I couldn't read list comprehensions until I actually started playing
> around with them. After experimenting and writing (oh about 3 or 4) I
> discovered I could read them ;)
These responses are good to hear. I'm happy to know that
after only 5 or 6 experiments (I'm a little slower than you
guys :-), I'll find list comprehensions less cryptic and
much more readable.
Greg Ewing
>
> answer = [process(x) for x in my_list while is_good(x)]
Looks quite nice. But what should
answer = [process(x,y) for x in a for y in b while f(x,y)]
do? Should the break terminate the whole comprehension,
or just the enclosing for-clause?
--
Greg Ewing, Computer Science Dept, University of Canterbury,
Christchurch, New Zealand
To get my email address, please visit my web page:
http://www.cosc.canterbury.ac.nz/~greg
Paul Svensson
>Kevin Lacker wrote:
>>
>> answer = [process(x) for x in my_list while is_good(x)]
>
>Looks quite nice. But what should
>
> answer = [process(x,y) for x in a for y in b while f(x,y)]
>
>do? Should the break terminate the whole comprehension,
>or just the enclosing for-clause?
In my view, it should obviously end the whole thing; but then I
never see list comprehensions a procedural description for how
to generate the list, but rather a symbolic description of
the list content. With that view it shouldn't matter which
for-clause gets evaluated innermost, and so the only unambigous
thing for the while clause to end, is the whole comprehension.
/Paul
Steven D. Majewski
On 30 Jul 2001, Paul Svensson wrote:
> In my view, it should obviously end the whole thing; but then I
> never see list comprehensions a procedural description for how
> to generate the list, but rather a symbolic description of
> the list content. With that view it shouldn't matter which
> for-clause gets evaluated innermost, and so the only unambigous
> thing for the while clause to end, is the whole comprehension.
You hit the nail on the head there, Paul!
The 'while' clause seems useful, and I don't see anything inherently
ambiguous about it, but sticking a procedural clause into what's
otherwise a declarative seens to go against the spirit -- what's
nice about list comprehensions is that they are just like set
notations. ( well -- the left hand side is often procedural -- it
would be even more readable if it were more clearly separated
with a "where" clause, but that would be Yet Another Keyword... )
-- Steve Majewski
Tom Good
>
> The 'while' clause seems useful, and I don't see anything inherently
> ambiguous about it, but sticking a procedural clause into what's
> otherwise a declarative seens to go against the spirit -- what's
> nice about list comprehensions is that they are just like set
> notations. ( well -- the left hand side is often procedural -- it
> would be even more readable if it were more clearly separated
> with a "where" clause, but that would be Yet Another Keyword... )
>
> -- Steve Majewski
That is a compelling argument. However, I am still struggling with
how to best work with infinite sets (which are now easy to create with
generators). With set notation, you can refer to an infinite set
without actually creating each member of the set. List comprehensions
try to "fill up" the list, which causes problems if an unbounded
generator is involved.
If g is a generator that generates all positive integers, I would like
to write something like:
L = [x * 2 for x in g()]
print L[:10]
This does not work, of course, but that is how I would tend to think
about it in my head. List comprehensions with lazy evaluation? Is it
possible?
Tom
Steve Holden
news:ac677656.01073...@posting.google.com...
inherently impossible about the introduction of such a concept. A major
problem with existing list comprehensions, however, is that they produce a
list! Seems that a "lazy comprehension" would have to encapsulate a
generator somehow, and limit the operations you could perform on it to those
for generators.
I believe a separate syntax would be required to represent such a radically
different object. But you, of course, may know better ... if you're looking
for a syntax, one possibility would be:
[[x * 2 for x in g()]]
regards
Steve
--
http://www.holdenweb.com/
Peter Hansen
>
> I believe a separate syntax would be required to represent such a radically
> different object. But you, of course, may know better ... if you're looking
> for a syntax, one possibility would be:
>
> [[x * 2 for x in g()]]
If that doesn't refer to a list comprehension formed inside another list,
you'd be well on your way to reinventing another language whose name begins
with P...
Ken Seehof
That particular syntax won't do since it would seem to require two new
tokens
"[[" and "]]" which would be really nasty since there's already semantics
for those
character sequences (i.e. [[1,2,3]] is a list containing the list [1,2,3]).
You (Steve) are right IMO, that new language syntax would be required. In
addition to the reasons you mention, it would not always unreasonable to
return
a list when a generator is being iterated, since the generator might be
finite.
Conversely, one might want a generator comprehension based on some other
arbitrary kind of sequence object.
How about [... x * 2 for x in g()]
- Ken Seehof
Paul Prescod
>
> ...
>
> How about this:
>
> def f():
> for x in g():
> yield x * 2
>
> Of course it gives a generator, not a list... but that
> seems to be a reasonable form to represent an infinite sequence.
How about if we combine your ideas:
y = [yield x*2 for x in g()]
--
Take a recipe. Leave a recipe.
Python Cookbook! http://www.ActiveState.com/pythoncookbook
Michael Chermside
> radically
> > different object. But you, of course, may know better ... if you're
> looking
> > for a syntax, one possibility would be:
> >
> > [[x * 2 for x in g()]]
> >
> > regards
> > Steve
>
> That particular syntax won't do since
>
> How about [... x * 2 for x in g()]
>
How about this:
def f():
for x in g():
yield x * 2
Of course it gives a generator, not a list... but that
seems to be a reasonable form to represent an infinite sequence.
-- Michael Chermside
Ken Seehof
> Michael Chermside wrote:
> >
> > ...
> >
> > How about this:
> >
> > def f():
> > for x in g():
> > yield x * 2
> >
> > Of course it gives a generator, not a list... but that
> > seems to be a reasonable form to represent an infinite sequence.
>
>
> y = [yield x*2 for x in g()]
Geeze, that's like, way obvious. Why didn't I say that in the first place?
Good stuff is always way obvious. Nice one Paul :-)
What do you think Guido? => CVS?
Cliff Crawford
|
| I believe a separate syntax would be required to represent such a radically
| different object. But you, of course, may know better ... if you're looking
| for a syntax, one possibility would be:
|
| [[x * 2 for x in g()]]
Another possibility would be to have list comprehensions always return a
generator, and then if you need a list instead, use the built-in
function list(), for example:
list([x*2 for x in g()])
But this will cause code breakage..maybe the solution to *that* would be
to add __getitem__(), __len__(), __contains__(), etc. to generator
objects, so that they can be manipulated as if they were lists. (This
would solve the original poster's problem too -- you could do g()[:50]
to get the first 50 items of a generator.)
--
Cliff Crawford http://www.sowrong.org/
A sign should be posted over every campus toilet:
"This flush comes to you by courtesy of capitalism."
-- Camille Paglia
Guido van Rossum
Guido van Rossum
> >
> > y = [yield x*2 for x in g()]
>
> Geeze, that's like, way obvious. Why didn't I say that in the first place?
> Good stuff is always way obvious. Nice one Paul :-)
>
> What do you think Guido? => CVS?
I don't think so. Generators are most useful when they do more than
iterate over a pre-existing sequence, so I doubt that this would be a
common pattern. Also I'm not sure I like overloading of the [...]
brackets, which (to me) very strongly say "real concrete list here".
--Guido van Rossum (home page: http://www.python.org/~guido/)
Ken Seehof
I'm assuming that this would be used mostly when g() is a generator,
but the sequence being iterated could be any lazy or non-lazy sequence.
In practice, this puts a filter on an existing lazy sequence. Unfortunately
the filter is limited to a one-to-one mapping of the input sequence, but
that's true of list comprehensions too isn't it?
I think the yield keyword clearly (to me) disambiguates the expression.
Well, come to think of it, I can't think of a really good use for generator
comprehensions anyway, but you have to admit the idea is cute. :-)
- Ken Seehof http://www.neuralintegrator.com/kseehof
Roman Suzi
>Michael Chermside wrote:
>>
>> ...
>>
>> How about this:
>>
>> def f():
>> for x in g():
>> yield x * 2
>>
>> Of course it gives a generator, not a list... but that
>> seems to be a reasonable form to represent an infinite sequence.
>
>How about if we combine your ideas:
>
>y = [yield x*2 for x in g()]
O, yes! Generators need lambda form too!
Sincerely yours, Roman Suzi
--
_/ Russia _/ Karelia _/ Petrozavodsk _/ r...@onego.ru _/
_/ Tuesday, July 31, 2001 _/ Powered by Linux RedHat 6.2 _/
_/ "All things are green unless they are not." _/
Alex Martelli
news:mailman.996643861...@python.org...
...
> In practice, this puts a filter on an existing lazy sequence.
Unfortunately
> the filter is limited to a one-to-one mapping of the input sequence, but
> that's true of list comprehensions too isn't it?
Many-to-one, in both cases, e.g.
[somefun(x) for x in seq if somecond(x)]
One-to-many (thus clearly also many-to-many) can be obtained for
concrete sequences with the help of another 'nested' for:
[somefun2(x,i) for x in seq for i in range(3) if somecond2(x,i)]
may end up with 0 to 2 items of the output sequence for each
item x of the input sequence seq.
Alex
Tom Good
[snip]
> Another possibility would be to have list comprehensions always return a
> generator, and then if you need a list instead, use the built-in
> function list(), for example:
>
> list([x*2 for x in g()])
>
> But this will cause code breakage..maybe the solution to *that* would be
> to add __getitem__(), __len__(), __contains__(), etc. to generator
> objects, so that they can be manipulated as if they were lists. (This
> would solve the original poster's problem too -- you could do g()[:50]
> to get the first 50 items of a generator.)
You've inspired me to start writing a list-like wrapper for
generators. Here is what I have so far. It combines some list syntax
with lazy evaluation, and supports (some) slice operations on
unbounded generators.
from __future__ import generators
class GenList:
"wraps a generator so it will support some list operations"
def __init__(self, gen):
self.gen = gen
self.cachedResults = []
def genNext(self):
self.cachedResults.append(self.gen.next())
def __getitem__(self, i):
if i < 0:
raise IndexError
# negative indices would not work with
# unbounded generators
try:
while len(self.cachedResults) <= i:
self.genNext()
except StopIteration:
pass
return self.cachedResults[i]
def __getslice__(self, i, j):
if i < 0 or j < 0:
raise IndexError
# negative indices would not work with
# unbounded generators
try:
while len(self.cachedResults) <= j:
self.genNext()
except StopIteration:
pass
return self.cachedResults[i:j]
def main():
def gen():
x = 0
while 1:
x += 1
yield x
glist = GenList(gen())
print glist[4]
print glist[:10]
Alex Martelli
news:9k44u5$fua$1...@newsy.ifm.liu.se...
...
> never see list comprehensions a procedural description for how
> to generate the list, but rather a symbolic description of
> the list content. With that view it shouldn't matter which
> for-clause gets evaluated innermost, and so the only unambigous
Sorry, but it still does: [x-y for x in range(4) for y in range(4)]
is [0, -1, -2, -3, 1, 0, -1, -2, 2, 1, 0, -1, 3, 2, 1, 0], while
[x-y for y in range(4) for x in range(4)] is the different list
[0, 1, 2, 3, -1, 0, 1, 2, -2, -1, 0, 1, -3, -2, -1, 0]. They
might be the same if seen as sets, but, as lists, they aren't.
Alex
Cliff Crawford
|
| You've inspired me to start writing a list-like wrapper for
| generators. Here is what I have so far. It combines some list syntax
| with lazy evaluation, and supports (some) slice operations on
| unbounded generators.
|
| def main():
| def gen():
| x = 0
| while 1:
| x += 1
| yield x
|
| glist = GenList(gen())
| print glist[4]
| print glist[:10]
This is neat :) It looks like your class will work with any iterator,
not just generators. So you could use it with, say, file objects:
file = GenList(iter(open('file.txt')))
print file[4]
print file[3:17]
# etc.
Also, I would propose adding
def __add__(iter1, iter2):
def f():
for val in iter1:
yield val
for val in iter2:
yield val
return GenList(f)
Then you can do stuff like the following, which simulates the Unix 'cat'
command (warning: untested):
files = reduce(lambda x, y: x+y, GenList(iter(open(sys.argv[1:]))))
for line in files:
print line,
--
Cliff Crawford http://www.sowrong.org/
"Today's Goths, who generally tend to be in their teens and 20s,
have nothing to do with the Germanic Visigoths of Europe in the
third and fourth centuries A.D." -- Jimmy Stewart
Cliff Crawford
|
| Then you can do stuff like the following, which simulates the Unix 'cat'
| command (warning: untested):
|
| files = reduce(lambda x, y: x+y, GenList(iter(open(sys.argv[1:]))))
Ooops, that should be
files = reduce(lambda x, y: x+y,
[GenList(iter(open(filename))) for filename in sys.argv[1:]])
Alex Martelli
news:L_nc7.241033$EF2.29...@typhoon.nyroc.rr.com...
> * Cliff Crawford <cj...@nospam.cornell.edu> menulis:
> |
> | Then you can do stuff like the following, which simulates the Unix 'cat'
> | command (warning: untested):
> |
> | files = reduce(lambda x, y: x+y, GenList(iter(open(sys.argv[1:]))))
>
> Ooops, that should be
>
> files = reduce(lambda x, y: x+y,
> [GenList(iter(open(filename))) for filename in sys.argv[1:]])
I think it should be
files = reduce(operator.add,
# whatever
after an import operator somewhere, of course. I strongly
suggest NOT using lambda to duplicate the functions already
supplied by module operator -- it's slow and messy!-)
Alex