Here is a question about list comprehensions [lc]. The question is dumb because I can do without [lc]; but I am posing the question because I am curious.
This:
>>> data = [['foo','bar','baz'],['my','your'],['holy','grail']] >>> result = [] >>> for d in data:
Take advantage of the fact that you can have more than one 'for' in a list comprehension:
>>> data = [['foo','bar','baz'],['my','your'],['holy','grail']] >>> [item for item_list in data for item in item_list] ['foo', 'bar', 'baz', 'my', 'your', 'holy', 'grail']
Will Stuyvesant wrote: >>>> data = [['foo','bar','baz'],['my','your'],['holy','grail']] >>>> result = [] >>>> for d in data: > ... for w in d: > ... result.append(w) >>>> print result > ['foo', 'bar', 'baz', 'my', 'your', 'holy', 'grail']
> puts all the words in a list, like I want.
> How to do this with [lc] instead of for-loops? >>> data = [['foo','bar','baz'],['my','your'],['holy','grail']] >>> [w for d in data for w in d]
def unroll(ary): unrolled = [] for item in ary: # add test for your favorite sequence type if ( type(item) == types.ListType or \ type(item) == types.TupleType \ ): unrolled.extend(unroll(item)) else: unrolled.append(item) return unrolled
> Here is a question about list comprehensions [lc]. The > question is dumb because I can do without [lc]; but I am > posing the question because I am curious.
> This: > >>> data = [['foo','bar','baz'],['my','your'],['holy','grail']] > >>> result = [] > >>> for d in data:
> I tried funnies like [[w for w in L] for L in data], > that is correct syntax, but you'd never guess.
> I know, silly! No need for [lc]! So there's my > question. I am sure a one-liner using [lc] will be very > enlightening. Like studying LISP.
-- James Stroud, Ph.D. UCLA-DOE Institute for Genomics and Proteomics 611 Charles E. Young Dr. S. MBI 205, UCLA 951570 Los Angeles CA 90095-1570 http://www.jamesstroud.com/
Will Stuyvesant wrote: > Here is a question about list comprehensions [lc]. The > question is dumb because I can do without [lc]; but I am > posing the question because I am curious.
> This:
>>>>data = [['foo','bar','baz'],['my','your'],['holy','grail']] >>>>result = [] >>>>for d in data:
There's a way to avoid generating the intermediate list if you don't actually need it (e.g. you want to feed the sequence to another method):
.>>> data = [['foo','bar','baz'],['my','your'],['holy','grail']] .>>> from itertools import chain .>>> result = "".join(chain(*data)) 'foobarbazmyyourholygrail'
Some timing with integers:
C:\>python -m timeit -s "data = [range(x) for x in range(1000)]" "L= []; map(L.e xtend, data); max(L)" 10 loops, best of 3: 78.5 msec per loop
C:\>python -m timeit -s "data = [range(x) for x in range(1000)]; from Tkinter im port _flatten" "max(_flatten(data))" 10 loops, best of 3: 58.4 msec per loop
C:\>python -m timeit -s "data = [range(x) for x in range(1000)]; from itertools import chain" "max(chain(*data))" 10 loops, best of 3: 43 msec per loop
And with strings:
C:\>python -m timeit -s "data = [map(str, range(x)) for x in range(1000)]" "L= [ ]; map(L.extend, data); ''.join(L)" 10 loops, best of 3: 106 msec per loop
C:\>python -m timeit -s "data = [map(str, range(x)) for x in range(1000)]; from Tkinter import _flatten" "''.join(_flatten(data))" 10 loops, best of 3: 85.4 msec per loop
C:\>python -m timeit -s "data = [map(str, range(x)) for x in range(1000)]; from itertools import chain" "''.join(chain(*data))" 10 loops, best of 3: 1.2 sec per loop ****** OUCH!!
C:\>python -m timeit -s "data = [map(str, range(x)) for x in range(1000)]; from itertools import chain" "''.join(list(chain(*data)))" 10 loops, best of 3: 107 msec per loop
Yikes - looks like chain() really sucks for str.join. However, the addition of the 'list' call makes a big difference. Maybe PySequence_Fast should be called PySequence_SlowAsADeadDingo() in this case ;)
(FYI, I filed bug report #1085744 on SF about this)
Diez B. Roggisch wrote: > >>> data = [['foo','bar','baz'],['my','your'],['holy','grail']] > >>> [e for l in data for e in l] > ['foo', 'bar', 'baz', 'my', 'your', 'holy', 'grail']
Okay, I tried this in an interactive Python session and it works as stated. My question is, why? How is the interpreter parsing that list expression that it makes sense?
Matthew Moss wrote: >> >>> data = [['foo','bar','baz'],['my','your'],['holy','grail']] >> >>> [e for l in data for e in l] >> ['foo', 'bar', 'baz', 'my', 'your', 'holy', 'grail']
> Okay, I tried this in an interactive Python session and it works as > stated. My question is, why? How is the interpreter parsing that list > expression that it makes sense?
"... the elements of the new list are those that would be produced by considering each of the 'for' or 'if' clauses a block, nesting from left to right, and evaluating the expression to produce a list element each time the innermost block is reached."
or, conceptually, the compiler strips off the expression, and then inserts newlines, colons and indentation, and wraps the expression in an append statement.
so
[e for l in data for e in l]
behaves like:
result = [] for l in data: for e in l: result.append(e)
except that it's an expression, and the result variable is hidden. (and the compiler and the runtime is of course free to implement this in a more efficient way)
Okay that was fun. Enlightening as I hoped. unroll() in Python, for arbitrary depth, _flatten in Tkinter (what else is in Tkinter!), sum() abuse.
The sum(data,[]) was funniest, it works like ((['foo','bar'] + []) + ['my','your']) + ['holy','grail']. Before I think of such things I have already coded an algorithm in imperative style. Guess I have not been exposed to functional programming enough.
>>>>> data = [['foo','bar','baz'],['my','your'],['holy','grail']] >>>>> result = [] >>>>> for d in data:
> .>>> data = [['foo','bar','baz'],['my','your'],['holy','grail']] > .>>> from itertools import chain > .>>> result = "".join(chain(*data)) > 'foobarbazmyyourholygrail'
This is the first time I see that and I totally like the idea of writing ".>>>" instead of ">>>" at the beginning of a line. Thank you Dr. Dobb! It's unfortunate for c.l.py that Python uses ">>>" as the default prompt as it messes up the display on mail/news readers that provide "syntax highlighting" for quotes.
I wish everyone would write examples that way! On the other hand - three levels of quoting are really rare, maybe it would be easier to change that in the mail readers...
>>>>>> data = [['foo','bar','baz'],['my','your'],['holy','grail']] >>>>>> result = [] >>>>>> for d in data:
>> .>>> data = [['foo','bar','baz'],['my','your'],['holy','grail']] >> .>>> from itertools import chain >> .>>> result = "".join(chain(*data)) >> 'foobarbazmyyourholygrail'
> This is the first time I see that and I totally like the idea of writing > ".>>>" instead of ">>>" at the beginning of a line. Thank you Dr. Dobb! > It's unfortunate for c.l.py that Python uses ">>>" as the default prompt > as it messes up the display on mail/news readers that provide "syntax > highlighting" for quotes.
Off topic, but indeed: I use Quote Colors in Mozilla which is very nice for reading mails or news posts with quotes, but it's very confusing with Python's prompt.
Prepending every line with . is not an ideal solution though... I think it gets tiresome very quickly.
Roel Schroeven wrote: > Stefan Behnel wrote: >> This is the first time I see that and I totally like the idea of >> writing ".>>>" instead of ">>>" at the beginning of a line. Thank you >> Dr. Dobb! It's unfortunate for c.l.py that Python uses ">>>" as the >> default prompt as it messes up the display on mail/news readers that >> provide "syntax highlighting" for quotes.
I use Thunderbird, and started doing it so I could read my own posts. I did copy it from someone, though (but I can't recall who).
The trick can also be useful for web tools that strip leading whitespace.
> Prepending every line with . is not an ideal solution though... I think > it gets tiresome very quickly.
Aye, can't argue with that. It does have the virtues of reliability and portability, though :)
Nick Coghlan wrote: > (FYI, I filed bug report #1085744 on SF about this)
And Raymond Hettinger was able to decipher my somewhat incoherent rambling (tip: don't try to write bug reports in the wee hours of the morning) and produce a potentially useful modification to PySequence_Tuple.
Anyway, I guess the results I got emphasizes the fact that it's important to measure performance of the actual methods you're using on representative input data, rather than relying on overly simple demonstrators.
And, of course, don't be *too* concerned about optimisations until you know you actually have a performance problem. . .
>>> This is the first time I see that and I totally like the idea of >>> writing ".>>>" instead of ">>>" at the beginning of a line. Thank you >>> Dr. Dobb! It's unfortunate for c.l.py that Python uses ">>>" as the >>> default prompt as it messes up the display on mail/news readers that >>> provide "syntax highlighting" for quotes.
> I use Thunderbird, and started doing it so I could read my own posts. I > did copy it from someone, though (but I can't recall who).
> The trick can also be useful for web tools that strip leading whitespace.
>> Prepending every line with . is not an ideal solution though... I >> think it gets tiresome very quickly.
> Aye, can't argue with that. It does have the virtues of reliability and > portability, though :)
> Cheers, > Nick.
$ python Python 2.4 (#1, Dec 4 2004, 20:10:33) [GCC 3.3.3 (cygwin special)] on cygwin Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.ps1 = ".>>> "; sys.ps2 = ".... " .>>> print """\ .... It isn't that hard""" It isn't that hard .>>>
Would it work, I wonder, with a leading space on the prompt? That might be a valid change to the standard prompt. Let's see
>>>> This is the first time I see that and I totally like the idea of >>>> writing ".>>>" instead of ">>>" at the beginning of a line. Thank >>>> you Dr. Dobb! It's unfortunate for c.l.py that Python uses ">>>" as >>>> the default prompt as it messes up the display on mail/news readers >>>> that provide "syntax highlighting" for quotes.
>> I use Thunderbird, and started doing it so I could read my own posts. >> I did copy it from someone, though (but I can't recall who).
>> The trick can also be useful for web tools that strip leading whitespace.
>>> Prepending every line with . is not an ideal solution though... I >>> think it gets tiresome very quickly.
>> Aye, can't argue with that. It does have the virtues of reliability >> and portability, though :)
>> Cheers, >> Nick.
> $ python > Python 2.4 (#1, Dec 4 2004, 20:10:33) > [GCC 3.3.3 (cygwin special)] on cygwin > Type "help", "copyright", "credits" or "license" for more information. > >>> import sys > >>> sys.ps1 = ".>>> "; sys.ps2 = ".... " > .>>> print """\ > .... It isn't that hard""" > It isn't that hard > .>>>
> Would it work, I wonder, with a leading space on the prompt? That might > be a valid change to the standard prompt. Let's see
> >>> This line isn't really quoted three times.
What I do is set Python's sys.ps1 variable to something else. I have a module called "interactive" that I import implicitly by shell alias:
py='python -i -c '\''import interactive'\'
Which, among other things, sets the prompt to "Python> "
433 $ py Python> print "This has no leader that screws up email programs." This has no leader that screws up email programs. Python>
-- \/ \/ (O O) -- --------------------oOOo~(_)~oOOo---------------------------------------- Keith Dart <kd...@kdart.com> public key: ID: F3D288E4 =========================================================================== =