Appending a list's elements to another list using a list comprehension
Debajit Adhikary
a = [1, 2, 3]
b = [4, 5, 6]
What I'd like to do is append all of the elements of b at the end of
a, so that a looks like:
a = [1, 2, 3, 4, 5, 6]
I can do this using
map(a.append, b)
How do I do this using a list comprehension?
(In general, is using a list comprehension preferable (or more
"pythonic") as opposed to using map / filter etc.?)
Marc 'BlackJack' Rintsch
> I have two lists:
>
> a = [1, 2, 3]
> b = [4, 5, 6]
>
> What I'd like to do is append all of the elements of b at the end of
> a, so that a looks like:
>
> a = [1, 2, 3, 4, 5, 6]
>
> I can do this using
>
> map(a.append, b)
This is a bad idea as it creates a useless list of `None`\s, one for each
element in `b`.
> How do I do this using a list comprehension?
Not at all. The obvious solution here is ``a.extend(b)``.
> (In general, is using a list comprehension preferable (or more
> "pythonic") as opposed to using map / filter etc.?)
Some say yes.
Ciao,
Marc 'BlackJack' Rintsch
Carsten Haese
> I have two lists:
>
> a = [1, 2, 3]
> b = [4, 5, 6]
>
> What I'd like to do is append all of the elements of b at the end of
> a, so that a looks like:
>
> a = [1, 2, 3, 4, 5, 6]
>
> I can do this using
>
> map(a.append, b)
>
> How do I do this using a list comprehension?
You don't.
> (In general, is using a list comprehension preferable (or more
> "pythonic") as opposed to using map / filter etc.?)
In general, a list comprehension is more Pythonic nowadays, but in your
particular case the answer is neither map nor a list comprehension, it's
this:
a += b
HTH,
--
Carsten Haese
http://informixdb.sourceforge.net
Paul Hankin
Yes, using a list comprehension is usually more pythonic than using
map/filter. But here, the right answer is:
a.extend(b). The first thing you should always do is check the python
libraries for a function or method that does what you want. Even if
you think you know the library quite well it's still worth checking:
I've lost count of the number of times I've discovered a library
function that does exactly what I wanted.
Anyway, if extend didn't exist, the pythonic version of map(a.append,
b) would be
for x in b:
a.append(x)
Rather than
[a.append(x) for x in b]
List comprehensions and map produce a new list. That's not what you
want here: you're using the side-effect of the append method - which
modifies a. This makes using regular iteration the right idea, because
by using map or a comprehension, you're also constructing a list of
the return values of append (which is always None). You can see this
in the interpreter:
>>> map(a.append, b)
[None, None, None]
>>> a
[1, 2, 3, 4, 5, 6]
HTH
--
Paul Hankin
Debajit Adhikary
Thanks a ton :)
What in general is a good way to learn about little things like these?
(I'm fairly new to the language)
A google search for 'python list methods" did not turn up the +
operator anywhere for me. Where could I find the official
documentation for built in structures like the list? (I just noticed
that the + operator for lists is mentioned in Beazley's Python
Essential Reference -- in the opening pages, which I didn't look at
when I was writing the earlier code.)
How does "a.extend(b)" compare with "a += b" when it comes to
performance? Does a + b create a completely new list that it assigns
back to a? If so, a.extend(b) would seem to be faster. How could I
verify things like these?
Paul Hankin
> How does "a.extend(b)" compare with "a += b" when it comes to
> performance? Does a + b create a completely new list that it assigns
> back to a? If so, a.extend(b) would seem to be faster. How could I
> verify things like these?
Use a += b rather than a.extend(b): I'm not sure what I was thinking.
Anyway, look at 'timeit' to see how to measure things like this, but
my advice would be not to worry and to write the most readable code -
and only optimise if your code's runnign too slowly.
To answer your question though: a += b is *not* the same as a = a + b.
The latter would create a new list and assign it to a, whereas a += b
updates a in-place.
--
Paul Hankin
Debajit Adhikary
> To answer your question though: a += b is *not* the same as a = a + b.
> The latter would create a new list and assign it to a, whereas a += b
> updates a in-place.
I know I'm being a little finicky here, but how would someone know
that a+=b is not the same as a=a+b?
Any documentation or reference that mentions this?
> Use a += b rather than a.extend(b): I'm not sure what I was thinking.
> Anyway, look at 'timeit' to see how to measure things like this, but
> my advice would be not to worry and to write the most readable code -
> and only optimise if your code's runnign too slowly.
I understand. Thanks :)
At the same time, however, as someone new learning the language, I
feel it always helps to know what the best practices and patterns are
at the very outset (at least for someone who chooses to become a good
programmer in that language). I mean, for me it's like this, I just
don't want to get the work done, I would really want to know why I do
something a certain way and not some other way :)
Robert Kern
> On Oct 17, 5:40 pm, Paul Hankin <paul.han...@gmail.com> wrote:
>> To answer your question though: a += b is *not* the same as a = a + b.
>> The latter would create a new list and assign it to a, whereas a += b
>> updates a in-place.
>
> I know I'm being a little finicky here, but how would someone know
> that a+=b is not the same as a=a+b?
> Any documentation or reference that mentions this?
http://docs.python.org/ref/augassign.html
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
Steven D'Aprano
> On Oct 17, 5:40 pm, Paul Hankin <paul.han...@gmail.com> wrote:
>> To answer your question though: a += b is *not* the same as a = a + b.
>> The latter would create a new list and assign it to a, whereas a += b
>> updates a in-place.
>
> I know I'm being a little finicky here, but how would someone know that
> a+=b is not the same as a=a+b?
> Any documentation or reference that mentions this?
Have you Read The Fine Manual at the Fine Website?
Try typing "Python documentation" into your favourite search engine, or
try going to http://www.python.org/ which links directly to
http://www.python.org/doc/
Unfortunately, searching for operators is always tricky (try searching
for "*" some day...) but the section you want is here:
http://docs.python.org/ref/augassign.html
Also, it isn't always true that a += b is not the same as a = a+b. It
depends on what a and b are.
--
Steven.
Steven D'Aprano
> On Oct 17, 10:03 pm, Debajit Adhikary <debaj...@gmail.com> wrote:
>> How does "a.extend(b)" compare with "a += b" when it comes to
>> performance? Does a + b create a completely new list that it assigns
>> back to a? If so, a.extend(b) would seem to be faster. How could I
>> verify things like these?
>
> Use a += b rather than a.extend(b): I'm not sure what I was thinking.
Neither am I. Why do you say that?
> Anyway, look at 'timeit' to see how to measure things like this, but my
> advice would be not to worry and to write the most readable code - and
> only optimise if your code's runnign too slowly.
Always good advice, but of course what a person considers "the most
readable code" changes with their experience.
> To answer your question though: a += b is *not* the same as a = a + b.
It might be. It depends on what a and b are.
--
Steven
Gabriel Genellina
escribió:
> What in general is a good way to learn about little things like these?
> (I'm fairly new to the language)
>
> A google search for 'python list methods" did not turn up the +
> operator anywhere for me. Where could I find the official
> documentation for built in structures like the list?
Operators are hard to find... For the available list methods, see
http://docs.python.org/lib/typesseq.html and
http://docs.python.org/lib/typesseq-mutable.html
From the Library Reference, you should read at least section 2 (Built-in
Objects) and section 3 (Built-in types), and the remaining section titles
so you know they exist at least.
--
Gabriel Genellina
Hrvoje Niksic
> On Oct 17, 10:03 pm, Debajit Adhikary <debaj...@gmail.com> wrote:
>> How does "a.extend(b)" compare with "a += b" when it comes to
>> performance? Does a + b create a completely new list that it assigns
>> back to a? If so, a.extend(b) would seem to be faster. How could I
>> verify things like these?
>
> Use a += b rather than a.extend(b): I'm not sure what I was
> thinking.
Why? a.extend(b) is at least as readable, and is guaranteed to extend
the same list. In general, "a += b" can fall back to the slower "a =
a + b" for sequences that don't support +=.
Paul Hankin
Not to me: I can never remember which of a.append and a.extend is
which. Falling back to a = a + b is exactly what you want. For
instance:
a = (1, 2, 3)
a += (4, 5, 6)
works, whereas:
a = (1, 2, 3)
a.extend((4, 5, 6))
doesn't. So using += makes your code more general. There's no standard
sequence type that has extend and not +=, so worrying that += is
slower isn't a concern unless you're using a user-defined class. Even
then, it's probably a mistake that should be fixed in the class rather
than requiring .extend() to be used instead of +=.
--
Paul Hankin
Bruno Desthuilliers
> I have two lists:
>
> a = [1, 2, 3]
> b = [4, 5, 6]
>
> What I'd like to do is append all of the elements of b at the end of
> a, so that a looks like:
>
> a = [1, 2, 3, 4, 5, 6]
>
> I can do this using
>
> map(a.append, b)
And what about a.extend(b) ?
> How do I do this using a list comprehension?
Why would you want a list comp here ???
> (In general, is using a list comprehension preferable (or more
> "pythonic") as opposed to using map / filter etc.?)
Depends. Anyway, the pythonic solution here is to use the appropriate
list method !-)
Hrvoje Niksic
> Not to me: I can never remember which of a.append and a.extend is
> which.
Interesting, with me it's the other way around. Maybe it's because I
used Python before extend was available.
> Falling back to a = a + b is exactly what you want.
Not if you want to mutate the original object, possibly referenced
from elsewhere.
Alex Martelli
...
> How does "a.extend(b)" compare with "a += b" when it comes to
> performance? Does a + b create a completely new list that it assigns
> back to a? If so, a.extend(b) would seem to be faster. How could I
> verify things like these?
That's what the timeit module is for, but make sure that the snippet
you're timing has no side effects (since it's repeatedly executed).
E.g.:
brain:~ alex$ python -mtimeit -s'z=[1,2,3];b=[4,5,6]'
'a=z[:];a.extend(b)'
1000000 loops, best of 3: 0.769 usec per loop
brain:~ alex$ python -mtimeit -s'z=[1,2,3];b=[4,5,6]' 'a=z[:];a+=b'
1000000 loops, best of 3: 0.664 usec per loop
brain:~ alex$ python -mtimeit -s'z=[1,2,3];b=[4,5,6]'
'a=z[:];a.extend(b)'
1000000 loops, best of 3: 0.769 usec per loop
brain:~ alex$ python -mtimeit -s'z=[1,2,3];b=[4,5,6]' 'a=z[:];a+=b'
1000000 loops, best of 3: 0.665 usec per loop
brain:~ alex$
The repetition of the measurements show them very steady, so now you
know that += is about 100 nanoseconds faster (on my laptop) than extend
(the reason is: it saves the tiny cost of looking up 'extend' on a; to
verify this, use much longer lists and you'll notice that while overall
times for both approaches increase, the difference between the two
approaches remains about the same for lists of any length).
But the key point to retain is: make sure that the snippet is free of
side effects, so that each of the MANY repetitions that timeit does is
repeating the SAME operation. If we initialized a in the -s and then
just extended it in the snippet, we'd be extending a list that keeps
growing at each repetition -- a very different operation than extending
a list of a certain fixed starting length (here, serendipitously, we'd
end up measuring the same difference -- but in the general case, where
timing difference between approaches DOES depend on the sizes of the
objects involved, our measurements would instead become meaningless).
Therefore, we initialize in -s an auxiliary list, and copy it in the
snippet. That's better than the more natural alternative:
brain:~ alex$ python -mtimeit 'a=[1,2,3];a+=[4,5,6]'
1000000 loops, best of 3: 1.01 usec per loop
brain:~ alex$ python -mtimeit 'a=[1,2,3];a.extend([4,5,6])'
1000000 loops, best of 3: 1.12 usec per loop
brain:~ alex$ python -mtimeit 'a=[1,2,3];a+=[4,5,6]'
1000000 loops, best of 3: 1.02 usec per loop
brain:~ alex$ python -mtimeit 'a=[1,2,3];a.extend([4,5,6])'
1000000 loops, best of 3: 1.12 usec per loop
as in this "more natural alternative" we're also paying each time
through the snippet the cost of building the literal lists; this
overhead (which is a lot larger than the difference we're trying to
measure!) does not DISTORT the measurement but it sure OBSCURES it to
some extend (losing us about one significant digit worth of difference
in this case). Remember, the WORST simple operation you can do in
measurement is gauging a small number delta as the difference of two
much larger numbers X and X+delta... so, make X as small as feasible to
reduce the resulting loss of precision!-)
You can find more details on commandline use of timeit at
<http://docs.python.org/lib/node808.html> (see adjacent nodes in Python
docs for examples and details on the more advanced use of timeit inside
your own code) but I hope these indications may be of help anyway.
Alex
J. Clifford Dyer
>
> Not to me: I can never remember which of a.append and a.extend is
> which. Falling back to a = a + b is exactly what you want. For
> instance:
>
> a = (1, 2, 3)
> a += (4, 5, 6)
>
> works, whereas:
>
> a = (1, 2, 3)
> a.extend((4, 5, 6))
>
> doesn't. So using += makes your code more general. There's no standard
> sequence type that has extend and not +=, so worrying that += is
> slower isn't a concern unless you're using a user-defined class. Even
> then, it's probably a mistake that should be fixed in the class rather
> than requiring .extend() to be used instead of +=.
>
I was going to argue that in fact += is not more general, it just covers a different set of use cases, but then I tested my hypothesis...
>>> a = [1,2,3]
>>> b = a
>>> c = [4,5,6]
>>> d = c
>>> e = [7,8,9]
>>> a.extend(e)
>>> b
[1, 2, 3, 7, 8, 9]
>>> c += e
>>> d # I expected [4, 5, 6]
[4, 5, 6, 7, 8, 9]
>>> c = c + e # But += doesn't do the same as this
>>> c
[4, 5, 6, 7, 8, 9, 7, 8, 9]
>>> d
[4, 5, 6, 7, 8, 9]
So I learned something new.
Cheers,
Cliff
Debajit Adhikary
> You can find more details on commandline use of timeit at
> <http://docs.python.org/lib/node808.html> (see adjacent nodes in Python
> docs for examples and details on the more advanced use of timeit inside
> your own code) but I hope these indications may be of help anyway.
Thanks for the wonderful explanation on timeit.
Thats one more tool now in my arsenal :P