Re: map is useless!
James Mills
> So can anyone explain this poor excuse for a map function? Maybe GVR
> should have taken it out in 3.0? *scratches head*
Let me get this straight... You're complaining about some trivial
code you've written and a 0.002 or less execution time ?
I must be missing something!
--James
Roald de Vries
> Everyone knows i'm a Python fanboy so nobody can call me a troll for
> this...
>
> Python map is just completely useless. For one it so damn slow why
> even bother putting it in the language? And secondly, the total "girl-
> man" weakness of lambda renders it completely mute!
>
> Ruby has a very nice map
>
>>>> [1,2,3].map{|x| x.to_s}
>
> Have not done any benchmarking but far more useful from the
> programmers POV. And that really stinks because map is such a useful
> tool it's a shame to waste it. Here are some test to back up the rant.
>
>
>>>> import time
>>>> def test1():
> l = range(10000)
> t1 = time.time()
> map(lambda x:x+1, l)
> t2= time.time()
> print t2-t1
>
>
>>>> def test2():
> l = range(10000)
> t1 = time.time()
> for x in l:
> x + 1
> t2 = time.time()
> print t2-t1
>
>
>>>> test1()
> 0.00200009346008
>>>> test2()
> 0.000999927520752
>>>> def test3():
> l = range(10000)
> t1 = time.time()
> map(str, l)
> t2= time.time()
> print t2-t1
>
>
>>>> def test4():
> l = range(10000)
> t1 = time.time()
> for x in l:
> str(x)
> t2= time.time()
> print t2-t1
>
>
>>>> test3()
> 0.00300002098083
>>>> test4()
> 0.00399994850159
>>>>
>
> So can anyone explain this poor excuse for a map function? Maybe GVR
> should have taken it out in 3.0? *scratches head*
Use list comprehension. It's nicer than Ruby's map:
[x.to_s for x in 1, 2, 3]
Duncan Booth
> Python map is just completely useless. For one it so damn slow why
> even bother putting it in the language? And secondly, the total "girl-
> man" weakness of lambda renders it completely mute!
Do you realise that you don't have to use lambda? If you need more than a
single expression just create a named function.
Alain Ketterlin
> Python map is just completely useless. [...]
>>>> import time
>>>> def test1():
> l = range(10000)
> t1 = time.time()
> map(lambda x:x+1, l)
> t2= time.time()
> print t2-t1
>>>> def test2():
> l = range(10000)
> t1 = time.time()
> for x in l:
> x + 1
> t2 = time.time()
> print t2-t1
>
>>>> test1()
> 0.00200009346008
>>>> test2()
> 0.000999927520752
>>>> def test3():
Well, not building the resulting list saves some time. But even if you
do r.append(x+1) map appears to be slower...
Try this:
def test3():
l = range(10000)
t1 = time.time()
[ x+1 for x in l]
t2 = time.time()
print t2-t1
I've not used map since I learned about list comprehensions.
-- Alain.
Lie Ryan
> On Jun 6, 12:02 pm, Alain Ketterlin <al...@dpt-info.u-strasbg.fr>
> wrote:
>> rantingrick <rantingr...@gmail.com> writes:
>> I've not used map since I learned about list comprehensions.
>
> O... about 2 years now and i don't think i've ever used map in a
> script even one time until a month or so ago. After a few unit tests
> it seems i was right all along. But the funny thing is in other
> languages i use map all the time. It's just too awkward in Python and
> i wish it were not so... Oh well?
In the most naive uses, map appears to have no advantage over list
comprehension; but one thing that map can do that list comprehension
still can't do without a walk around the park:
def foo(func, args):
g = lambda x: x+1
return [func(g, x) for x in args]
foo(map, [[4, 6, 3], [6, 3, 2], [1, 3, 5]])
I'm not going to argue how often that would be useful though.
Thomas Jollans
> So can anyone explain this poor excuse for a map function? Maybe GVR
> should have taken it out in 3.0? *scratches head*
>
>
more, what once was itertools.imap is now map.
Sometimes Py2.x map is useful, sometimes list comprehension is nicer.
Sometimes Py3.x map / Py2.x itertools.imap is useful, sometimes
generator expressions are more elegant.
Same goes for filter, by the way.
Richard Thomas
what it does. I don't know much about Ruby I have to say but looking
at that piece of syntax you gave I had no idea how to interpret it.
Anyway, I looked it up.
Calling an method on each of a collection of objects is best
accomplished without map. It is semantically different to mapping a
function over a set.
As far as speed goes, Python has an overhead for making a function
call which means that its often faster to use a for loop. It seems
like a rather small difference in speed though and if what you want to
do is semantically a map you should write it is a map thereby making
your code read like what it does. If it later turns out to slow down
your program too much optimise it then.
In your second pair of tests the map is faster because str is a
builtin and doesn't have that overhead. Additionally the name 'str' is
only looked up once rather than 10000 times. :-)
Richard.
D'Arcy J.M. Cain
Lie Ryan <lie....@gmail.com> wrote:
> In the most naive uses, map appears to have no advantage over list
> comprehension; but one thing that map can do that list comprehension
> still can't do without a walk around the park:
>
> def foo(func, args):
> g = lambda x: x+1
> return [func(g, x) for x in args]
>
> foo(map, [[4, 6, 3], [6, 3, 2], [1, 3, 5]])
foo = lambda x: [y + 1 for y in x]
[foo(x) for x in [[4, 6, 3], [6, 3, 2], [1, 3, 5]]]
Didn't seem like such a long walk.
--
D'Arcy J.M. Cain <da...@druid.net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.
Lie Ryan
> On Mon, 07 Jun 2010 05:27:43 +1000
> Lie Ryan <lie....@gmail.com> wrote:
>> In the most naive uses, map appears to have no advantage over list
>> comprehension; but one thing that map can do that list comprehension
>> still can't do without a walk around the park:
>>
>> def foo(func, args):
>> g = lambda x: x+1
>> return [func(g, x) for x in args]
>>
>> foo(map, [[4, 6, 3], [6, 3, 2], [1, 3, 5]])
>
> foo = lambda x: [y + 1 for y in x]
> [foo(x) for x in [[4, 6, 3], [6, 3, 2], [1, 3, 5]]]
>
> Didn't seem like such a long walk.
>
that's because you're simplifying the problem, the correct walk is:
def foo(func, args):
g = lambda x: x+1
return [func(g, x) for x in args]
foo((lambda g, a: [g(x) for x in a]), [[4, 6, 3], [6, 3, 2], [1, 3, 5]])
Terry Reedy
> Everyone knows i'm a Python fanboy so nobody can call me a troll for
> this...
Non sequitor. It depends on your intention in posting this...
> Python map is just completely useless. For one it so damn slow
Posting invalid speed comparisons stacked against the feature you are
dissing is either trollish or lame.
> why even bother putting it in the language?
Map was put into the language about a decade before comprehensions and,
I believe, zip. It encapsulates a basic functional programming idiom.
Consider the following snippet: (in 2.x, delete 'list(' and ...')'):
from operator import add
l1 = range(10)
l2 = range(15,30)
print(list(map(add, l1, l2)))
# [15, 17, 19, 21, 23, 25, 27, 29, 31, 33]
Now replace map with a for loop, no zip or listcomp or genexp allowed.
Time how long it takes. Do you get it right the first time, as I did
with the above?. Your replacememt may or may not *run* faster, but even
if so, it will hardly be enough to make much different in most uses.
> Maybe GVR should have taken it out in 3.0?
That may have been considered, but map is shorter than the alternative,
some prefer it stylistically, it can be passed as a function argument
(including to functool.partial), and its removal would have broken code
without much gain. It is also handy for explaing generator expressions
and comprehensions.
Terry Jan Reedy
Steven D'Aprano
> Everyone knows i'm a Python fanboy so nobody can call me a troll for
> this...
The first rule of trolling is, always deny being a troll, no matter how
obvious the trolling. But on the chance I'm wrong, and for the benefit of
others, your tests don't measure what you think they are measuring and
consequently your results are invalid. Read on.
> Python map is just completely useless. For one it so damn slow why even
> bother putting it in the language? And secondly, the total "girl- man"
> weakness of lambda renders it completely mute!
Four trolls in three sentences. Way to go "fanboy".
(1) "Completely" useless? It can't do *anything*?
(2) Slow compared to what?
(3) Are you implying that map relies on lambda?
(4) What's wrong with lambda anyway?
By the way, nice sexist description there. "Girl-man weakness" indeed.
Does your mum know that you are so contemptuous about females?
> Ruby has a very nice map
I'm thrilled for them. Personally I think the syntax is horrible.
>>>> [1,2,3].map{|x| x.to_s}
>
> Have not done any benchmarking
"... but by counting under my breath while the code runs, I'm POSITIVE
Ruby is much faster that Python!"
By complaining about Python being too slow while admitting that you
haven't actually tested the speed of your preferred alternative, you have
*negative* credibility.
> but far more useful from the programmers
> POV. And that really stinks because map is such a useful tool it's a
> shame to waste it. Here are some test to back up the rant.
>
>
>>>> import time
>>>> def test1():
> l = range(10000)
> t1 = time.time()
> map(lambda x:x+1, l)
> t2= time.time()
> print t2-t1
That's a crappy test.
(1) You include the cost of building a new function each time.
(2) You make no attempt to protect against the inevitable variation in
speed caused by external processes running on a modern multi-process
operating system.
(3) You are reinventing the wheel (badly) instead of using the timeit
module.
>>>> def test2():
> l = range(10000)
> t1 = time.time()
> for x in l:
> x + 1
> t2 = time.time()
> print t2-t1
The most obvious difference is that in test1, you build a 10,000 item
list, while in test2, you don't. And sure enough, not building a list is
faster than building a list:
>>>> test1()
> 0.00200009346008
>>>> test2()
> 0.000999927520752
>>>> def test3():
> l = range(10000)
> t1 = time.time()
> map(str, l)
> t2= time.time()
> print t2-t1
>
>
>>>> def test4():
> l = range(10000)
> t1 = time.time()
> for x in l:
> str(x)
> t2= time.time()
> print t2-t1
>
>
>>>> test3()
> 0.00300002098083
>>>> test4()
> 0.00399994850159
Look ma, not building a list is still faster than building a list!
> So can anyone explain this poor excuse for a map function? Maybe GVR
> should have taken it out in 3.0? *scratches head*
So, let's do some proper tests. Using Python 2.6 on a fairly low-end
desktop, and making sure all the alternatives do the same thing:
>>> from timeit import Timer
>>> t1 = Timer('map(f, L)', 'f = lambda x: x+1; L = range(10000)')
>>> t2 = Timer('''accum = []
... for item in L:
... accum.append(f(item))
...
... ''', 'f = lambda x: x+1; L = range(10000)')
>>>
>>> min(t1.repeat(number=1000))
3.5182700157165527
>>> min(t2.repeat(number=1000))
6.702117919921875
For the benefit of those who aren't used to timeit, the timings at the
end are the best-of-three of repeating the test code 1000 times. The time
per call to map is 3.5 milliseconds compared to 6.7 ms for unrolling it
into a loop and building the list by hand. map is *much* faster.
How does it compare to a list comprehension? The list comp can avoid a
function call and do the addition inline, so it will probably be
significantly faster:
>>> t3 = Timer('[x+1 for x in L]', "L = range(10000)")
>>> min(t3.repeat(number=1000))
2.0786428451538086
And sure enough it is. But when you can't avoid the function call, the
advantage shifts back to map:
>>> t4 = Timer('map(str, L)', "L = range(10000)")
>>> t5 = Timer('[str(x) for x in L]', "L = range(10000)")
>>> min(t4.repeat(number=1000))
3.8360331058502197
>>> min(t5.repeat(number=1000))
6.6693520545959473
Lessons are:
(1) If you're going to deny being a troll, avoid making inflammatory
statements unless you can back them up.
(2) Understand what you are timing, and don't compare apples to snooker
balls just because they're both red.
(3) Timing tests are hard to get right. Use timeit.
(4) map is plenty fast.
Have a nice day.
--
Steven
D'Arcy J.M. Cain
Lie Ryan <lie....@gmail.com> wrote:
> > foo = lambda x: [y + 1 for y in x]
> > [foo(x) for x in [[4, 6, 3], [6, 3, 2], [1, 3, 5]]]
> >
> > Didn't seem like such a long walk.
> >
>
> that's because you're simplifying the problem, the correct walk is:
Well, since it gives the same answer and you didn't actually state the
problem I'm not sure how you can make that statement.
Show me the unit test that defines the problem.
Lie Ryan
> On Mon, 07 Jun 2010 05:59:02 +1000
> Lie Ryan <lie....@gmail.com> wrote:
>>> foo = lambda x: [y + 1 for y in x]
>>> [foo(x) for x in [[4, 6, 3], [6, 3, 2], [1, 3, 5]]]
>>>
>>> Didn't seem like such a long walk.
>>>
>>
>> that's because you're simplifying the problem, the correct walk is:
>
> Well, since it gives the same answer and you didn't actually state the
> problem I'm not sure how you can make that statement.
>
> Show me the unit test that defines the problem.
that you must use foo() and you can't change foo() (since foo is very
complex), and you give the same result as the original solution.
def solution(lst):
# make changes here only
return foo(map, lst)
def foo(func, args):
g = lambda x: x+1
return [func(g, x) for x in args]
import unittest
@unittest.FunctionTestCase
def test():
lst = [[4, 6, 3], [6, 3, 2], [1, 3, 5]]
ans = [[5, 7, 4], [7, 4, 3], [2, 4, 6]]
test.assertEqual(solution(lst), ans)
test.runTest()
D'Arcy J.M. Cain
Lie Ryan <lie....@gmail.com> wrote:
> On 06/07/10 09:56, D'Arcy J.M. Cain wrote:
> > Show me the unit test that defines the problem.
>
> that you must use foo() and you can't change foo() (since foo is very
> complex), and you give the same result as the original solution.
I reject the artificial restriction. If foo has a proper unit test I
can refactor it any time I want. If it doesn't then step one is to add
the missing unit tests.
In any case, the problem should be stated in terms of input and
output. That's the client requrements. Enforcing the solution is a
homework assignment, not a real requirements specification.
> def solution(lst):
> # make changes here only
> return foo(map, lst)
OK, so I can make changes here. My change would not use foo.
Carl Banks
> Everyone knows i'm a Python fanboy so nobody can call me a troll for
> this...
1. I don't remember you so I don't know if you're a Python fanboy or
not
2. If you act like a troll I'll call you one even if you are Python
fanboy
Actually, your post only came off as slightly trollish, so you have
that.
> Python map is just completely useless. For one it so damn slow why
> even bother putting it in the language? And secondly, the total "girl-
> man" weakness of lambda renders it completely mute!
>
> Ruby has a very nice map
>
> >>> [1,2,3].map{|x| x.to_s}
>
> Have not done any benchmarking but far more useful from the
> programmers POV. And that really stinks because map is such a useful
> tool it's a shame to waste it. Here are some test to back up the rant.
>
> >>> import time
> >>> def test1():
>
> l = range(10000)
> t1 = time.time()
> map(lambda x:x+1, l)
> t2= time.time()
> print t2-t1
>
> >>> def test2():
>
> l = range(10000)
> t1 = time.time()
> for x in l:
> x + 1
> t2 = time.time()
> print t2-t1
>
> >>> test1()
> 0.00200009346008
> >>> test2()
> 0.000999927520752
> >>> def test3():
>
> l = range(10000)
> t1 = time.time()
> map(str, l)
> t2= time.time()
> print t2-t1
>
> >>> def test4():
>
> l = range(10000)
> t1 = time.time()
> for x in l:
> str(x)
> t2= time.time()
> print t2-t1
>
>
>
> >>> test3()
> 0.00300002098083
> >>> test4()
> 0.00399994850159
>
> So can anyone explain this poor excuse for a map function? Maybe GVR
> should have taken it out in 3.0? *scratches head*
Since you claim to be a Python Fanboy, you probably know that you can
type "import this" at a Python prompt, and it brings up a list of
principles that guide the design of the language.
Tell me, do you see "Fast is better than slow" in that list? No?
Well that's your answer.
(The technical answer is that map isn't slow, function call overhead
is.)
Carl Banks
Terry Reedy
> On Sun, 06 Jun 2010 08:16:02 -0700, rantingrick wrote:
>
>> Everyone knows i'm a Python fanboy so nobody can call me a troll for
>> this...
>
> The first rule of trolling is, always deny being a troll, no matter how
> obvious the trolling.
Such as the exagerated-claim subject that ends with an exclamation!
> But on the chance I'm wrong, and for the benefit of
> others, your tests don't measure what you think they are measuring and
> consequently your results are invalid. Read on.
+1 on the rest. Thanks for posting it. I have nothing more to add.
Terry Jan Reedy
James Mills
<st...@remove-this-cybersource.com.au> wrote:
>> Ruby has a very nice map
>
> I'm thrilled for them. Personally I think the syntax is horrible.
I concur!
--James