map versus pmap
philip schwarz
be gentle please: I have only just started using clojure.
I run the following on an Intel Core 2 Duo CPU (starting clojure with
"java -Xms1000m -Xmx1000m -jar clojure.jar"):
user=> (time (nth (doall (map inc (range 10000000))) 9999999))
"Elapsed time: 5944.068355 msecs"
10000000
user=> (time (nth (doall (pmap inc (range 10000000))) 9999999))
"Elapsed time: 108837.451146 msecs"
10000000
I was expecting pmap to take less time then map, instead it took much
much longer.
Any thoughts?
Andy Fingerhut
Note that total CPU time for pmap will always have some extra overhead above the corresponding map - it is recommended to use it when that extra overhead is small compared to the total computation time required.
Andy
Sent from my iPhone
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
philip schwarz
On Nov 6, 6:04 pm, Andy Fingerhut <andy.finger...@gmail.com> wrote:
> The amount of work in each element of the pmap is so small that the overhead per element is dominating the CPU usage, e.g. overhead like constructing a future for each element of the collection you are mapping over. Try a test case where the computation being done in the function you supply to map is a lot more than inc.
>
> Note that total CPU time for pmap will always have some extra overhead above the corresponding map - it is recommended to use it when that extra overhead is small compared to the total computation time required.
>
> Andy
>
> Sent from my iPhone
>
nickikt
What would you do if you have to count to 10000.
Would you just start counting or would go and hire 2 more people
explain them what to do and then count together?
In this case you would just start counting because you know that you
will be faster but if you have to do 10000 difficult calculations you
would probebly hire somebody because the work to hire and cooradinate
the people is lower then the amount of work the could do.
André Thieme
Try
(time (count (map (fn [_] (Thread/sleep 1)) (range 1000))))
vs.
(time (count (pmap (fn [_] (Thread/sleep 1)) (range 1000))))
On my system this is 1002 msecs vs. 39 msecs.
Ken Wesson
Your system has TWENTY-FIVE CORES?!
André Thieme
> On Sat, Nov 6, 2010 at 11:25 PM, Andr� Thieme
>> Try
>> (time (count (map (fn [_] (Thread/sleep 1)) (range 1000))))
>> vs.
>> (time (count (pmap (fn [_] (Thread/sleep 1)) (range 1000))))
>>
>> On my system this is 1002 msecs vs. 39 msecs.
>
> Your system has TWENTY-FIVE CORES?!
I wished :-)
I have a dual core CPU.
user> (time (count (pmap (fn [_] (Thread/sleep 3000)) (range 10))))
"Elapsed time: 3001.318114 msecs"
10
user> (time (count (pmap (fn [_] (Thread/sleep 3000)) (range 20))))
"Elapsed time: 3004.243067 msecs"
20
user> (time (count (pmap (fn [_] (Thread/sleep 3000)) (range 30))))
"Elapsed time: 3005.278395 msecs"
30
user> (time (count (pmap (fn [_] (Thread/sleep 3000)) (range 100))))
"Elapsed time: 12002.598445 msecs"
100
I guess this has to do with Thread/sleep suspending the threads back
into the pool, so within just a few msecs. tons of threads were started
only to be suspended, and their wakeup timer fired +/- the same time.
That is my current interpretation, but I would like to hear others.
(Windows Vista, 32 Bit, Java 6 Update 22)
philip schwarz
Clojure:
(defn make-heavy [f]
(fn [& args]
(Thread/sleep 1000)
(apply f args)))
user=> (time (+ 5 5))
"Elapsed time: 0.06403 msecs"
10
user=> (time ((make-heavy +) 5 5))
"Elapsed time: 1000.622706 msecs"
10
user=> (time (doall (map (make-heavy inc) [1 2 3 4 5])))
"Elapsed time: 5003.314909 msecs"
(2 3 4 5 6)
user=> (time (doall (pmap (make-heavy inc) [1 2 3 4 5])))
"Elapsed time: 1000.720951 msecs"
(2 3 4 5 6)
in this case, pmap is faster.