Performance of defmulti in both ClojureScript and Clojure

[Timur]

Hi everyone,

There are situations where I want to dispatch functions using based on their certain properties. I can also use case statements instead but it looks more coupled and more change is required if I want to add new types. 

What I want to ask is if I need to avoid using multi-methods for performance reasons? I read somewhere that they are not really fast but the posts were old and the performance might have been improved in between. Should I favor case and cond branches instead of defmulti when I need performance? 

Thanks for your help!!!

Regards.

Timur

[Alex Miller]

On Saturday, April 25, 2015 at 9:33:18 AM UTC-5, Timur wrote:

Hi everyone,

There are situations where I want to dispatch functions using based on their certain properties. I can also use case statements instead but it looks more coupled and more change is required if I want to add new types. 

case is useful for the particular situation where you have a finite set of compile-time constants that you are looking for (integers, characters, etc). Then case will leverage a special JVM bytecode that performs a *constant-time* look-up (not linear-time like checking a series of cond conditions, etc). This is one of the two bytecode options (tableswitch, rather than lookupswitch) underlying the Java switch/case feature.

What I want to ask is if I need to avoid using multi-methods for performance reasons? I read somewhere that they are not really fast but the posts were old and the performance might have been improved in between. Should I favor case and cond branches instead of defmulti when I need performance? 

multimethods are slower than protocols - they must effectively invoke the dispatch function, perform a linear search for a match across the cases, and then invoke the actual implementation function. Protocols leverage JVM internals to select the right implementation, then invoke it. However, the search part of multimethods is cached, making that step fast after the initial call. The last time I benchmarked multimethods with class dispatch vs protocols on Java 1.8 + Clojure 1.7 alphas (can't remember which one), I found multimethods were about 5x slower.

If you want fastest type-based dispatch with open extension, then protocols are the best.

If you want any other type of dispatch with open extension, then multimethods are the best.

If you want a closed system of constant choices, then case is best (constant-time lookup!)

If you want a closed system of arbitrary conditions, then cond is best.

I have never compared the performance of cond vs multimethods for something like this. My guess is that multimethods are faster as they evaluate a single condition, then do a table lookup once cached vs evaluating a series of conditions every time. 

Alex 

[Alex Miller]

I should say all of that is for Clojure and likely bears no similarity to the nuances in ClojureScript.

[Ivan Reese]

Thanks for that great rundown, Alex!

I am also very interested in how the different approaches to dynamic dispatch compare in CLJS, if anyone is able to shed some light on that. Especially, whether or not case is const time — the doc string says it is, but your comments seem to suggest that it might not be the.. case.

Ivan Reese

[David Nolen]

Nearly all the performance notes about Clojure multimethods apply to ClojureScript.

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andy-]

Looks like they're pretty slow compared to a simple case:

http://jsperf.com/cljs-multimethods

https://github.com/rauhs/cljs-perf

[David Nolen]

A quick glance at your benchmarking setup, it's not clear that you are benchmarking what you think you are benchmarking, and jsperf is not a suitable benchmarking harness (irrespective of it's popularity). Benchmarking is hard, benchmarking JavaScript is harder, and benchmarking JavaScript that went through Google Closure is even more challenging than that. It's for this reason we have our own simple benchmarking suite that we run in the project repo itself against 4 JavaScript engines (V8, JavaScriptCore, SpiderMonkey, Nashorn) at the command line.

For people that want to see accurate benchmarking information posted to some public location, this is a great place to get involved contributing to ClojureScript without needing to dig into the compiler.

My earlier points about the general performance expectations of multimethods in ClojureScript still stands :)

David

--

[Andy-]

You're right. I didn't check it enough. I did check out the source (and btw, was very impressed with the result) but it looks like the function call ended up doing nothing. I just put in a side effect and I got the following results:

"[a :one], (multi-test), 1000000 runs, 2933 msecs"

"[a :one], (fn-test), 1000000 runs, 500 msecs"

Much more reasonable and I'm willing to pay for that for using multimethods.

I removed the jsperf and updated the repo. Sorry for spreading FUD.

Cheers

[Phillip Lord]

I think that the answer is, it depends, and, there might be some
surprises in store.

In my own use, I found multimethods collapse in performance as a result
of changes to the interface hierarchy in the library I was using (my
tests cases went from 45s to over 4mins).

I circumvented this by fiddling with the dispatch function, so that it
returned ONLY classes that exactly matched one of the ones in the
multi-method, and memoizing the look up of these classes. In effect,
this closes down the multi-method, so it can no longer be freely
extended. This dropped the time of my test cases dramatically.

How widely applicable these results are, I do not know, as I never got
down to the mechanistics underneath. Still, my conclusion is, you might
want to test things out first!

My full set of experiments are here...

http://www.russet.org.uk/blog/3007

[Alex Miller]

Sounds like you might have been running into the absence of multimethod caching for the default case (http://dev.clojure.org/jira/browse/CLJ-1429), which has been fixed in 1.7. 

Just on a side note, that repo does not change the default lein :jvm-opts, which are bad for benchmarking. I recommend at least:

  :jvm-opts ^:replace ["-server"]

Criterium now yells at you if you haven't done this. 

[Phillip Lord]

Alex Miller <al...@puredanger.com> writes:
> Sounds like you might have been running into the absence of multimethod
> caching for the default case (http://dev.clojure.org/jira/browse/CLJ-1429),
> which has been fixed in 1.7.

No, I don't think; my default case was and is "throw an exception".

> Just on a side note, that repo does not change the default lein :jvm-opts,
> which are bad for benchmarking. I recommend at least:
>
> :jvm-opts ^:replace ["-server"]
>
> Criterium now yells at you if you haven't done this.

I know. My use case, though, involves using Clojure through the REPL, so
I tested in that way, since this is how it runs in lein.

Phil

[Alex Miller]

I fleshed out some of this a bit more in a blog post with perf numbers in case anyone's interested:

http://insideclojure.org/2015/04/27/poly-perf/

On Saturday, April 25, 2015 at 9:39:06 PM UTC-5, Alex Miller wrote:

[Timur Sungur]

Thanks everyone for their answers and extra efforts. They are really helpful.

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---

You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/7oROqb6dGSU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.