Clojure performance on shootout
61 views
Skip to first unread message
Marko Kocić
Oct 26, 2010, 4:46:11 AM10/26/10
to Clojure
Clojure has recently been added to http://shootout.alioth.debian.org/
benchmarks. I know there are a lot of objections and excuses about
this benchmark, but that is defacto go to place when talking about
language implementation performance.
The problems is that performance is not that great as it could /
should be. Yes, it is a couple times faster then Ruby or Python, but
everyone's faster then those.
It still lags far behind Java, it is usually ~4 times slower then
Java, although the code seems heavily optimized thus making it even
more verbose then Java code. When running the same benchmarks with
Clojure 1.3 snapshots I didn't notice fundamental speed difference.
Scala performance is on par with Java.
Now, the questions are:
- Could benchmarks themselves be better optimized to better represent
Clojure?
- Is there a fundamental reason why Clojure can't reach Scala speed,
except for maturity?
- What are the plans regarding performance optimizations in near and
distant future?
I have seen some microbenchmarks where Clojure reaches "almost Java"
speed, but in general it is still much slower. I know that performace
is not "reason d'etre" of Clojure, but it is still important.
Regards,
Marko
benchmarks. I know there are a lot of objections and excuses about
this benchmark, but that is defacto go to place when talking about
language implementation performance.
The problems is that performance is not that great as it could /
should be. Yes, it is a couple times faster then Ruby or Python, but
everyone's faster then those.
It still lags far behind Java, it is usually ~4 times slower then
Java, although the code seems heavily optimized thus making it even
more verbose then Java code. When running the same benchmarks with
Clojure 1.3 snapshots I didn't notice fundamental speed difference.
Scala performance is on par with Java.
Now, the questions are:
- Could benchmarks themselves be better optimized to better represent
Clojure?
- Is there a fundamental reason why Clojure can't reach Scala speed,
except for maturity?
- What are the plans regarding performance optimizations in near and
distant future?
I have seen some microbenchmarks where Clojure reaches "almost Java"
speed, but in general it is still much slower. I know that performace
is not "reason d'etre" of Clojure, but it is still important.
Regards,
Marko
nicola...@gmail.com
Oct 26, 2010, 4:59:06 AM10/26/10
to clo...@googlegroups.com
Maybe optimizing now is a bit early, as a lot of the performance
caracteristics of clojure programs will
change with 1.3.
caracteristics of clojure programs will
change with 1.3.
Might be better to start writing programs that will be fast with 1.3.
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
Santosh Rajan
Oct 26, 2010, 5:27:25 AM10/26/10
to clo...@googlegroups.com
Here is another speed comparison. Of note is that there is another jvm based lisp dialect kawa.
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
B Smith-Mannschott
Oct 26, 2010, 6:06:15 AM10/26/10
to clo...@googlegroups.com
On Tue, Oct 26, 2010 at 11:27, Santosh Rajan <sant...@gmail.com> wrote:
> Here is another speed comparison. Of note is that there is another jvm based
> lisp dialect kawa.
> http://per.bothner.com/blog/2010/Kawa-in-shootout/
> Here is another speed comparison. Of note is that there is another jvm based
> lisp dialect kawa.
> http://per.bothner.com/blog/2010/Kawa-in-shootout/
Yes, this one is interesting. Might not figuring out what Kawa is
doing differently yield some ideas to improve Clojure perf?
// ben
Stuart Halloway
Oct 26, 2010, 7:17:03 AM10/26/10
to clo...@googlegroups.com
The 1.3 release for Clojure aims for parity with Java performance, with less explicit optimization and hinting than required in 1.2 (or in Java). It would be great to have people in the community share the effort of writing and testing benchmarks, and to facilitate this I have created a top-level skeleton project for a benchmark and regression suite for clojure (http://github.com/clojure/test.benchmark). Andy Fingerhut, who wrote most of the current alioth benchmarks, has signed a CA and will be contributing his code to this project.
We will primarily be targeting benchmarks for 1.3. performance, but will keep a separate 1.2 directory in the project for anyone who is interested.
As an aside: You will not be able to just run benchmarks carefully tuned for 1.2 under 1.3 and get best performance, as some of the performance tricks are now counter-indicated.
Stu
Marko Kocić
Oct 26, 2010, 8:17:20 AM10/26/10
to Clojure
It would be very good if 1.3 delivers on its goal, and it is nice to
see that performance is important to the core team.
Here's the quick test on binarytrees benchmark that I submitted to the
shootout. It is not optimized at all, besides eliminating reflection
warnings. On my box results for "binarytrees 20" are:
clojure-1.2 - ~4min
clojure-1.3-current-snapshot - ~3min
java -server - <1min.
As one can see, althoug 1.3 is nice improvement over 1.2, it's still
far from java :(
As another point, it is good that you created project for benchmarks,
it even make sense to make those benchmarks official regression test
suite, but it is also important to contribute benchmarks to the
shootout itself, since that is the place one goes to see "which
language is the fastest".
Regards,
Marko
On 26 окт, 13:17, Stuart Halloway <stuart.hallo...@gmail.com> wrote:
> The 1.3 release for Clojure aims for parity with Java performance, with less explicit optimization and hinting than required in 1.2 (or in Java). It would be great to have people in the community share the effort of writing and testing benchmarks, and to facilitate this I have created a top-level skeleton project for a benchmark and regression suite for clojure (http://github.com/clojure/test.benchmark). Andy Fingerhut, who wrote most of the current alioth benchmarks, has signed a CA and will be contributing his code to this project.
>
> We will primarily be targeting benchmarks for 1.3. performance, but will keep a separate 1.2 directory in the project for anyone who is interested.
>
> As an aside: You will not be able to just run benchmarks carefully tuned for 1.2 under 1.3 and get best performance, as some of the performance tricks are now counter-indicated.
>
> Stu
>
>
>
>
>
>
>
> > Clojure has recently been added tohttp://shootout.alioth.debian.org/
see that performance is important to the core team.
Here's the quick test on binarytrees benchmark that I submitted to the
shootout. It is not optimized at all, besides eliminating reflection
warnings. On my box results for "binarytrees 20" are:
clojure-1.2 - ~4min
clojure-1.3-current-snapshot - ~3min
java -server - <1min.
As one can see, althoug 1.3 is nice improvement over 1.2, it's still
far from java :(
As another point, it is good that you created project for benchmarks,
it even make sense to make those benchmarks official regression test
suite, but it is also important to contribute benchmarks to the
shootout itself, since that is the place one goes to see "which
language is the fastest".
Regards,
Marko
On 26 окт, 13:17, Stuart Halloway <stuart.hallo...@gmail.com> wrote:
> The 1.3 release for Clojure aims for parity with Java performance, with less explicit optimization and hinting than required in 1.2 (or in Java). It would be great to have people in the community share the effort of writing and testing benchmarks, and to facilitate this I have created a top-level skeleton project for a benchmark and regression suite for clojure (http://github.com/clojure/test.benchmark). Andy Fingerhut, who wrote most of the current alioth benchmarks, has signed a CA and will be contributing his code to this project.
>
> We will primarily be targeting benchmarks for 1.3. performance, but will keep a separate 1.2 directory in the project for anyone who is interested.
>
> As an aside: You will not be able to just run benchmarks carefully tuned for 1.2 under 1.3 and get best performance, as some of the performance tricks are now counter-indicated.
>
> Stu
>
>
>
>
>
>
>
Meikel Brandmeyer
Oct 26, 2010, 8:28:32 AM10/26/10
to Clojure
Hi,
On 26 Okt., 13:17, Stuart Halloway <stuart.hallo...@gmail.com> wrote:
> As an aside: You will not be able to just run benchmarks carefully tuned for 1.2 under 1.3 and get best performance, as some of the performance tricks are now counter-indicated.
Is there some documentation how tuning against 1.3 works? I got a list
implementation with fast random lookup written in Clojure (internally
it ended up basically like a vector type of thing) whose speed is on
par with that of a vector written in Java. I haven't tested it against
1.3 though. And optimising against 1.2 felt a little like dark
wizardry. So a short guide on how to optimise for 1.3 would be cool.
Sincerely
Meikel
On 26 Okt., 13:17, Stuart Halloway <stuart.hallo...@gmail.com> wrote:
> As an aside: You will not be able to just run benchmarks carefully tuned for 1.2 under 1.3 and get best performance, as some of the performance tricks are now counter-indicated.
implementation with fast random lookup written in Clojure (internally
it ended up basically like a vector type of thing) whose speed is on
par with that of a vector written in Java. I haven't tested it against
1.3 though. And optimising against 1.2 felt a little like dark
wizardry. So a short guide on how to optimise for 1.3 would be cool.
Sincerely
Meikel
Reply all
Reply to author
Forward
0 new messages