The Clash of Lambdas Benchmark

[Alexandru Nedelcu]

Hi,

There's an interesting benchmarks circulating lately:

http://cgi.di.uoa.gr/~biboudis/clashofthelambdas.pdf

The benchmark is sort of comparing libraries, rather than languages, but I think 2 things are sticking out of it in regards to the apparent poor performance of Scala:

1. collections really need specialization, as working with primitives is really common and the price paid for boxing primitives is a performance killer 

2. Scala's Views are really terrible - which is odd for a standard library of an FP language exposing immutable collections as the default 

I was wondering if there are plans to address the above.

--
Alexandru Nedelcu
www.bionicspirit.com

PGP Public Key:
https://bionicspirit.com/key.aexpk

[Johannes Rudolph]

On Thu, Jun 26, 2014 at 2:22 PM, Alexandru Nedelcu
<al...@bionicspirit.com> wrote:
> There's an interesting benchmarks circulating lately:
> http://cgi.di.uoa.gr/~biboudis/clashofthelambdas.pdf
>
> The benchmark is sort of comparing libraries, rather than languages, but I
> think 2 things are sticking out of it in regards to the apparent poor
> performance of Scala:

Interesting. Thanks for sharing.

--
Johannes

-----------------------------------------------
Johannes Rudolph
http://virtual-void.net

[Jason Zaugg]

On Thu, Jun 26, 2014 at 2:22 PM, Alexandru Nedelcu <al...@bionicspirit.com> wrote:

Hi,

There's an interesting benchmarks circulating lately:

http://cgi.di.uoa.gr/~biboudis/clashofthelambdas.pdf

The benchmark is sort of comparing libraries, rather than languages, but I think 2 things are sticking out of it in regards to the apparent poor performance of Scala:

1. collections really need specialization, as working with primitives is really common and the price paid for boxing primitives is a performance killer 

2. Scala's Views are really terrible - which is odd for a standard library of an FP language exposing immutable collections as the default 

I was wondering if there are plans to address the above.

I found it a bit of a pity that that benchmark only seems to include Double => Double transformation pipelines. A reader could wrongly draw the conclusion that Java lambdas are faster in general, which isn’t the case.

Under Scala 2.11 -Xexperimental, you can actually use the Java 8 streams API directly using Scala lambda syntax, so x => x * x can be treated as a java.util.function.DoubleFunction, rather than Function1<Int, Int>. You can then profit from the selective, hand specialization of the Java 8 streams API.

​

-jason

[Lanny Ripple]

Interesting as a snapshot in time, yes.  I would have loved to see in Future Work, "Since Scala is based on the JVM but has it's own development cycle we will revisit this benchmark in X timeframe".  I don't believe their conclusion in the discussion portion of the performance evaluation is supported by their work either --

  Our microbenchmarks paint a fairly clear picture of

  the current status of lambdas+streaming implementations, as well

  as their future prospects.  (emphasis added)

That said I drink the cool-aid around here so I know my thoughts on the subject are clouded.

[Alexandru Nedelcu]

That’s nice. Personally I also don’t like hand specialization, because once in a standard library it’s permanent and it doesn’t cover everything. Java’s collections are still not specialized either, you still have to use a third-party library like Trove.

I would prefer a generic approach.Scala has @specialized but it has many problems. I played with miniboxing and seems promising. Are there any plans for it to replace @specialized and to use it against Scala’s collections?

​

[Aggelos Biboudis]

Hi all!

I am Aggelos, one of the authors of that workshop paper. The most up-to-date version of our paper is (and will be) at http://arxiv.org/abs/1406.6631v1

We have also included benchmarks that exercise strict operations (without .view) and we are still working on adding an additional test, one with reference types and no automatic boxing.

Cheers,
Aggelos Biboudis.

[Jason Zaugg]

On Thu, Jun 26, 2014 at 4:50 PM, Alexandru Nedelcu <al...@bionicspirit.com> wrote:

That’s nice. Personally I also don’t like hand specialization, because once in a standard library it’s permanent and it doesn’t cover everything. Java’s collections are still not specialized either, you still have to use a third-party library like Trove.

I would prefer a generic approach.Scala has @specialized but it has many problems. I played with miniboxing and seems promising. Are there any plans for it to replace @specialized and to use it against Scala’s collections?

I believe that Vlad has a version of Miniboxing working against some collections and array wrappers. I didn't catch his talk at ScalaDays, but am waiting keenly to watch the recording.

It is still too early to say if/when we could switch to this approach. I'd suggest that 2.13 would be the first opportunity, as we've got a full roadmap for 2.12.

-jason

[√iktor Ҡlang]

Check this, Jason: scala-miniboxing.org/example_linkedlist.html

--
You received this message because you are subscribed to the Google Groups "scala-debate" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-debate...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Cheers,

√

[Daniel Armak]

If I may ask, what is the roadmap for 2.12? Is it summarized anywhere online? There are only 4 tickets on github marked with a 2.12 milestone, so I assume that isn't it.

It was probably discussed on this or another scala mailing list before I subscribed...

Daniel Armak

[Jason Zaugg]

On Thu, Jun 26, 2014 at 6:30 PM, Daniel Armak <dana...@gmail.com> wrote:

If I may ask, what is the roadmap for 2.12? Is it summarized anywhere online? There are only 4 tickets on github marked with a 2.12 milestone, so I assume that isn't it.

It was probably discussed on this or another scala mailing list before I subscribed...

We'll send out a detailed roadmap in the next week or two. The big ticket items are Java 8 support, switching over to the new bytecode generator and optimizer ("GenBCode"), and improving the compiler as a platform for type-aware lint tools. Stay tuned for more details!

-jason

[Vlad Ureche]

2014-06-26 17:44 GMT+02:00 Jason Zaugg <jza...@gmail.com>:

It is still too early to say if/when we could switch to this approach. I'd suggest that 2.13 would be the first opportunity, as we've got a full roadmap for 2.12.

IMO the first step is to build up trust in the miniboxing plugin. And the only way to do that is for people to use it in their projects, file bugs if they occur and report on the overall experience.

I've put together a roadmap from my perspective: http://scala-miniboxing.org/2014/06/27/miniboxing-roadmap.html

Cheers,

Vlad