Re: Potential benchmark: Clojure vs Ruby & Scala — Transient Newsgroups

[Alex Tkachman]

I will try to implement it

On Fri, Jan 1, 2010 at 2:24 PM, Paul King <paul.ki...@gmail.com> wrote:
>
> It might be nice to see groovypp (or even gpars) versions of the
> benchmark mentioned here:
>
> http://www.bestinclass.dk/index.php/2009/12/clojure-vs-ruby-scala-transient-newsgroups/
>

[Paul King]

I added some plain vanilla Groovy versions to the blog.
They don't try to do any efficiency optimizations at all.

Paul.

[Paul King]

12-line plain vanilla version (with a few inefficiencies to get the
small line count) which is still awaiting moderation:

t1 = System.currentTimeMillis()
counts = [:]
new File(“/path/to/20_newsgroups/”).eachFileRecurse{ f ->
if (!f.isDirectory()) {
f.text.toLowerCase().eachMatch(/\w+/) { w ->
def c = counts[w] ?: 0
counts[w] = c + 1 } } }
new File(“counts-descreasing-groovy”).withWriter { out ->
counts.sort { a, b -> b.value <=> a.value }.each { k, v -> out « “$k
\t$v\n” } }
new File(“counts-alphabetical-groovy”).withWriter { out ->
counts.sort { it.key }.each { k, v -> out « “$k\t$v\n” } }
println “Finished in ${System.currentTimeMillis() — t1} millis”

I guess it will need at least a method surrounding everything to allow
a @Typed annotation.

[Alex Tkachman]

"@Typed package 123" as 1st line should be enough

[Alex Tkachman]

I put 3 versions of benchmark in to groovypptest source control. It is
original version of Paul, the same code with required minimum of type
information and much faster but a bit longer in terms of LOC
statically typed concurrent version. Statically typed versions require
latest build 0.1.05

On Sat, Jan 2, 2010 at 6:49 AM, Paul King <paul.ki...@gmail.com> wrote:

[Paul King]

Roughly what numbers did you get?

[Paul King]

I also noticed the alphabetical sorting was reversed. Any reason?

Paul.

[Alex Tkachman]

Yes, I have bug in compiler, so I need to use version which sorts map
entries, not keys. Probably because of that order has changed :)

[Paul King]

Benchmark results for me:

12-LOC Groovy version 95+ sec
60-LOC Java 29 sec
13-LOC Static Groovy 22 sec
29-LOC Parallel Static Groovy with 2 processors 12 sec

Note: it wasn't my Java version - I just compiled up someone else's
example from the blog comments - so I haven't checked whether it is
exactly equivalent - output is more or less the same.

[Alex Tkachman]

To avoid effects of file system cache and enjoy JIT optimizations I've
wrapped the benchmark in to the loop for (i in 0..<10) here are output
of all three programs (all run -Xmx512m -server on 2 core MacBook Pro)

Normal groovy

Finished in 37685 millis
Finished in 42410 millis
Finished in 50140 millis
Finished in 41651 millis
Finished in 41429 millis
Finished in 44983 millis
Finished in 44386 millis
Finished in 48751 millis
Finished in 43538 millis
Finished in 47009 millis

Static groovy

Finished in 28613 millis
Finished in 29644 millis
Finished in 26429 millis
Finished in 26314 millis
Finished in 27365 millis
Finished in 25513 millis
Finished in 26835 millis
Finished in 30981 millis
Finished in 24683 millis
Finished in 24331 millis

Concurrent static groovy

Finished in 18506 millis
Finished in 8747 millis
Finished in 8055 millis
Finished in 9776 millis
Finished in 9705 millis
Finished in 9232 millis
Finished in 9141 millis
Finished in 8750 millis
Finished in 8470 millis
Finished in 8483 millis

[Paul King]

The Java version isn't an exact copy of Groovy version - it is using a
TreeMap to store results hence when sorting by decreasing count
occurrence, entries with the same value will be alphabetically sorted
even though that isn't a requirement. It might be faster without this
behavior.

Paul.

On Jan 4, 12:04 am, Paul King <paul.king.as...@gmail.com> wrote:
> Benchmark results for me:
>
> 12-LOC Groovy version 95+ sec
> 60-LOC Java 29 sec
> 13-LOC Static Groovy 22 sec
> 29-LOC Parallel Static Groovy with 2 processors 12 sec
>
> Note: it wasn't my Java version - I just compiled up someone else's
> example from the blog comments - so I haven't checked whether it is
> exactly equivalent - output is more or less the same.
>

> On Sun, Jan 3, 2010 at 10:58 PM, Alex Tkachman <alex.tkach...@gmail.com> wrote:
> > Yes, I have bug in compiler, so I need to use version which sorts map
> > entries, not keys. Probably because of that order has changed :)
>

> > On Sun, Jan 3, 2010 at 1:35 PM, Paul King <paul.king.as...@gmail.com> wrote:
> >> I also noticed the alphabetical sorting was reversed. Any reason?
>
> >> Paul.
>

> >> On Sun, Jan 3, 2010 at 9:34 PM, Paul King <paul.king.as...@gmail.com> wrote:
> >>> Roughly what numbers did you get?
>

> >>> On Sun, Jan 3, 2010 at 4:30 AM, Alex Tkachman <alex.tkach...@gmail.com> wrote:
> >>>> I put 3 versions of benchmark in to groovypptest source control. It is
> >>>> original version of Paul, the same code with required minimum of type
> >>>> information and much faster but a bit longer in terms of LOC
> >>>> statically typed concurrent version. Statically typed versions require
> >>>> latest build 0.1.05
>

[Alex Tkachman]

And now life became even more interested.

First of all I just commited new concurrent benchmark, which implement
a bit different algorithm for concurrency (blocking conveyor?). So far
it gives a bit better results

But most interesting thing is that I did tiny modification of previous
benchmark without concurrency and it became more than twice faster.

Paul, if you will have time to rerun benchmarks with new code it would be great.

Here are my results

Static groovy no concurrency

Finished in 14851 millis
Finished in 10770 millis
Finished in 11228 millis
Finished in 14160 millis
Finished in 12246 millis
Finished in 11537 millis
Finished in 11295 millis
Finished in 10140 millis
Finished in 10745 millis
Finished in 9710 millis

Blocking conveyor

Finished in 14005 millis
Finished in 9222 millis
Finished in 8941 millis
Finished in 8933 millis
Finished in 8727 millis
Finished in 8800 millis
Finished in 8374 millis
Finished in 8334 millis
Finished in 8216 millis
Finished in 9895 millis