benchmarks? java vs .net
King Raz
includes C# Mono vs Java but not C# .NET vs Java. So I went through
all the benchmark on the site ...
http://kingrazi.blogspot.com/2008/05/shootout-c-net-vs-java-benchmarks.html
Just to keep the post on topic for my friends at comp.lang.c++, how do
I play default windows sounds with C++?
Andrea Francia
This guy had trolled the comp.lang.java.programmer and comp.lang.c++
newsgroup for many weeks with pretentious post about the performance
comparison of C++ and Java.
The comparison was done in a not scientific way with only purpose of get
replies.
To get replies he/she send many replies to each message he/she receives.
For reach is target he/she changed his/hers email address many times to
avoid killfiles.
I suggest the people not interested who have thunderbird to install the
"Right click ingore/watch thread" extension and use it for ignoring this
thread.
--
Andrea Francia
http://andreafrancia.blogspot.com/
Razii
<andrea....@REMOVE-FROM-HERE.ohoihihoihoih.TO-HERE.gmx.it> wrote:
>This guy had trolled the comp.lang.java.programmer and comp.lang.c++
>newsgroup for many weeks with pretentious post about the performance
>comparison of C++ and Java.
This got to be the funniest post. She (or is that he?) cross-posts
(spams?) three newsgroups with the subject "Please don't feed the
troll" and this post was the only response to my thread. Thanks for
feeding me. She/He also quotes my entire post so anyone who missed my
first post now read it :) Now I know at least one person read my
post; that is the brilliant: Andrea Francia, for Italy :)
Were you born this stupid or did someone hit you with a baseball bal?
Razii
>This got to be the funniest post. She (or is that he?) cross-posts
>(spams?) three newsgroups with the subject "Please don't feed the
>troll" and this post was the only response to my thread. Thanks for
>feeding me. She/He also quotes my entire post so anyone who missed my
>first post now read it :) Now I know at least one person read my
>post; that is the brilliant: Andrea Francia, for Italy :)
Arrrgh. I meant "from Italy"
In any case, if you think I am a troll, why would you feed me by
responding to my post? Why would you quote my entire post, advertising
it for free? People who missed the first post now must have read my
post, thanks to Andrea Francia.
Well, it seems, the programming-related newsgroups have some of
the dumbest people on USENET. I wonder why?
Jeff Higgins
Andrea Francia wrote:
> I suggest...
!Thanks for the respam. I'd have never been aware of this
post had it not been for your announcement and sage advice.
I'm sure k.rzi thanks you for reposting a link to his site.
Jon Harrop
Yes. I have ported a variety of benchmarks and found there is no significant
difference between the performance of .NET and Java. The one result that
stood out was .NET being 2x faster at the Mersenne Twister PRNG.
--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com/products/?u
Razii
wrote:
>Yes. I have ported a variety of benchmarks and found there is no significant
>difference between the performance of .NET and Java. The one result that
>stood out was .NET being 2x faster at the Mersenne Twister PRNG.
.NET is twice slower in four benchmarks: binarytrees, mandelbrot,
regexdna, sumcol. .NET was twice faster only in trig related
benchmark.
Razii
wrote:
>Yes. I have ported a variety of benchmarks and found there is no significant
>difference between the performance of .NET and Java. The one result that
>stood out was .NET being 2x faster at the Mersenne Twister PRNG.
.NET is twice slower in four benchmarks: binarytrees, mandelbrot,
Jon Harrop
You have not optimized the .NET code:
On the mandelbrot benchmark, most of the time is spent doing unbuffered IO.
Use buffered IO and .NET becomes ~10% faster than Java.
On the regexdna benchmark, the regular expressions are not being compiled.
Ask for compilation and the .NET is <20% slower than Java.
The sumcol benchmark is also using unbuffered IO.
So neither VM is significantly faster overall.
Razii
wrote:
>On the mandelbrot benchmark, most of the time is spent doing unbuffered IO.
>Use buffered IO and .NET becomes ~10% faster than Java.
The output was directed to /NULL so I don't see how buffering makes
the difference. In any case, post the version that makes it 10%
faster. Right now it's 2 times slower.
>On the regexdna benchmark, the regular expressions are not being compiled.
>Ask for compilation and the .NET is <20% slower than Java.
The version that compiles regex is slower that then the version that
doesn't compile on Mono
http://shootout.alioth.debian.org/gp4/benchmark.php?test=regexdna&lang=all
That's because the regular expression is used only once and compiling
it makes it slower. I doubt the compiled version will be faster on
.NET. It will be slower.
>The sumcol benchmark is also using unbuffered IO.
Post the right version that buffers then.
>So neither VM is significantly faster overall.
.NET is 2 times slower in 4 of the benchmarks. Post the version that
makes it faster so we can verify. You haven't done it yet.
Razii
>The version that compiles regex is slower that then the version that
>doesn't compile on Mono
>
>http://shootout.alioth.debian.org/gp4/benchmark.php?test=regexdna&lang=all
Ok, I did try compiled version on .NET and it was faster than
uncompiled version.
9.539s (uncompiled)
5.165s (compiled)
the java version was 3.956s
Razii
wrote:
>The sumcol benchmark is also using unbuffered IO.
I changed the line to
using (StreamReader r = new
StreamReader(Console.OpenStandardInput(8192)))
Buffereing 8192 bytes now?
It made no difference.
4.861s (unbuffered)
4.839s (buffered)
still two times slower.
Razii
wrote:
>On the mandelbrot benchmark, most of the time is spent doing unbuffered IO.
>Use buffered IO and .NET becomes ~10% faster than Java.
Doesn't this line mean
http://shootout.alioth.debian.org/gp4/benchmark.php?test=mandelbrot&lang=csharp&id=2
Stream s = Console.OpenStandardOutput(1024);
it's buffering already, 1024 bytes?
vs this java version
http://shootout.alioth.debian.org/gp4/benchmark.php?test=mandelbrot&lang=javaxx&id=0
it's 3.4 times slower.
Mark Thornton
Rather amusing really as unintentional use of unbuffered IO is a
frequent cause of Java benchmarks running more slowly than they should.
It seems that .NET copied that characteristic as well.
Mark Thornton
Jon Harrop
> Rather amusing really as unintentional use of unbuffered IO is a
> frequent cause of Java benchmarks running more slowly than they should.
> It seems that .NET copied that characteristic as well.
Yes. I've no idea why they do that. Isn't buffered IO a better default?!
Razii
<mtho...@optrak.co.uk> wrote:
>Rather amusing really as unintentional use of unbuffered IO is a
>frequent cause of Java benchmarks running more slowly than they should.
>It seems that .NET copied that characteristic as well.
Harpo was wrong though. mandelbrot is buffered
Stream s = Console.OpenStandardOutput(1024);
It's still 3.4 times slower.
Verify it yourself.
http://shootout.alioth.debian.org/gp4/benchmark.php?test=mandelbrot&lang=csharp&id=2
vs
http://shootout.alioth.debian.org/gp4/benchmark.php?test=mandelbrot&lang=javaxx&id=0
3.4 times slower.
Jon Harrop
> On Tue, 03 Jun 2008 01:23:00 +0100, Jon Harrop <j...@ffconsultancy.com>
> wrote:
>>On the mandelbrot benchmark, most of the time is spent doing unbuffered
>>IO. Use buffered IO and .NET becomes ~10% faster than Java.
>
> The output was directed to /NULL so I don't see how buffering makes
> the difference. In any case, post the version that makes it 10%
> faster. Right now it's 2 times slower.
Change:
Stream s = Console.OpenStandardOutput(1024);
to:
BufferedStream s = new BufferedStream(Console.OpenStandardOutput());
and the entire program becomes 2.5x faster.
>>On the regexdna benchmark, the regular expressions are not being compiled.
>>Ask for compilation and the .NET is <20% slower than Java.
>
> Ok, I did try compiled version on .NET and it was faster than
> uncompiled version.
>
> 9.539s (uncompiled)
> 5.165s (compiled)
>
> the java version was 3.956s
Yes.
>>So neither VM is significantly faster overall.
>
> .NET is 2 times slower in 4 of the benchmarks.
Java is only 2x faster in one benchmark (binarytrees) here and that is not a
well formed benchmark. Java is still over 2x slower on both partialsums and
the Mersenne Twister.
So you cannot reasonably conclude that Java is faster.
Jon Harrop
> On Tue, 03 Jun 2008 07:55:07 +0100, Mark Thornton
> <mtho...@optrak.co.uk> wrote:
>>Rather amusing really as unintentional use of unbuffered IO is a
>>frequent cause of Java benchmarks running more slowly than they should.
>>It seems that .NET copied that characteristic as well.
>
> Harpo was wrong though. mandelbrot is buffered
>
> Stream s = Console.OpenStandardOutput(1024);
Use:
BufferedStream s = new BufferedStream(Console.OpenStandardOutput());
> It's still 3.4 times slower.
Not if you optimize it properly.
> Verify it yourself.
I did.
There is no merit in comparing unoptimized code on .NET with optimized Java.
Java is also not significant faster on any of the SciMark2 benchmarks. The
only significant difference I have seen is on the Mersenne Twister PRNG
where Java is 2x slower than .NET.
John B. Matthews
Razii <klgfsdf...@mail.com> wrote:
[...]
> Verify it yourself.
>
> http://shootout.alioth.debian.org/gp4/benchmark.php?test=mandelbrot&lang=cshar
> p&id=2
>
> vs
>
> http://shootout.alioth.debian.org/gp4/benchmark.php?test=mandelbrot&lang=javax
> x&id=0
>
>
> 3.4 times slower.
Indeed, I see C#/Mono = 7.15s and Java = 3.22s. I would say that Java
took (3.22s / 7.15s = 0.450) less that half as long as C#/Mono or that
Java was (7.15s / 3.22s = 2.220) over twice as fast. I don't see where
3.4 comes from. Just asking.
John
--
John B. Matthews
trashgod at gmail dot com
home dot woh dot rr dot com slash jbmatthews
Jon Harrop
> Indeed, I see C#/Mono = 7.15s and Java = 3.22s. I would say that Java
> took (3.22s / 7.15s = 0.450) less that half as long as C#/Mono or that
> Java was (7.15s / 3.22s = 2.220) over twice as fast. I don't see where
> 3.4 comes from. Just asking.
FWIW, F#/Mono is 3x slower than F#/.NET on the SciMark2 benchmark. I'd like
to know how performance compares between these platforms for parallel code.
Does Java have anything comparable to .NET's Task Parallel Library?
![Jon Skeet [C# MVP]'s profile photo](http://lh3.googleusercontent.com/a-/AD5-WCn7LjXGCZNQ6qJtv2iyb10EueHnmmvRjT7twL5uLwSR-dM=s40-c)
Jon Skeet [C# MVP]
> FWIW, F#/Mono is 3x slower than F#/.NET on the SciMark2 benchmark. I'd like
> to know how performance compares between these platforms for parallel code.
If Razii is genuinely comparing Java with Mono (I haven't looked at
any of the figures) it's a silly test to start with (unless you're
specifically interested in Mono, of course). The vast majority of C#
code runs on .NET rather than Mono - and while I applaud the Mono
team's work, I seriously doubt that it has quite has much effort going
into it as Microsoft is putting into .NET. I'd expect .NET to
outperform Mono, and on microbencharks like these the difference could
be quite significant in some cases.
> Does Java have anything comparable to .NET's Task Parallel Library?
I haven't used it, but I've heard about Doug Lea's Fork/Join
framework:
http://gee.cs.oswego.edu/dl/papers/fj.pdf
One problem with creating anything like Parallel Extensions (which I
assume is what you meant - AFAIK the TPL is just part of Parallel
Extensions, although I could be misunderstanding you completely) for
Java is that it doesn't (currently) have closures other than anonymous
inner classes which are ugly as hell.
The state of closures in Java 7 is currently up in the air.
Jon
Razii
wrote:
> Java is still over 2x slower on both partialsums and
>the Mersenne Twister.
The partialsums is only slower due to trig accuracy required by Java's
specification. Your C# results are not accurate enough to satisfy
Java's specification. It's easy to solve this problem. Try this Java
version of partialsums. It's two times faster.
http://shootout.alioth.debian.org/gp4/benchmark.php?test=partialsums&lang=java&id=4
In binarytrees C# is twice slower. You haven't posted any workaround
yet.
Razii
<nos...@nospam.com> wrote:
>I don't see where
>3.4 comes from. Just asking.
Did you use -server flag? It's 7.578 vs 2.198 on my computer.
7.578 / 2.198 = 3.44
Razii
wrote:
> BufferedStream s = new BufferedStream(Console.OpenStandardOutput());
>
>and the entire program becomes 2.5x faster.
Yes, that was faster. I submitted it to shootout site. I am not sure
if it will be faster on Mono too but I will let Gouy test it.
Razii
wrote:
>There is no merit in comparing unoptimized code on .NET with optimized Java.
That's why posted it here so you can "optimize it". You fixed
mandelbrot but C# is still twice slower in three other benchmarks:
binarytrees
(the command line argument should be -server -Xms64m binarytrees)
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=binarytrees&lang=javaxx&id=2
vs
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=binarytrees&lang=csharp&id=0
revcomp
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=revcomp&lang=javaxx&id=4
vs
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=revcomp&lang=csharp&id=2
sumcol
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=sumcol&lang=javaxx&id=4
vs
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=sumcol&lang=csharp&id=0
also, C# significantly slower in recursion
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=recursive&lang=javaxx&id=0
vs
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=recursive&lang=csharp&id=0
>I have seen is on the Mersenne Twister PRNG where Java is 2x slower than .NET.
Post the benchmark. Let me see...
Razii
<sk...@pobox.com> wrote:
>If Razii is genuinely comparing Java with Mono (I haven't looked at
>any of the figures) it's a silly test to start with (unless you're
>specifically interested in Mono, of course).
If you have no clue what is this thread about, why post at all?
Jon Skeet [C# MVP]
> >If Razii is genuinely comparing Java with Mono (I haven't looked at
> >any of the figures) it's a silly test to start with (unless you're
> >specifically interested in Mono, of course).
>
> If you have no clue what is this thread about, why post at all?
Well gee, you keep post URLs comparing Mono with Java. That's *not*
comparing .NET with Java - and is thus relatively pointless IMO. (It's
also made more pointless by you not mentioning in your blog post which
version of Java you're using, or which version of .NET.)
To make the conversation significant (well, as significant as this kind
of thing can be):
1) Don't bother posting about the Mono benchmarks at all. Keep it to
.NET and Java.
2) Explain the precise runtime environments.
The IO suggestions so far seem to be an indication of the general merit
of the benchmark, mind you.
--
Jon Skeet - <sk...@pobox.com>
Web site: http://www.pobox.com/~skeet
Blog: http://www.msmvps.com/jon.skeet
C# in Depth: http://csharpindepth.com
Razii
<sk...@pobox.com> wrote:
>Well gee, you keep post URLs comparing Mono with Java. That's *not*
>comparing .NET with Java - and is thus relatively pointless IMO. (It's
>also made more pointless by you not mentioning in your blog post which
>version of Java you're using, or which version of .NET.)
The link I posted was this:
It's clear what it is about.
Jon Harrop
It makes no difference on Mono and the shootout does not test .NET.
Razii
<sk...@pobox.com> wrote:
>The IO suggestions so far seem to be an indication of the general merit
>of the benchmark, mind you.
The criteria is same for both sides. For now C# is slower by wide
margin in these benchmarks. How about fixing them? If you can't, I
will conclude C# is just slower.
Mark Thornton
> Mark Thornton wrote:
>> Rather amusing really as unintentional use of unbuffered IO is a
>> frequent cause of Java benchmarks running more slowly than they should.
>> It seems that .NET copied that characteristic as well.
>
> Yes. I've no idea why they do that. Isn't buffered IO a better default?!
>
I think the reasoning is simplicity --- you create what you want through
composition of a smaller number of classes. Otherwise you need more
classes or extra options on constructors to provide for all possible
cases. Unfortunately they spoiled this a bit by providing a few
composites (java.io.FileReader for example). It is also a bit confusing
that InputStreamReader includes some buffering, but adding buffering to
an already buffered stream doesn't usually increase the cost by much.
Mark Thornton
Jon Harrop
> The criteria is same for both sides. For now C# is slower by wide
> margin in these benchmarks. How about fixing them?
I did.
> If you can't, I will conclude C# is just slower.
You had drawn that conclusion before you even tried to measure anything.
Mark Thornton
>
>> Does Java have anything comparable to .NET's Task Parallel Library?
>
> I haven't used it, but I've heard about Doug Lea's Fork/Join
> framework:
> http://gee.cs.oswego.edu/dl/papers/fj.pdf
>
haven't used .NET so can't compare them.
For more information and downloads of the package:
http://g.oswego.edu/dl/concurrency-interest/
Mark Thornton
Jon Skeet [C# MVP]
That was the link you posted *initially*. Since then you have posted
multiple links to http://shootout.alioth.debian.org which is about
Mono.
Jon Harrop
> Jon Skeet [C# MVP] wrote:
>>> Does Java have anything comparable to .NET's Task Parallel Library?
>>
>> I haven't used it, but I've heard about Doug Lea's Fork/Join
>> framework:
>> http://gee.cs.oswego.edu/dl/papers/fj.pdf
>>
> I have used it and find it works quite well even without closures. I
> haven't used .NET so can't compare them.
The Task Parallel Library is awesome, even though it is only a CTP for now.
I highly recommend it. For example, read the article:
"Surviving in the multicore era with Microsoft's ParallelFX" - The F#.NET
Journal, 30th April 2008
http://www.ffconsultancy.com/products/fsharp_journal/?cs
Jon Harrop
> On Tue, 03 Jun 2008 16:08:17 +0100, Jon Harrop <j...@ffconsultancy.com>
> wrote:
>> Java is still over 2x slower on both partialsums and
>>the Mersenne Twister.
>
> The partialsums is only slower due to trig accuracy required by Java's
> specification. Your C# results are not accurate enough to satisfy
> Java's specification. It's easy to solve this problem. Try this Java
> version of partialsums. It's two times faster.
But still 2x slower than everything else and now 2x more bloated as well:
C 1.300s
C# 1.363s
Java 2.750s
Java 3.840s (original)
> In binarytrees C# is twice slower. You haven't posted any workaround
> yet.
Optimizing binarytrees is trivial because the benchmark is fundamentally
flawed. I described this in detail on the shootout mailing list and urged
the maintainers to persue well defined benchmarks but they did not take
heed.