[Caml-list] Is OCaml fast?
Thanassis Tsiodras
I am on the fence about whether to learn OCaml or not, and while
reading an article called "Why OCaml"
(http://www.cs.ubc.ca/~murphyk/Software/Ocaml/why_ocaml.html), I saw
that OCaml was praised for the speed of the executables it generates -
and was referred to, speed-wise, as "second to none", except C and
C++.
However, when I actually went to the Language Shootout page suggested
in the article, I found out that OCaml is not 2nd, it is 13th, behind
languages like Haskell and C#...
(http://shootout.alioth.debian.org/u32/which-programming-languages-are-fastest.php)
Is it just hype, then? Or am I missing something?
--
What I gave, I have; what I spent, I had; what I kept, I lost. (Old Epitaph)
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Gregory Bellier
choice.
It depends on the kind of apps you'd like to write. Even though O'Caml is
fast, it's not the first criteria I have in mind which would be security :
no segfault, no need to handle horrible stuff like in C, ...
2010/11/22 Thanassis Tsiodras <ttsi...@gmail.com>
> I apologize beforehand if this is not the forum to ask.
>
> I am on the fence about whether to learn OCaml or not, and while
> reading an article called "Why OCaml"
> (http://www.cs.ubc.ca/~murphyk/Software/Ocaml/why_ocaml.html<http://www.cs.ubc.ca/%7Emurphyk/Software/Ocaml/why_ocaml.html>),
Thomas Fischbacher
Gregory Bellier wrote:
> If you want speed then you should learn assembly or choose C as a second
> choice.
Certainly not assembly. Modern microprocessors just see assembly as a
kind-of high level language which they interpret in a funny way, doing
all sorts of re-schedulings, register renamings, delayed branching,
speculative execution etc. Producing code that can take advantage of
this is best left to the compiler - unless one has to deal with
instruction set extensions for which there is no good compiler
support yet.
> It depends on the kind of apps you'd like to write. Even though O'Caml
> is fast, it's not the first criteria I have in mind which would be
> security : no segfault, no need to handle horrible stuff like in C, ...
I'd say the OCaml native compiler is reasonably fast for pretty much all
applications that need the speed of compiled code. But I'd say the same
for GHC and SBCL, say. There are some good reasons to take a closer look
at OCaml, but these are related to other qualities of the language.
In my view, its greatest benefit is that it makes working with closures
simple while being fast and by far not as intimidating to (prospective)
PhD students as Scheme/Lisp (but that only because most of them have
been spoilt by imperative languages beforehand).
--
best regards,
Thomas Fischbacher
t.fisc...@soton.ac.uk
Sylvain Le Gall
On 22-11-2010, Thanassis Tsiodras <ttsi...@gmail.com> wrote:
> I apologize beforehand if this is not the forum to ask.
>
> I am on the fence about whether to learn OCaml or not, and while
> reading an article called "Why OCaml"
> (http://www.cs.ubc.ca/~murphyk/Software/Ocaml/why_ocaml.html), I saw
> that OCaml was praised for the speed of the executables it generates -
> and was referred to, speed-wise, as "second to none", except C and
> C++.
>
> However, when I actually went to the Language Shootout page suggested
> in the article, I found out that OCaml is not 2nd, it is 13th, behind
> languages like Haskell and C#...
> (http://shootout.alioth.debian.org/u32/which-programming-languages-are-fastest.php)
Shootout benchmarks doesn't always allow to set some variables that
could greatly improve the speed of OCaml.
>
> Is it just hype, then? Or am I missing something?
>
You could write very fast application with OCaml -- even beating C code
in some case. But what OCaml is really helpful for, is that you can do
algorithmic optimizations that improve the speed. It means that OCaml
is terse enough to allow you to write complex algorithm without making
too much bugs.
All in all: development time + execution time is fast.
But if you spend 10x development time on the same C code, you will
obviously get something faster in C.
Regards,
Sylvain Le Gall
Dario Teixeira
> I am on the fence about whether to learn OCaml or not, and while
> reading an article called "Why OCaml"
> (http://www.cs.ubc.ca/~murphyk/Software/Ocaml/why_ocaml.html),
> I saw that OCaml was praised for the speed of the executables it
> generates - and was referred to, speed-wise, as "second to none",
> except C and C++.
Yes, Ocaml is fast (more on that later), but you should consider speed
as just the icing in the cake. Besides all the cool language features,
what makes me recommend Ocaml to people who -- like you -- may be on
the fence, is that among the not so mainstream languages, Ocaml is
perhaps the one people are more likely to stick with for actual work
after they get past the learning stage. Remember that though Ocaml
favours the functional paradigm, it is flexible enough to support those
(few) situations where it actually makes sense to be imperative and/or
use an object oriented approach, and where a purely functional solution
is just intellectual masturbation.
> However, when I actually went to the Language Shootout page suggested
> in the article, I found out that OCaml is not 2nd, it is 13th, behind
> languages like Haskell and C#...
> (http://shootout.alioth.debian.org/u32/which-programming-languages-are-fastest.php)
>
> Is it just hype, then? Or am I missing something?
There's lies, damn lies, and shootout statistics. Getting good performance
in Haskell is often something of a black art that baffles even gurus. In
contrast, Ocaml has very good "performance predictability". Moreover, it
is possible to get good performance from fairly idiomatic Ocaml programmes,
whereas in other languages you may need to write ugly code which looks like
nothing one would write for elegance and sanity.
Best regards,
Dario Teixeira
Gerd Stolpmann
> I apologize beforehand if this is not the forum to ask.
>
> I am on the fence about whether to learn OCaml or not, and while
> reading an article called "Why OCaml"
> (http://www.cs.ubc.ca/~murphyk/Software/Ocaml/why_ocaml.html), I saw
> that OCaml was praised for the speed of the executables it generates -
> and was referred to, speed-wise, as "second to none", except C and
> C++.
>
> However, when I actually went to the Language Shootout page suggested
> in the article, I found out that OCaml is not 2nd, it is 13th, behind
> languages like Haskell and C#...
> (http://shootout.alioth.debian.org/u32/which-programming-languages-are-fastest.php)
>
> Is it just hype, then? Or am I missing something?
I think the shootout is not a good data source. There are definitely
some very poor Ocaml results there, so I'd guess the shootout got
recently more attention by enthusiasts of other languages, and the
current Ocaml programs there are not very good. (I remember Ocaml was #1
at the shootout a few years ago, faster than C.) So maybe a good
opportunity to post better Ocaml solutions there?
Gerd
--
------------------------------------------------------------
Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany
ge...@gerd-stolpmann.de http://www.gerd-stolpmann.de
Phone: +49-6151-153855 Fax: +49-6151-997714
------------------------------------------------------------
Sylvain Le Gall
> Am Montag, den 22.11.2010, 15:21 +0200 schrieb Thanassis Tsiodras:
>> I apologize beforehand if this is not the forum to ask.
>>
>> I am on the fence about whether to learn OCaml or not, and while
>> reading an article called "Why OCaml"
>> (http://www.cs.ubc.ca/~murphyk/Software/Ocaml/why_ocaml.html), I saw
>> that OCaml was praised for the speed of the executables it generates -
>> and was referred to, speed-wise, as "second to none", except C and
>> C++.
>>
>> However, when I actually went to the Language Shootout page suggested
>> in the article, I found out that OCaml is not 2nd, it is 13th, behind
>> languages like Haskell and C#...
>> (http://shootout.alioth.debian.org/u32/which-programming-languages-are-fastest.php)
>>
>> Is it just hype, then? Or am I missing something?
>
> I think the shootout is not a good data source. There are definitely
> some very poor Ocaml results there, so I'd guess the shootout got
> recently more attention by enthusiasts of other languages, and the
> current Ocaml programs there are not very good. (I remember Ocaml was #1
> at the shootout a few years ago, faster than C.) So maybe a good
> opportunity to post better Ocaml solutions there?
>
Maybe it is time to join forces and update the shootout for OCaml?
If some people are interested here, I can setup a repository on the forge
to update all this tests.
If I get 2 people interested, I will setup a VCS repository + project on
the forge, this afternoon. Drop me an email + preferred VCS (among svn,
git, darcs) and your account login on http://forge.ocamlcore.org. I will
take care, when ready, to made this code available in the shootout once
finished.
Regards,
Sylvain Le Gall
bluestorm
> I think the shootout is not a good data source. There are definitely
> some very poor Ocaml results there, so I'd guess the shootout got
> recently more attention by enthusiasts of other languages, and the
> current Ocaml programs there are not very good. (I remember Ocaml was #1
> at the shootout a few years ago, faster than C.) So maybe a good
> opportunity to post better Ocaml solutions there?
>
As Sylvain noticed, some (in not most) of the OCaml poor performances in the
shootout are actually not due to bad OCaml programs, but to arbitrary
restrictions in the shootout rules. For example, one of the bad-performing
benchmark for OCaml is the binary-tree benchmark, where it is nearly four
times slower than C, but on closer inspection you discover that this is due
to the arbitrary choice to forbid any change of the GC parameters. With
appropriate GC parameters, the very same OCaml program is exactly as fast as
C.
http://shootout.alioth.debian.org/u32/performance.php?test=binarytrees
« Note: these programs are being measured with *the default initial heap
size* - the measurements may be very different with a larger initial heap
size or GC tuning. »
C version : 12.11 secs
OCaml version : 47.22 secs
OCaml version with GC parameters tuned ("interesting alternative" section) :
12.67 secs
Therefore, there is nothing that can be changed to the OCaml submission for
this benchmark to improve performances, except changing the default GC
parameters; while this might be a good idea in general, changing it only for
the sake of shootout-obsessed people is ridiculous.
Török Edwin
Isn't it possible for the GC to realise its doing too many collections
and increase the minor heap size on its own?
Or isn't it possible to determine at compile time an approximation for
a good heap size, and use that?
Best regards,
--Edwin
Gerd Stolpmann
> On Mon, Nov 22, 2010 at 3:04 PM, Gerd Stolpmann
> <in...@gerd-stolpmann.de> wrote:
> I think the shootout is not a good data source. There are
> definitely
> some very poor Ocaml results there, so I'd guess the shootout
> got
> recently more attention by enthusiasts of other languages, and
> the
> current Ocaml programs there are not very good. (I remember
> Ocaml was #1
> at the shootout a few years ago, faster than C.) So maybe a
> good
> opportunity to post better Ocaml solutions there?
>
>
> As Sylvain noticed, some (in not most) of the OCaml poor performances
> in the shootout are actually not due to bad OCaml programs, but to
> arbitrary restrictions in the shootout rules. For example, one of the
> bad-performing benchmark for OCaml is the binary-tree benchmark, where
> it is nearly four times slower than C, but on closer inspection you
> discover that this is due to the arbitrary choice to forbid any change
> of the GC parameters. With appropriate GC parameters, the very same
> OCaml program is exactly as fast as C.
>
>
> http://shootout.alioth.debian.org/u32/performance.php?test=binarytrees
>
>
> heap size - the measurements may be very different with a larger
> initial heap size or GC tuning. »
> C version : 12.11 secs
> OCaml version : 47.22 secs
> OCaml version with GC parameters tuned ("interesting alternative"
> section) : 12.67 secs
>
>
>
>
> Therefore, there is nothing that can be changed to the OCaml
> submission for this benchmark to improve performances, except changing
> the default GC parameters; while this might be a good idea in general,
> changing it only for the sake of shootout-obsessed people is
> ridiculous.
It's in deed an unfair comparison: In C they use the Apache runtime
which provides memory pools. This is something that does not extend to
most real world programs.
Because it's ridiculous anyway: Encode the tree in an array. Not really
idiomatic, but in C they also do not use the idiomatic memory management
(malloc/free).
Goswin von Brederlow
Maybe. The GC would have to see that with a larger minor heap it has
sufficient space so that most values die before the collection. It could
see that most values die after 1 major sweep and increasing the minor
heap would likely keep them out of the major heap alltogether.
> Or isn't it possible to determine at compile time an approximation for
> a good heap size, and use that?
Usualy that is dependent on the input.
MfG
Goswin
Fabrice Le Fessant
> So maybe a good opportunity to post better Ocaml solutions there?
I spent some time improving OCaml solutions, and most of the time, my
solutions were refused: the organizers don't let you unroll loops, fix
GC parameters, etc. One strength of OCaml is that it runs fairly fast
immediatly, but you can make it run even faster with some tuning,
something that you cannot do with many other languages. Such tuning is
not allowed there, so, it is not a good place for OCaml hackers. If
the organizers had been consistent, they would have forbidden Haskell
programmers from adding strictness in their programs, but then, there
programs would have been 2x slower than OCaml programs.
Well, actually, things might improve in the future, if we manage to
improve the performance of the compiler itself, and have more
"official" libraries. For example, most parallel solutions in OCaml
currently use fork() and sockets for communication. Using an official
shared-memory library, we could directly improve the performances of
many OCaml solutions.
--Fabrice
Fabrice Le Fessant
> Isn't it possible for the GC to realise its doing too many collections
> and increase the minor heap size on its own?
Indeed, it could notice that a lot of data is being moved to the major
heap, and double its size in consequence, until a maximum limit is
reached.
The problem is that it is the kind of things that are application
dependent, and should be put in the program itself (the program would
have a trigger on each minor heap collection, and, depending on the
moved bytes, would increase the size of the minor heap). The problem
is that the Shootout does not allow that, so the winner is the
language whose runtime allocates the most memory at the beginnning...
--Fabrice
Oliver Bandel
[...]
> (I remember Ocaml was #1
> at the shootout a few years ago, faster than C.) So maybe a good
> opportunity to post better Ocaml solutions there?
Yes I also remember that.
I hope that the new OCaml compilers did not
make OCaml lessperformance by enhancing other features.
And I don't realy think so.
But were the old code-snippets emoved, or what was going on,
that OCaml degraded that much?
From my experience - normally using ocamlc for most of my OCaml
programs - it's even fast enough with this bytecode.
If it will be not fast enough, I have an option as a joker:
compiling to native code. :)
Doing this is rather psychologically, as I like to have a potential
for enhancement. :)
If you really always need the best performance, of course compiling
to native code will be best choice.
But at lest for my work I have not found out really bottlenecks,
and always was faster than doing things in Perl or Python or so.
Learning OCaml IMHO does make sense in any case. :)
But be aware one big disadvantage: if you have learned Ocaml,
this may absolutely kill your motivation in learning other languages
afterwards.So,if you need some other languages for your job, learn
them first, and learn OCaml after this, for your pleasure.
But you may never have fun at work then, until your boss allows you to
use OCaml.
And this is not a joke, I mean it as I wrote it!
Ciao,
Oliver
David Rajchenbach-Teller
On Nov 22, 2010, at 6:02 PM, Oliver Bandel wrote:
> Zitat von "Gerd Stolpmann" <in...@gerd-stolpmann.de>:
> [...]
>> (I remember Ocaml was #1
>> at the shootout a few years ago, faster than C.) So maybe a good
>> opportunity to post better Ocaml solutions there?
> [...]
>
> Yes I also remember that.
> I hope that the new OCaml compilers did not
> make OCaml lessperformance by enhancing other features.
>
> And I don't realy think so.
>
> But were the old code-snippets emoved, or what was going on,
> that OCaml degraded that much?
>
_______________________________________________
Oliver Bandel
..hmhhh..
..looks like they are biased...
... not that we are not ;)
..but... as the GC-stuff is available FROM WITHING the language,
in the standard-lib, this is nothing added on later.
And I think it should also be allowed to be used.
To reject environment variables, I can see as acceptable in this case,
but rejecting the GC-stuff does not make sense, because, as just
mentioned, it is avalable by the programmer from within the code.
What about compiling parameters?
I mean: in C you can use -O for optimization.
This should also be forbidden then.... Is it?
There are so much possibilities to influence the results,
that blocking Gc-module is idiotic, IMHO.
Ciao,
Oliver
P.S.:
I looked at one of the C-makefiles:
usr/bin/gcc -pipe -Wall -O3 -fomit-frame-pointer -march=native
-fopenmp -D_FILE_OFFSET_BITS=64 -I/usr/include/apr-1.0 -lapr-1 -lgomp
binarytrees.gcc-7.c -o binarytrees.gcc-7.gcc_run
rm binarytrees.gcc-7.c
So, -O3 is allowed.
AFAIK with O3 and higher, inline does work.
__inline__ must be forbidden as well as -O3
Optimization should be switched off completely, if
OCaml's optimizations are also not allowed.
Zitat von "David Rajchenbach-Teller" <David....@univ-orleans.fr>:
David Rajchenbach-Teller
On Nov 22, 2010, at 6:23 PM, Oliver Bandel wrote:
>
> ...hmhhh..
>
> ...looks like they are biased...
>
>
> .... not that we are not ;)
Oliver Bandel
Zitat von "Thanassis Tsiodras" <ttsi...@gmail.com>:
> I apologize beforehand if this is not the forum to ask.
>
> I am on the fence about whether to learn OCaml or not, and while
> reading an article called "Why OCaml"
> (http://www.cs.ubc.ca/~murphyk/Software/Ocaml/why_ocaml.html), I saw
> that OCaml was praised for the speed of the executables it generates -
> and was referred to, speed-wise, as "second to none", except C and
> C++.
[...]
I remember one argument on that topic, many years ago,
maybe rom Xavier leoy or other core developers, and it goes like that:
Benchmarks on such small problem solving is ONE comparision, what
makes sense, but it must be added by comparisions of "optimizations"
of the code in large applications.
The lower the level of the öanguage the harder it is to keep track of
possible optimizations on a very big program.
But the higher the language level is, the easier it is to
keep that in mind.
So: when optmizing small snippets of code this helps optimizing
performance in maybe critical sections, but to get an overview on all
interdependencies is close to unmanagable with lower leveled languages.
I have not seen a comparison of performance in this repect,
but I also would think higher level languages might be a better choice here.
Any comparisions on that scale of programming, which you know of?
Then please throw in a link here.
Thanks.
Ciao,
Oliver
Török Edwin
Fabrice Le Fessant <fab...@lefessant.net> wrote:
> 2010/11/22 Török Edwin <edwin...@gmail.com>:
> > Isn't it possible for the GC to realise its doing too many
> > collections and increase the minor heap size on its own?
>
> Indeed, it could notice that a lot of data is being moved to the major
> heap, and double its size in consequence, until a maximum limit is
> reached.
I did some benchmarks on 2 CPUs, and here are the wall clock times
for GC minor heap size:
"minorheap"," phenomII",,," core2",,,"cycle diff %"
," time",," cycles","time",," cycles",
32," 17.76","17.74",56.8," 48.60","48.86",58.48,2.96
64," 16.81","16.80",53.78,"44.79","44.79",53.75,-0.06
128," 15.41","15.38",49.26,"41.38","40.76",49.28,0.04
256," 14.39","14.35",45.98,"39.75","38.98",47.24,2.74
512," 12.97","13.15",41.79,"35.75","35.59",42.8,2.42
1024," 12.89","12.94",41.33,"33.57","33.13",40.02,-3.17
2048," 11.05","11.09",35.42,"29.26","29.12",35.03,-1.1
4096," 9.79"," 9.81",31.36,"26.21","25.96",31.3,-0.19
8192," 8.71"," 8.85",28.1," 23.36","23.34",28.02,-0.28
16384," 7.89"," 7.86",25.2," 21.41","21.54",25.77,2.26
32768," 7.55"," 7.55",24.16,"20.74","20.88",24.97,3.35
(minorheap is in KWords, time is in seconds, cycles is divided by 10^9)
Increasing minor heap beyond that yielded no improvement (number of
minor heap collections stayed the same).
phenomII has 64 Kb L1 data cache, 512Kb L2 cache, 6144Kb L3 cache
(shared), runs at 3.2Ghz. That would be 516k words if only 1 core used.
core2 has 32 Kb L1 data cache, 4MB L2 cache, runs at 1.2Ghz.
That would be 840k words if only 1 core used.
Both used exact same binaries on 64-bit Linux, ocaml 3.11.2.
Despite the difference in CPUs and heap sizes the number of CPU cycles
spent for a given size of the minor heap is about the same (within 3%
difference).
Not sure what the max should be for the minor heap increase, but based
on this benchmark increasing size of minor heap never slows down the
program. Even when size of minor heap exceeds what fits in the cache.
I guess there is another microbenchmark that can show the opposite
though.
Is there some real world benchmark for OCaml's GC that can be used
to get some better values for minor heap size based on CPU's L2/L3 size
(as suggested in the 'Optimizing garbage collection' thread)?
>
> The problem is that it is the kind of things that are application
> dependent,
Yes, maybe optimal size of minor heap doesn't depend on CPU's cache
size but on the application, in which case a heuristic for
increasing/decreasing the heap size may be better?
> and should be put in the program itself (the program would
> have a trigger on each minor heap collection, and, depending on the
> moved bytes, would increase the size of the minor heap). The problem
> is that the Shootout does not allow that, so the winner is the
> language whose runtime allocates the most memory at the beginnning...
If tuning minor heap can double performance of the program, then the GC
should have some heuristics to tune it automatically. Sure applications
which are not happy with that tuning should tune it themselves.
Best regards,
--Edwin
Eray Ozkural
time in parallel computing, when some sharp programmer decides to implement
a parallel algorithm in Java. :) I keep asking myself questions like do you
also program ciphers in Flash? Graph algorithms in PHP?
In my experience, ocaml is pretty fast, quite comparable to C++, for
computation-intensive tasks and of course excellent at traditional
functional programming tasks. Since I use both kinds of algorithms in my
research, ocaml is better than C++ for me (despite the great boost libs),
except for shared memory parallelism, which is still non-existent in ocaml.
Try implementing something that has a large time/memory complexity like a
data mining algorithm, and you'll see the difference from Haskell/Java/Blah.
Just run the algorithms in Weka, the famous data mining suite written in
Java, and compare them to any sensible implementation of the same
algorithms. Measure the difference, it's unbelievable how awful Java is at
any kind of serious computing! I think some conservative humans from the
future sent the designers of Java back in time to stop the singularity.
Cheers,
--
Eray Ozkural, PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara
http://groups.yahoo.com/group/ai-philosophy
http://myspace.com/arizanesil http://myspace.com/malfunct
Jeff Schultz
> ...looks like they are biased...
> .... not that we are not ;)
> To reject environment variables, I can see as acceptable in this case,
> but rejecting the GC-stuff does not make sense, because, as just mentioned,
> it is avalable by the programmer from within the code.
I haven't looked at the "benchmark game" or its predecessor for many
years, but all this really says is that the programs it measures are
sufficiently unlike real applications that the OCaml developers
haven't been tempted to tune the GC defaults for them.
The conclusion to draw would be that the shootout results are a
poor, and probably misleading, evidence base for selecting a
programming language for execution time performance. Measured
programming language implementation performance gets faster or
slower over the years due to its own rule changes and language and
library implementation changes, but I doubt that this conclusion
changes much ;-)
The one thing the shootout programs are useful for is to get a feel
for a variety of different languages' expressivity for a common set
of "small" problems.
Jeff Schultz
Isaac Gouy
-snip-
> Is it just hype, then? Or am I missing something?
Note the "Why Ocaml?" date - December 2002.
Isaac Gouy
> I can confirm that old code-snippets were removed (and that both faster
solutions and environment
> variable tweaks were rejected).
Even back in 2001, Doug Bagley had noted all the things that were
wrong with the tasks on his "The Great Computer Language Shootout".
During autumn 2004, on Brent Fulgham's website, new tasks were added
and old Doug Bagley tasks removed.
By 2005 only 2 tasks remained from that old Doug Bagley website.
By August 2008 none of those old Doug Bagley tasks were measured for
the current benchmarks game.
Isaac Gouy
-snip-
> I think the shootout is not a good data source. There are definitely
> some very poor Ocaml results there, so I'd guess the shootout got
> recently more attention by enthusiasts of other languages, and the
> current Ocaml programs there are not very good. (I remember Ocaml was #1
> at the shootout a few years ago, faster than C.) So maybe a good
> opportunity to post better Ocaml solutions there?
Even back in 2001, on Doug Bagley's website called "The Great Computer Language
Shootout" Ocaml was not #1-
http://web.archive.org/web/20010602051335/www.bagley.org/~doug/shootout/craps.shtml
Doug Bagley stopped updating his website in Fall 2001.
Even back in 2004, when Brent Fulgham started his website using the same
programs, OCaml was not #1
http://web.archive.org/web/20040717184911/shootout.alioth.debian.org/craps.php
Isaac Gouy
-snip-
> With appropriate GC parameters, the very same OCaml program is exactly as fast
as C.
>
>
> http://shootout.alioth.debian.org/u32/performance.php?test=binarytrees
>
> « Note: these programs are being measured with the default initial heap size -
the measurements may be very different with a larger initial heap size or GC
tuning. »
>
>
> C version : 12.11 secs
> OCaml version : 47.22 secs
> OCaml version with GC parameters tuned ("interesting alternative" section) :
12.67 secs
And of course you know because that GC tuned OCaml program is shown on the
benchmarks game website ;-)
http://shootout.alioth.debian.org/u32/program.php?test=binarytrees&lang=ocaml&id=1
Isaac Gouy
-snip-
> There's lies, damn lies, and shootout statistics.
-snip-
After all, facts are facts,
and although we may quote one to another with a chuckle
the words of the Wise Statesman, 'Lies--damned lies--and statistics,'
still there are some easy figures the simplest must understand,
and the astutest cannot wriggle out of.
Leonard Henry Courtney, 1895
:-)
Fabrice Le Fessant
America" by Cynthia Crossen ?
--Fabrice
Isaac Gouy wrote, On 11/23/2010 03:20 AM:
> Dario Teixeira <darioteixeira <at> yahoo.com> writes:
> -snip-
>> There's lies, damn lies, and shootout statistics.
> -snip-
>
> After all, facts are facts,
> and although we may quote one to another with a chuckle
> the words of the Wise Statesman, 'Lies--damned lies--and statistics,'
> still there are some easy figures the simplest must understand,
> and the astutest cannot wriggle out of.
>
> Leonard Henry Courtney, 1895
_______________________________________________
Jon Harrop
spectral-norm, Mandelbrot, n-body, fannkuch-redux, k-nucleotide, regex-dna)
abuse GHC's FFI in order to work around Haskell.
The k-nucleotide benchmark in Haskell even uses malloc! Ketil Malde crafted
a much better solution but noted:
"This is considered cheating, since it is the easy and natural way to do
it." -
http://www.haskell.org/haskellwiki/Shootout/Knucleotide
Cheers,
Jon.
> -----Original Message-----
> From: caml-lis...@yquem.inria.fr [mailto:caml-list-
> bou...@yquem.inria.fr] On Behalf Of Christophe TROESTLER
> Sent: 23 November 2010 10:38
> To: igo...@yahoo.com
> Cc: caml...@inria.fr
> Subject: Re: [Caml-list] Re: Is OCaml fast?
>
> On Tue, 23 Nov 2010 02:03:48 +0000, Isaac Gouy wrote:
> >
> > > C version : 12.11 secs
> > > OCaml version : 47.22 secs
> > > OCaml version with GC parameters tuned ("interesting alternative"
> > section) : 12.67 secs
> >
> > And of course you know because that GC tuned OCaml program is shown
> > on the
> > benchmarks game website ;-)
> > http://shootout.alioth.debian.org/u32/program.php?test=binarytrees&
> > lang=ocaml&id=1
>
> Since you are here, please explain why C can use memory pools and vec
> tor instructions but tuning the GC of OCaml ― although it is part of
> the standard library ― is considered an "alternative".
>
> Best,
> C.
Jon Harrop
the -H command-line parameter:
http://shootout.alioth.debian.org/u64q/program.php?test=regexdna&lang=ghc&id
=2
> -----Original Message-----
> From: caml-lis...@yquem.inria.fr [mailto:caml-list-
> bou...@yquem.inria.fr] On Behalf Of Christophe TROESTLER
> Sent: 23 November 2010 10:38
> To: igo...@yahoo.com
> Cc: caml...@inria.fr
> Subject: Re: [Caml-list] Re: Is OCaml fast?
>
> On Tue, 23 Nov 2010 02:03:48 +0000, Isaac Gouy wrote:
> >
> > > C version : 12.11 secs
> > > OCaml version : 47.22 secs
> > > OCaml version with GC parameters tuned ("interesting alternative"
> > section) : 12.67 secs
> >
> > And of course you know because that GC tuned OCaml program is shown
> > on the
> > benchmarks game website ;-)
> > http://shootout.alioth.debian.org/u32/program.php?test=binarytrees&
> > lang=ocaml&id=1
>
> Since you are here, please explain why C can use memory pools and vec
> tor instructions but tuning the GC of OCaml ― although it is part of
> the standard library ― is considered an "alternative".
>
> Best,
> C.
>
Isaac Gouy
>
> On Tue, 23 Nov 2010 02:03:48 +0000, Isaac Gouy wrote:
> >
> > > C version : 12.11 secs
> > > OCaml version : 47.22 secs
> > > OCaml version with GC parameters tuned ("interesting alternative"
> > section) : 12.67 secs
> >
> > And of course you know because that GC tuned OCaml program is shown
> > on the
> > benchmarks game website
> > http://shootout.alioth.debian.org/u32/program.php?test=binarytrees&
> > lang=ocaml&id=1
>
> Since you are here, please explain why C can use memory pools and vec
> tor instructions but tuning the GC of OCaml ― although it is part of
> the standard library ― is considered an “alternative”.
You seem to be suggesting that "tuning the GC" is considered "alternative" only
for OCaml programs.
You seem to be suggesting that "tuning the GC" is considered "alternative" for
every task.
Neither is true.
You seem to be suggesting some kind of equivalence between vector instructions
and "tuning the GC".
You haven't said why they should be considered equivalent.
Nor have you said why you think C should not be allowed to use memory pools.
Isaac Gouy
Fabrice Le Fessant <fabrice.le_fessant <at> inria.fr> writes:
> Maybe you should read "Tainted Truth: The Manipulation of Fact In
> America" by Cynthia Crossen ?
Isaac Gouy
>
> Note that the regex-dna solution for Haskell tweaks its GC parameters via
> the -H command-line parameter:
Note that there is no restriction on "tuning the GC" for regex-dna.
Note that there is no restriction on "tuning the GC" for any task except
binary-trees.
Isaac Gouy
> Ketil Malde crafted
> a much better solution but noted:
>
> "This is considered cheating, since it is the easy and natural way to do
> it." -
> http://www.haskell.org/haskellwiki/Shootout/Knucleotide
Not even cheating - just an answer to a different question.
Török Edwin
Isaac Gouy <igo...@yahoo.com> wrote:
> Jon Harrop <jonathandeanharrop <at> googlemail.com> writes:
>
> >
> > Note that the regex-dna solution for Haskell tweaks its GC
> > parameters via the -H command-line parameter:
>
>
> Note that there is no restriction on "tuning the GC" for regex-dna.
>
> Note that there is no restriction on "tuning the GC" for any task
> except binary-trees.
Sounds good. Then Ocaml could still win if it performs well on the
other benchmarks.
Best regards,
--Edwin
Gerd Stolpmann
> Christophe TROESTLER <Christophe.Troestler+ocaml <at> umh.ac.be> writes:
>
> >
> > On Tue, 23 Nov 2010 02:03:48 +0000, Isaac Gouy wrote:
> > >
> > > > C version : 12.11 secs
> > > > OCaml version : 47.22 secs
> > > > OCaml version with GC parameters tuned ("interesting alternative"
> > > section) : 12.67 secs
> > >
> > > And of course you know because that GC tuned OCaml program is shown
> > > on the
> > > benchmarks game website
> > > http://shootout.alioth.debian.org/u32/program.php?test=binarytrees&
> > > lang=ocaml&id=1
> >
> > Since you are here, please explain why C can use memory pools and vec
> > tor instructions but tuning the GC of OCaml ― although it is part of
> > the standard library ― is considered an “alternative”.
>
>
> You seem to be suggesting that "tuning the GC" is considered "alternative" only
> for OCaml programs.
>
> You seem to be suggesting that "tuning the GC" is considered "alternative" for
> every task.
>
> Neither is true.
>
> You seem to be suggesting some kind of equivalence between vector instructions
> and "tuning the GC".
> You haven't said why they should be considered equivalent.
>
> Nor have you said why you think C should not be allowed to use memory pools.
Quite easy: because you are comparing results that cannot be compared.
The reader of this benchmark (binary trees) might have the impression
that C is generally that fast - however, this would be no longer true if
these binary trees were used as library in a bigger program where using
memory pools is inappropriate, e.g. because the data managed by the
binary trees has an unpredictable lifetime.
I do not say that it is complete nonsense to do this comparison, but
only that it is more specific than a reader would assume. The innocent
reader expects a comparison of binary tree performance, not of methods
of managing memory (and this is it finally). The true name of this test
should be "manage_many_small_memory_cells_where_pools_are_allowed". (It
would be actually interesting to compare various versions of this test
with different memory management methods.)
Gerd
>
>
>
>
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
--
------------------------------------------------------------
Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany
ge...@gerd-stolpmann.de http://www.gerd-stolpmann.de
Phone: +49-6151-153855 Fax: +49-6151-997714
------------------------------------------------------------
Mark Diekhans
The source of this pharse is unknown, however Mark Twain is credited with
making it well known:
"There are three kinds of lies: lies, damned lies and statistics."
Isaac Gouy
>
>
> The source of this pharse is unknown, however Mark Twain is credited with
> making it well known:
>
> "There are three kinds of lies: lies, damned lies and statistics."
http://www.york.ac.uk/depts/maths/histstat/lies.htm
Isaac Gouy
-snip-
> Sounds good. Then Ocaml could still win if it performs well on the
> other benchmarks.
The main benchmarks game summary is median and quartiles so one measurement
doesn't have much influence.
There are several tasks which have worse performing OCaml programs than
binary-trees.
Isaac Gouy
-snip-
> I do not say that it is complete nonsense to do this comparison, but
> only that it is more specific than a reader would assume.
A reader's wrong assumptions are their own responsibility:
http://shootou.alioth.debian.org/flawed-benchmarks.php
> The innocent reader expects a comparison of binary tree performance,
> not of methods of managing memory (and this is it finally).
Perhaps rather than "innocent reader" you mean careless reader who didn't bother
to read what the programs should do?
http://shootout.alioth.debian.org/u32q/benchmark.php?test=binarytrees&lang=all#about
> The true name of this test should be
> "manage_many_small_memory_cells_where_pools_are_allowed".
"binary-trees benchmark : Allocate and deallocate many many binary trees"
> (It would be actually interesting to compare various versions of this test
> with different memory management methods.)
So do that comparison and publish the results.
Gerd Stolpmann
> > (It would be actually interesting to compare various versions of this test
> > with different memory management methods.)
>
> So do that comparison and publish the results.
Please don't tell me what I am supposed to do. I'm not a troll like
others.
Gerd
--
------------------------------------------------------------
Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany
ge...@gerd-stolpmann.de http://www.gerd-stolpmann.de
Phone: +49-6151-153855 Fax: +49-6151-997714
------------------------------------------------------------
_______________________________________________
Christophe TROESTLER
>
> Christophe TROESTLER writes:
>
> > Since you are here, please explain why C can use memory pools and vec
> > tor instructions but tuning the GC of OCaml ― although it is part of
> > the standard library ― is considered an “alternative”.
>
> instructions and "tuning the GC". You haven't said why they should
> be considered equivalent.
I did not say they are equivalent. It is a matter of what is allowed
(or not) and for what reason. The question is why is C allowed to use
an external library for managing its memory but — for this specific
benchmark which is precisely about managing memory — OCaml is not
authorized to make use of its very own library!
On Tue, 23 Nov 2010 18:03:10 +0000, Isaac Gouy wrote:
>
> Note that there is no restriction on "tuning the GC" for regex-dna.
>
> Note that there is no restriction on "tuning the GC" for any task except
> binary-trees.
Note that you did not answer my question.
Isaac Gouy
> > > (It would be actually interesting to compare various versions of this test
> > > with different memory management methods.)
> >
> > So do that comparison and publish the results.
>
> Please don't tell me what I am supposed to do. I'm not a troll like
> others.
The tone of my comment was not as harsh as you heard.
Jon Harrop
> -----Original Message-----
> From: caml-lis...@yquem.inria.fr [mailto:caml-list-
Isaac Gouy
-snip-
> The question is why is C allowed to use
> an external library for managing its memory
I asked why you think C should not be allowed to use memory pools - you haven't
tried to answer that question.
If you think that C should not be allowed to use memory pools because they are
provided in an external library then I disagree - I think it's fine to show use
of popular external libraries.
Of course, a C program that just uses malloc is also shown.
> but — for this specific
> benchmark which is precisely about managing memory — OCaml is not
> authorized to make use of its very own library!
OCaml is authorized to make use of its very own library.
Just like all the other programming language implementations, for binary-trees,
OCaml is not authorized to tune the GC.
oli...@first.in-berlin.de
> Hi,
>
> > I am on the fence about whether to learn OCaml or not, and while
> > reading an article called "Why OCaml"
> > (http://www.cs.ubc.ca/~murphyk/Software/Ocaml/why_ocaml.html),
> > I saw that OCaml was praised for the speed of the executables it
> > generates - and was referred to, speed-wise, as "second to none",
> > except C and C++.
>
> Yes, Ocaml is fast (more on that later), but you should consider speed
> as just the icing in the cake.
[...]
But if it were not performant enough, I (and many others too, I think)
would consider it being a nice toy language.
AFAIK in the past, functional langauges were not adapted, because they were
very unperformant - at least this is one reason.
Another reason might be, that the available functional languages in the past
were overloaded with parenthess ;)
Ciao,
Oliver
oli...@first.in-berlin.de
> David Rajchenbach-Teller <David.Teller <at> univ-orleans.fr> writes:
>
>
> > I can confirm that old code-snippets were removed (and that both faster
> solutions and environment
> > variable tweaks were rejected).
>
>
> Even back in 2001, Doug Bagley had noted all the things that were
> wrong with the tasks on his "The Great Computer Language Shootout".
And what was wrong in his eyes?
>
> During autumn 2004, on Brent Fulgham's website, new tasks were added
> and old Doug Bagley tasks removed.
>
> By 2005 only 2 tasks remained from that old Doug Bagley website.
>
> By August 2008 none of those old Doug Bagley tasks were measured for
> the current benchmarks game.
So, now the comparisions are perfect?
What problems were removed?
Ciao,
Oliver
Jon Harrop
> AFAIK in the past, functional langauges were not adapted, because they
> were
> very unperformant - at least this is one reason.
> Another reason might be, that the available functional languages in the
> past
> were overloaded with parenthess ;)
That was also true of early ML implementations. When I was first taught ML at university we used the Cambridge ML interpreter and you actually had to sit there and wait for it to solve the 8-queens problem. At the time, I thought ML was a complete joke and could see no use for it outside its very specific domain of theorem proving and actually really resented being taught it on a general CS course. I know better now though. ;-)
Later language implementations inherited many of these inefficiencies though. Many of the things that can make OCaml and Java slow were inherited from Lisp. They are, in effect, designs that still bear the burden of dynamic typing despite being statically typed.
Cheers,
Jon.
Isaac Gouy
> > Note that there is no restriction on "tuning the GC" for regex-dna.
> >
> > Note that there is no restriction on "tuning the GC" for any task except
> > binary-trees.
>
> Don't you think this is a pretty ludicrous restriction?
>
> Tuning the GC / adjusting the size of pools or mallocs is essential
> for optimizing for modern processors. I bet the C programs are doing
> this, except that it won't obviously be called "tuning the GC"
> although it amounts to precisely the same thing.
What you say would be interesting if you'd taken the trouble to look at the C
programs.
Isaac Gouy
> > Even back in 2001, Doug Bagley had noted all the things that were
> > wrong with the tasks on his "The Great Computer Language Shootout".
>
> And what was wrong in his eyes?
Find out for yourself:
http://web.archive.org/web/20010617014807/www.bagley.org/~doug/shootout/
> So, now the comparisions are perfect?
Has anyone said so?
> What problems were removed?
All of them.
Erik de Castro Lopo
> Many of the things that can make OCaml and Java slow were inherited
> from Lisp. They are, in effect, designs that still bear the burden
> of dynamic typing despite being statically typed.
I'm curious, what in your opinion are those things?
Erik
--
----------------------------------------------------------------------
Erik de Castro Lopo
http://www.mega-nerd.com/
Eray Ozkural
I think that this benchmark is lacking in the algorithms department. Where
is a dynamic programming problem? A graph algorithm? Anything with
non-trivial time/space complexity? Anything a little more complex than
matrix product?
Also, it's not uncommon to disallow low-level optimizations such as writing
memory allocators and async file access when comparing implementations of an
algorithm, but such restrictions should be carried out uniformly. In such a
benchmark I would expect each entry to stick to their guns, i.e. use only
the standard libraries and programming styles for instance. Linking in
foreign libraries must most definitely be disallowed. So, if in Java, it's
necessary to call the garbage collector explicitly from time to time, and we
had to do that for a long time, so be it. Or again, if in Java, performance
will suffer unless you only use arrays of integral types, the implementer
may wish to implement as much as is possible with arrays, though I wonder if
it is not better to choose the most natural implementation style for the
particular language. In the case of Java, the claim was that object-oriented
was some kind of a programming-aid that can replace talented programmers :)
It's unfortunate of course that some kinds of optimizations always have to
be made by hand, for instance in functional languages many compilers do not
have deforestation.
Otherwise, of course, any implementation may include a compiler for the
fastest language and present a program in that language, which is not the
objective.
An alternative objective could be to compare the shortest and most
comprehensible, if possible line-to-line compatible implementation of a
given pseudocode in different languages. That would be extremely informative
for serious algorithm researchers! If a computer scientist isn't sure of the
performance of the primitives, he cannot make sure his implementation will
comply with the time-complexity of the given algorithm.
Best Regards,
On Wed, Nov 24, 2010 at 2:23 AM, Isaac Gouy <igo...@yahoo.com> wrote:
> <oliver <at> first.in-berlin.de> writes:
>
> > > Even back in 2001, Doug Bagley had noted all the things that were
> > > wrong with the tasks on his "The Great Computer Language Shootout".
> >
> > And what was wrong in his eyes?
>
> Find out for yourself:
>
> http://web.archive.org/web/20010617014807/www.bagley.org/~doug/shootout/<http://web.archive.org/web/20010617014807/www.bagley.org/%7Edoug/shootout/>
>
>
> > So, now the comparisions are perfect?
>
> Has anyone said so?
>
>
> > What problems were removed?
>
> All of them.
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
--
Eray Ozkural, PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara
http://groups.yahoo.com/group/ai-philosophy
http://myspace.com/arizanesil http://myspace.com/malfunct
Jeff Meister
probably already done so. We know what your rules are for
binary-trees; repeating them does not help. Richard's objection, which
you dismissed out of hand, was that your no-GC-tuning rule is silly in
the light of actual uses of garbage collected programming languages on
modern processors. It makes your results unrealistic, and an
unrealistic benchmark is misleading, or at best merely useless. You
are free to tersely reject our constructive criticism, but the only
meaningful consequence will be that OCaml users consider the shootout
untrustworthy and completely ignore its results... what good are the
"language comparisons" your project makes if the communities behind
those languages don't support your benchmarking methods?
On Tue, Nov 23, 2010 at 6:13 PM, Isaac Gouy <igo...@yahoo.com> wrote:
> Eray Ozkural <examachine <at> gmail.com> writes:
>
>
>> Hello, I think that this benchmark is lacking ...
>
> http://shootout.alioth.debian.org/help.php#why
>
>
> Please make the kind of comparison you think should be done and publish it.
Andrew
>Everyone in this thread is capable of reading your site and has
>probably already done so. We know what your rules are for
>binary-trees; repeating them does not help. Richard's objection, which
>you dismissed out of hand, was that your no-GC-tuning rule is silly in
>the light of actual uses of garbage collected programming languages on
>modern processors. It makes your results unrealistic, and an
>unrealistic benchmark is misleading, or at best merely useless. You
>are free to tersely reject our constructive criticism, but the only
>meaningful consequence will be that OCaml users consider the shootout
>untrustworthy and completely ignore its results... what good are the
>"language comparisons" your project makes if the communities behind
>those languages don't support your benchmarking methods?
+1. Seriously, Isaac, try to calm down, everything is fine. You might want
to read what others write, I have the feeling that many people were making
valid points, whereas you have mostly been turning down any objection by
pointing people to the same webpage and again.
Andrew.
Isaac Gouy
> We know what your rules are for
> binary-trees; repeating them does not help.
When Christophe TROESTLER wrongly states - "OCaml is not authorized to make use
of its very own library!" - he shows that those rules are not known.
> Richard's objection, which you dismissed out of hand, was that your
> no-GC-tuning rule is silly in the light of actual uses of garbage collected
> programming languages on modern processors.
When said Richard opines about programs he apparently hasn't bothered to read, I
take that as a sign his opinions might not be based on anything solid.
> It makes your results unrealistic, and an
> unrealistic benchmark is misleading, or at best merely useless.
You should think that benchmarks (not just these) are unrealistic - your
application is the ultimate benchmark.
Useless? Wouldn't that depend on the objectives? The post you replied to
linked-to a 3 line statement of objectives - did you read it?
> You are free to tersely reject our constructive criticism
Do you really think saying something is "ludicrous" or "silly" is constructive
criticism? :-)
David Rajchenbach-Teller
On Nov 24, 2010, at 12:21 AM, evil sloot wrote:
> Hi,
>
> Regarding alternative solutions to the shootout problems:
> The thread ring problem can be easily implemented using Lwt
> yielding (pun intended ;)) performance
> comparable to the haskell solution
>
> have fun,
>
> Romain.
David Rajchenbach-Teller
Isaac Gouy
> +1. Seriously, Isaac, try to calm down, everything is fine. You might want
> to read what others write, I have the feeling that many people were making
> valid points, whereas you have mostly been turning down any objection by
> pointing people to the same webpage and again.
Seriously, Andrew, I have been calm.
You might want to read what others write.
I have the feeling that a list of those "valid points" would be a better basis
for discussion than your feeling.
A quick count shows 8 URIs all different - so you are plainly wrong, I have not
been "pointing people to the same webpage and again".
Christophe Troestler
>
> Jeff Meister <nanaki <at> gmail.com> writes:
>
> > We know what your rules are for binary-trees; repeating them does
> > not help.
>
> When Christophe TROESTLER wrongly states - "OCaml is not authorized
> to make use of its very own library!" - he shows that those rules
> are not known.
Isolating a sentence gives a misleading idea of what I said. Must I
really repeat that I was asking WHY is C allowed to use an external
library to enhance its memory managementน but OCaml cannot use its own
library to do that. I am not asking WHAT the rules are but a
JUSTIFICATION for them (which you have been incapable of providing so
far).
C.
น Apparently the "Please don't implement your own custom memory pool
or free list." does not count if it's implemented by others!
David Allsopp
> I apologize beforehand if this is not the forum to ask.
>
> I am on the fence about whether to learn OCaml or not, and while reading
> an article called "Why OCaml"
> (http://www.cs.ubc.ca/~murphyk/Software/Ocaml/why_ocaml.html), I saw that
> OCaml was praised for the speed of the executables it generates - and was
> referred to, speed-wise, as "second to none", except C and
> C++.
Do remember that speed isn't always everything - there are many applications where it's just not critical. Maintainability, readability, ease-of-use, availability of libraries are large concerns too.
> However, when I actually went to the Language Shootout page suggested in
> the article, I found out that OCaml is not 2nd, it is 13th, behind
> languages like Haskell and C#...
> (http://shootout.alioth.debian.org/u32/which-programming-languages-are-
> fastest.php)
As the kinda large number of messages in this thread suggests, benchmarks can be very subjective :o)
The Wiktionary definition of "to benchmark" is "To measure the performance of (an item) relative to another similar item in an impartial scientific manner". IMHO the problem benchmarking "programming languages" (note - "programming languages", not "problems solved in programming languages") lies in the word "similar" in that definition. Reducing an entire programming language's strengths (or weaknesses!) to a single number is just not really realistic - the truth is more complex than one single-precision floating point number (or even an array of them) can describe. (NB. The shootout doesn't claim that the final ranking displayed is anything other than a score of how well the languages did at the various benchmarks given - but a graph like that is easy to interpret erroneously in that way)
> Is it just hype, then? Or am I missing something?
OCaml's big strengths (by no means unique to OCaml, incidentally) for me are its ML legacy - automatic memory (de-)allocation, type inference, static type-checking (i.e. no BizarreRuntimeTypeException's here...) and polymorphism - all things which allow me to write less (and clearer) code to solve a given problem. I also find some of the more exotic features useful - private types and polymorphic variants, for example. I'm forced to program in a few less desirable languages from time to time and sorely miss the basic features of ML. I'd also echo Sylvain's point: I too find the time spent testing and debugging OCaml is much, much lower than, say, equivalent C(++) or Java code.
Personally, I've found the best way to find out if you like a new programming language is to write a non-trivial program in it (maybe with the occasional assistance of skilled people in its community) and then see what happens when you go back to programming in something you used before.
And if speed really is worrying you, the big thing you may have noticed from the, ahem, slightly animated discussion that's followed is that there are plenty of people on this list who know how to help you identify what's causing the problem and then offer suggestions to tweak it. Personally, in six years of programming in OCaml for a variety of tasks I've never hit the brick wall where I felt the task I was carrying out would have been orders of magnitude faster in another language *and also easy to code in that language*!
HTH,
David
Eray Ozkural
Christophe.Tr...@umh.ac.be<Christophe.Troestler%2Bo...@umh.ac.be>
> wrote:
> On Wed, 24 Nov 2010 06:50:15 +0000, Isaac Gouy wrote:
> >
> > Jeff Meister <nanaki <at> gmail.com> writes:
> >
> > > We know what your rules are for binary-trees; repeating them does
> > > not help.
> >
> > When Christophe TROESTLER wrongly states - "OCaml is not authorized
> > to make use of its very own library!" - he shows that those rules
> > are not known.
>
> Isolating a sentence gives a misleading idea of what I said. Must I
> really repeat that I was asking WHY is C allowed to use an external
> library to enhance its memory managementน but OCaml cannot use its own
> library to do that. I am not asking WHAT the rules are but a
> JUSTIFICATION for them (which you have been incapable of providing so
> far).
>
>
I didn't get it, either.
Best,
Cedric Cellier
higher level language than C to gain some expressive power but without
willing to sacrifice C's speed. I consulted a lot of benchmarks from
the shootout, and also many webpages similar to the one you refer to,
some praising a language some criticizing it.
The various contenders of the shootout seamed to fall in three categories :
languages that are fast compared to C, but does not reduce much the size
of the source code (C++, java, APS...), languages that allow concise
manipulations of non-trivial data structures (Lisps, MLs, Haskells), and
non-compiled languages that I never considered adopting.
I eliminated Haskell because GHC was only usable on x86 architecture at
that time, so I tried various Lisp at first. Then a friend pointed me to
Ocaml since, according to him, it's easier to cross the gap between C
and Ocaml than between C and anything else. I thus finally tried Ocaml
and, despite my initial apprehensions, I felt myself comfortable with
its syntax after am couple of days.
I am now using it whenever I have the choice to, after many years of C,
mixing with C or ASM when necessary (which is quite easy), because I
feel it's pleasant to use, and that I have still many things to learn
from it, without the bad feeling to waste CPU cycles along the way.
I like to think that if yon consider only speed and terseness of the
code, Ocaml is still amongst the top languages ; probably not the #1 as
stated by the (quite old) webpage you pointed to, but still excellent.
Vincent Aravantinos
Le 22 nov. 10 à 14:21, Thanassis Tsiodras a écrit :
> I apologize beforehand if this is not the forum to ask.
>
> (...)
>
> Is it just hype, then? Or am I missing something?
may we know, after all this intense discussion, what is your feeling?
Did this debate enlightened your views?
Did it make arise new questions?
Are you going to learn Ocaml?
At least you know now that Ocaml has a lively community :)
I just hope it did not disgust you...
Cheers,
V. Aravantinos
Thanassis Tsiodras
<vincent.a...@gmail.com> wrote:
> may we know, after all this intense discussion, what is your feeling?
Well... (ducks, wears helmet).
Dr Jon Harrop communicated with me directly (two days ago)... and when
I expressed my lack of faith after reading his "Rise and fall of
OCaml" article at
http://flyingfrogblog.blogspot.com/2010/08/rise-and-fall-of-ocaml.html
, he in fact counter-suggested that F# is now the horse to bet on -
so... what do you guys think?
Over the last couple of days, I've played a lot with ocaml (to be
exact, Linux/ocamlopt, since my interest in the speed of what I make
remains dominant) as well as F# (with Visual Studio 2008). To my
limited understanding, the differences between OCaml and F# are small
- and the benefits of direct access to the .NET ecosystem of libraries
seems to counter the ... uncertain status of OCaml libraries. I just
begun looking into all this, so I could be very wrong, of course - but
I am spoiled rotten with Python's libraries, so not having "batteries
included" in OCaml seemed like quite a problem... until I realized F#
completely covers this.
So, to conclude - what do you guys think about F# ?
>
> Did this debate enlightened your views?
> Did it make arise new questions?
> Are you going to learn Ocaml?
>
> At least you know now that Ocaml has a lively community :)
> I just hope it did not disgust you...
>
> Cheers,
> V. Aravantinos
--
What I gave, I have; what I spent, I had; what I kept, I lost. (Old Epitaph)
Oliver Bandel
Zitat von "Thanassis Tsiodras" <ttsi...@gmail.com>:
[...]
> Over the last couple of days, I've played a lot with ocaml (to be
> exact, Linux/ocamlopt, since my interest in the speed of what I make
> remains dominant)
[...]
OK, then I asume, you will be one of the programmers who
pick out the right datastructures for his applications.
When looking at C because of speed, this is completely ok.
But I would estimate that 80% or 90% of the programmers
just don#t think enough on that topic.
Many people who work as programmers, just pick a language
and a library.... just that library that is the most mainstream
library and start working.
Also many C-programmers I have seen have just done
seemingly simple hacking and not thought about data structures enough.
For simple programs and little amount of input data that's often
ok, but in the history of a program will turn out as bad decision.
So: what are all these benchmarks on different data structures worth,
if C-programmers always stick to arrays, or, maybe, to linked lists?
When I see how Gnome (and I would guess other desktop environments
will handle it thje same) handles areas of the screen.... that it does
not use trees that are appropriate for handling two-dimensional
spatial data, and then experience slow behaviour, then all is said....
I mean: OK, a low performant language will definitley make things
slower than a higher performant one - when usd with the same algorithms.
But when one does not use intelligent programming, this all does not
help much.
Then you can use C and be slower than, say, Python (which btw also
offers some nice datastructures, like Sets and so on).
But maybe what I write here is nothing new to you.
I just wanted to have it mentioned, that benchmarks alone does not
guarantee fast running programs.
A langauge that supports you to easily implement the datastructures you
would like to use, is much better supporting you then in this respect,
because if it needs toomuch effort to implement such a datastructure,
lazyness of the programmer might be stronger than the will to use
beter data structures.... especially if you have to fight with
dangling pointers on the one hand, where you on the other hand will be
warned in advance, that you just didn't got the types right and the
program will not compile then... which yields in fixing errors at
compilation stage.
Ciao,
Oliver
Vincent Aravantinos
Le 24 nov. 10 à 16:30, Thanassis Tsiodras a écrit :
> On Wed, Nov 24, 2010 at 4:34 PM, Vincent Aravantinos
> <vincent.a...@gmail.com> wrote:
>> may we know, after all this intense discussion, what is your feeling?
>
> Well... (ducks, wears helmet).
>
> Dr Jon Harrop communicated with me directly (two days ago)... and when
> I expressed my lack of faith after reading his "Rise and fall of
> OCaml" article at
> http://flyingfrogblog.blogspot.com/2010/08/rise-and-fall-of-ocaml.html
> , he in fact counter-suggested that F# is now the horse to bet on -
> so... what do you guys think?
>
> Over the last couple of days, I've played a lot with ocaml (to be
> exact, Linux/ocamlopt, since my interest in the speed of what I make
> remains dominant) as well as F# (with Visual Studio 2008). To my
> limited understanding, the differences between OCaml and F# are small
> - and the benefits of direct access to the .NET ecosystem of libraries
> seems to counter the ... uncertain status of OCaml libraries. I just
> begun looking into all this, so I could be very wrong, of course - but
> I am spoiled rotten with Python's libraries, so not having "batteries
> included" in OCaml seemed like quite a problem... until I realized F#
> completely covers this.
>
> So, to conclude - what do you guys think about F# ?
I personally do not know much of F#. Particularly because I do not
have a Windows machine :( We all know here that Jon is very fond of F#
after having been fond of Ocaml for a while.
The socalled "Ocaml mass exodus" mentionned in Jon's article seems to
me as as objective as the language shootout benchmarks ;) In
particular the plot I made myself of the posts to the mailing list is
definitely not as clear as the one presented in the article (you can
do it yourself from the figures at <http://caml.inria.fr/pub/ml-archives/caml-list/index.fr.html
>). Jon also mentions (in the comments) the figures from Google
Trends about Ocaml vs F# (<http://www.google.co.uk/trends?q=f
%23%2Cocaml>). But funnily enough, you can see from the excerpts
selected by Google that F# deals as much with the language than
with... ahem, other stuff. Furthermore the plot was already increasing
before the release of F#, so is this increase really significative? My
whole point here is not that Jon is wrong or right: maybe he is right,
I just say that the supposed exodus does not seem significative to me.
Anyhow I do not "feel" it. I would even say that I feel the contrary
(and I am not the only one: see Paolo's comment in Jon's article).
Then Jon suggests in his article that that this is due to the
"inability of Ocaml's GC to [deal with parallelism]". Again, all of us
here know that Jon has been quite frustratred with this for a while
(and not only him, cf some very recent thread <http://caml.inria.fr/pub/ml-archives/caml-list/2010/11/642bb03b01227d825e71b0a46ae7e73f.fr.html
>). However there are also plenty of guys who are not bothered by
this (see the very same thread). Choose your camp. On one hand, if you
go to F# you won't have to choose. On the other hand, this does not
make F# necessarily faster than Ocaml (as Jon himself proved it: <http://flyingfrogblog.blogspot.com/2010/08/pure-f-now-only-2-slower-than-ocaml.html
>).
Of course if you listen to Jon you will be convincend that you should
choose F#. Many people here will probably react by saying that you
should choose Ocaml. I am personnally neutral here: in the end, I
would say that both languages are great (about F# I should say "look
great" since I never tried it). So whatever is your final choice I
guess you won't be disappointed.
Cheers,
Vincent
PS: BTW about the complain in Jon's article about the lack of a native
REPL in ocaml, I think this recent post also answers it very partly: <http://caml.inria.fr/pub/ml-archives/caml-list/2010/11/aef35ed6ad039ce80c5c66175e80fcc5.fr.html
>. Probably nothing compared, though, to the F# REPL (trusting Jon on
that).
Isaac Gouy
> Isolating a sentence gives a misleading idea of what I said.
gmane demands "Prune quoted stuff."
> Must I
> really repeat that I was asking WHY is C allowed to use an external
> library to enhance its memory management¹ but OCaml cannot use its own
> library to do that.
Do the OCaml binary-trees programs that are shown not use OCaml memory
management?
Do the C programs that are shown tune the external library memory management
they use?
> I am not asking WHAT the rules are but a JUSTIFICATION for them (which you
> have been incapable of providing so far).
I feel no need to provide a JUSTIFICATION to you for anything.
> ¹ Apparently the "Please don't implement your own custom memory pool
> or free list." does not count if it's implemented by others!
Many C and C++ programmmers rolled their own custom free list implementations as
part of their program.
Library memory pool and free list implementations have to be more generally
useful (and probably less efficient) than custom solutions created specifically
for this tiny task.
"Please don't implement your own custom memory pool or free list." means exactly
what it says.
Christophe Raffalli
Here is a test of gctweak.ml on the "now famous" binary-tree shootout bench ...
As you can see it is a 30% speed up which is not too bad, just adding
a file on the compilation command line !
I reattached the file, because I correct a few comments in it ... and a syntax error
that is only visible when not using camlp4 in ocaml-3.11.2 ( { get () with ... } is
valid with camlp4 and invalid without ???) ... Anyway, more work on Gctweak is still needed ...
raffalli@d45-lama:~/Caml$ ocamlopt -o binary_tree gctweak.ml binary_tree.ml ; time ./binary_tree 20
stretch tree of depth 21 check: -1
2097152 trees of depth 4 check: -2097152
524288 trees of depth 6 check: -524288
131072 trees of depth 8 check: -131072
32768 trees of depth 10 check: -32768
8192 trees of depth 12 check: -8192
2048 trees of depth 14 check: -2048
512 trees of depth 16 check: -512
128 trees of depth 18 check: -128
32 trees of depth 20 check: -32
long lived tree of depth 20 check: -1
real 0m19.212s
user 0m18.960s
sys 0m0.180s
raffalli@d45-lama:~/Caml$ ocamlopt -o binary_tree binary_tree.ml ; time ./binary_tree 20
stretch tree of depth 21 check: -1
2097152 trees of depth 4 check: -2097152
524288 trees of depth 6 check: -524288
131072 trees of depth 8 check: -131072
32768 trees of depth 10 check: -32768
8192 trees of depth 12 check: -8192
2048 trees of depth 14 check: -2048
512 trees of depth 16 check: -512
128 trees of depth 18 check: -128
32 trees of depth 20 check: -32
long lived tree of depth 20 check: -1
real 0m27.484s
user 0m27.270s
sys 0m0.110s
Here is the run with debug := 1 and you see that minor heap size is guessed at
524288, with almost no promoted word (model = 1 means no promoted word)
raffalli@d45-lama:~/Caml$ ocamlopt -o binary_tree gctweak.ml binary_tree.ml ; time ./binary_tree 20
MHS DOUBLED <- 65536 (model 3.996155)
MHS DOUBLED <- 131072 (model 3.000397)
stretch tree of depth 21 check: -1
MHS DOUBLED <- 262144 (model 2.495375)
MHS DOUBLED <- 524288 (model 1.027698)
2097152 trees of depth 4 check: -2097152
524288 trees of depth 6 check: -524288
131072 trees of depth 8 check: -131072
32768 trees of depth 10 check: -32768
8192 trees of depth 12 check: -8192
2048 trees of depth 14 check: -2048
512 trees of depth 16 check: -512
128 trees of depth 18 check: -128
32 trees of depth 20 check: -32
long lived tree of depth 20 check: -1
real 0m19.342s
user 0m19.100s
sys 0m0.170s
--
Christophe Raffalli
Universite de Savoie
Batiment Le Chablais, bureau 21
73376 Le Bourget-du-Lac Cedex
tel: (33) 4 79 75 81 03
fax: (33) 4 79 75 87 42
mail: Christoph...@univ-savoie.fr
www: http://www.lama.univ-savoie.fr/~RAFFALLI
---------------------------------------------
IMPORTANT: this mail is signed using PGP/MIME
At least Enigmail/Mozilla, mutt or evolution
can check this signature. The public key is
stored on www.keyserver.net
---------------------------------------------
Isaac Gouy
-snip-
> However, when I actually went to the Language Shootout page suggested
> in the article, I found out that OCaml is not 2nd, it is 13th, behind
> languages like Haskell and C#...
>
(http://shootout.alioth.debian.org/u32/which-programming-languages-are-fastest.php)
You've noted the rank of OCaml (2nd and then 13th)
Can the rank position of OCaml change without the performance of the fastest
programs or the OCaml programs changing?
Can rank position in some list indicate how much faster or slower OCaml may be
than the fastest language implementation?
Is the rank position of OCaml stated on that old Doug Bagley scorecard?
http://web.archive.org/web/20010602051335/www.bagley.org/~doug/shootout/craps.shtml
Is the rank position of OCaml stated on the current benchmarks game page you
linked-to?
Isaac Gouy
-snip-
> Reducing an entire programming language's strengths (or
> weaknesses!) to a single number is just not really realistic - the truth is
more complex than one
> single-precision floating point number (or even an array of them) can
describe. (NB. The shootout
> doesn't claim that the final ranking displayed is anything other than a score
of how well the languages did
> at the various benchmarks given - but a graph like that is easy to interpret
erroneously in that way)
-snip-
That summary page
> > (http://shootout.alioth.debian.org/u32/which-programming-languages-are-
> > fastest.php)
shows box plots (quartiles, outliers, median) of the normalized timing
measurements for each of the tasks, for each of the language implementations.
A graph like that shows some of those language implementations are very fast for
some benchmark programs and very slow for others.
To characterize that as "reducing an entire programming language's strengths (or
weaknesses!) to a single number" seems kind-of misleading.
Especially when the question that page answers is stated 3 times - "Which
programming language implementations have the fastest benchmark programs?"
Ed Keith
>
>
> > I am not asking WHAT the rules are but a JUSTIFICATION
> for them (which you
> > have been incapable of providing so far).
>
> I feel no need to provide a JUSTIFICATION to you for
> anything.
>
Am I to interpret this to mean that the rules are purely arbitrary and capricious with no though behind them?
I hope I am misinterpreting this statement, but can think of no other interpretation
-EdK
Ed Keith
e_...@yahoo.com
Blog: edkeith.blogspot.com
Isaac Gouy
> > > I am not asking WHAT the rules are but a JUSTIFICATION
> > for them (which you
> > > have been incapable of providing so far).
> >
> > I feel no need to provide a JUSTIFICATION to you for
> > anything.
> >
>
> Am I to interpret this to mean that the rules are purely arbitrary and
capricious with no though behind them?
Tendentious.
A great deal of thought has been given to all aspects of the benchmarks game.
> I hope I am misinterpreting this statement, but can think of no other
interpretation
Poverty of imagination.
David Rajchenbach-Teller
Time to stop this, please.
Thanks,
David
David Allsopp
> David Allsopp <dra-news <at> metastack.com> writes:
>
> -snip-
> > Reducing an entire programming language's strengths (or
> > weaknesses!) to a single number is just not really realistic - the
> > truth is more complex than one single-precision floating point
> > number (or even an array of them) can describe. (NB. The shootout
> > doesn't claim that the final ranking displayed is anything other than
> > a score of how well the languages did
> > at the various benchmarks given - but a graph like that is easy to
> > interpret erroneously in that way)
> -snip-
>
> That summary page
>
> > > (http://shootout.alioth.debian.org/u32/which-programming-languages-a
> > > re-
> > > fastest.php)
>
> shows box plots (quartiles, outliers, median) of the normalized timing
> measurements for each of the tasks, for each of the language
> implementations.
>
> A graph like that shows some of those language implementations are very
> fast for some benchmark programs and very slow for others.
I'm not sure I disputed that anywhere, in fact I think if anything implicitly agreed with it...
> To characterize that as "reducing an entire programming language's
> strengths (or weaknesses!) to a single number" seems kind-of misleading.
I'm not clear how you extracted that from what I said which was that reducing a programming language to a single number (or three numbers, if you want bars) is not a good summary of the *total* strengths of that programming language. I then went on to stress that the combined results from the shootout don't claim to do that, they simply show an interpretation of the combined results from the different benchmarks.
My attempt at a point was that ranking programming languages (that's just "programming languages" not "execution speed of programming languages") is a largely futile activity, because it's just too subjective and hard to quantify in a rigorous manner, and importantly that the shootout doesn't try to do that. The relevance and value of that point to Thanassis' original question is for him to decide...
> Especially when the question that page answers is stated 3 times - "Which
> programming language implementations have the fastest benchmark programs?"
Which part of my statement "The shootout doesn't claim that the final ranking displayed is anything other than a score of how well the languages did at the various benchmarks given" (prefixed with the Latin abbreviation for "Note well") caused you to need to write this?
David
Oliver Bandel
halting problem discovered practically ;)
Zitat von "David Rajchenbach-Teller" <David....@univ-orleans.fr>:
Goswin von Brederlow
> Am Dienstag, den 23.11.2010, 17:53 +0000 schrieb Isaac Gouy:
>> Christophe TROESTLER <Christophe.Troestler+ocaml <at> umh.ac.be> writes:
>>
>> >
>> > On Tue, 23 Nov 2010 02:03:48 +0000, Isaac Gouy wrote:
>> > >
>> > > > C version : 12.11 secs
>> > > > OCaml version : 47.22 secs
>> > > > OCaml version with GC parameters tuned ("interesting alternative"
>> > > section) : 12.67 secs
>> > >
>> > > And of course you know because that GC tuned OCaml program is shown
>> > > on the
>> > > benchmarks game website
>> > > http://shootout.alioth.debian.org/u32/program.php?test=binarytrees&
>> > > lang=ocaml&id=1
>> >
>> > Since you are here, please explain why C can use memory pools and vec
>> > tor instructions but tuning the GC of OCaml ― although it is part of
>> > the standard library ― is considered an “alternative”.
>>
>>
>> You seem to be suggesting that "tuning the GC" is considered "alternative" only
>> for OCaml programs.
>>
>> You seem to be suggesting that "tuning the GC" is considered "alternative" for
>> every task.
>>
>> Neither is true.
>>
>> You seem to be suggesting some kind of equivalence between vector instructions
>> and "tuning the GC".
>> You haven't said why they should be considered equivalent.
>>
>> Nor have you said why you think C should not be allowed to use memory pools.
>
> Quite easy: because you are comparing results that cannot be compared.
> The reader of this benchmark (binary trees) might have the impression
> that C is generally that fast - however, this would be no longer true if
> these binary trees were used as library in a bigger program where using
> memory pools is inappropriate, e.g. because the data managed by the
> binary trees has an unpredictable lifetime.
>
> I do not say that it is complete nonsense to do this comparison, but
> only that it is more specific than a reader would assume. The innocent
> reader expects a comparison of binary tree performance, not of methods
> of managing memory (and this is it finally). The true name of this test
> should be "manage_many_small_memory_cells_where_pools_are_allowed". (It
> would be actually interesting to compare various versions of this test
> with different memory management methods.)
>
> Gerd
So write an ocaml programm that uses and array or bigarray to pool its
memory. That is the same as C does.
MfG
Goswin
Isaac Gouy
> > > (NB. The shootout
> > > doesn't claim that the final ranking displayed is anything other than
> > > a score of how well the languages did
> > > at the various benchmarks given - but a graph like that is easy to
> > > interpret erroneously in that way)
> > -snip-
> Which part of my statement "The shootout doesn't claim that the final ranking
displayed is anything other
> than a score of how well the languages did at the various benchmarks given"
(prefixed with the Latin
> abbreviation for "Note well") caused you to need to write this?
The part you haven't included in your quotation ;-)
Jon Harrop
reduce the density of references in the heap. Structs and reified generics
can give huge performance improvements. Hash tables and complex arithmetic
are some examples where I often see F# running 5-20× faster than idiomatic
OCaml/Java code.
Cheers,
Jon.
> -----Original Message-----
> From: caml-lis...@yquem.inria.fr [mailto:caml-list-
> bou...@yquem.inria.fr] On Behalf Of Erik de Castro Lopo
> Sent: 24 November 2010 01:24
> To: caml...@inria.fr
> Subject: Re: [Caml-list] Is OCaml fast?
>
> Jon Harrop wrote:
>
> > Many of the things that can make OCaml and Java slow were inherited
> > from Lisp. They are, in effect, designs that still bear the burden
> > of dynamic typing despite being statically typed.
>
> I'm curious, what in your opinion are those things?
>
> Erik
> --
> ----------------------------------------------------------------------
> Erik de Castro Lopo
> http://www.mega-nerd.com/
Stefan Monnier
> no-GC-tuning rule is silly in the light of actual uses of garbage
> results unrealistic, and an unrealistic benchmark is misleading, or at
> best merely useless.
To the extent that this rule is the same for all languages and that most
languages on the shootout are also garbage collected, I think OCaml's
problem with this benchmark do point at a weakness of the current
GC code.
Of course, the shootout could be improved. E.g. maybe it could allow
extra submissions that break the rules, along with a description of
which rules were broken and how. Then there could be a "score according
to the rules", then a "score when all gloves are off", together with
some kind of "measure" of what was needed to go from one to the other.
This way people could maybe get a better feel for the languages's
performance and how (and how much) that performance can be affected.
Doesn't seem like an easy undertaking, tho.
Stefan
Isaac Gouy
-snip-
> Of course, the shootout could be improved.
Of course, anyone interested can do their own more extensive comparisons for a
couple of language implementations, for example -
http://blog.marketcetera.com/2007/03/08/java-and-the-computer-language-shootout/
> E.g. maybe it could allow
> extra submissions that break the rules, along with a description of
> which rules were broken and how.
-snip-
Page search "alternative" on
http://shootout.alioth.debian.org/u32q/performance.php?test=binarytrees
Fabrice Le Fessant
> Stefan wrote:
>> I think OCaml's problem with this benchmark do point at a weakness of the current
>> GC code.
What makes you think that ?
I have contributed to some of the solutions that you can find there (and
some other ones were rejected because caching results is not allowed too
;-) ), and the GC was _NEVER_ a problem !
(except for binary-trees, but for absurd reasons)
The main problem was that other languages have bigger standard
libraries, whereas OCaml has a very small one (just what is needed to
compile the compiler, actually). In many problems, you could benefit
from using a very simple shared-memory library (in mandelbrot, the ocaml
multicore solution has to copy the image in a socket between processes,
whereas it could just be in a shared memory segment), and in general,
many solutions could benefit from specialised data structures that are
provided in other languages by their standard libraries, and from some
system calls that are currently not in the Unix library.
Anyway, I decided to stop trying to improve the solutions, and work on
improving the compiler and its libraries instead. It might benefit to
ocaml ranking in the shootout, but more importantely, it will benefit to
everybody in the community also.
--Fabrice
Christophe Raffalli
> To the extent that this rule is the same for all languages and that most
> languages on the shootout are also garbage collected, I think OCaml's
> problem with this benchmark do point at a weakness of the current
> GC code.
default
value of the minor heap size is the correct value for reactive programs
where you want fast minor GC slice, because they interrupt the program ...
Maybe the Gc library could provide reasonable default settings for both
cases
accessible via a simple function GC.is_reactive : bool -> unit ... And
the shootout
would allow to call such a function ?
Clearly, all benchmarks which just measure speed and space should use a
larger
minor heap size (except if reactive communication between processes is
important in the
benchmark).
More importantly, there is little hope to discover at runtime that a
program is
a reactive one ... In other words there is no hope for a perfect GC for
all situations.
So it is a good thing that OCaml allows tuning the GC.
Moreover, even when you automatically tune the GC (like changing the
number of generations in some modern GC) you do it based on past
observation of the behavior of your program ... and the behavior of the
program can change just after you changed some GC parameters ... So
begin able to help the GC, in a comprehensive way,
is good ...
Cheers,
Christophe
Stefan Monnier
>>> of the current GC code.
> What makes you think that ?
I don't really understand the question: it was just stating the obvious.
OCaml's GC (including its default settings) is generally very good, but
like all GCs it has its weaknesses. This is fundamentally unavoidable.
There should be no shame admitting that this particular case hits one of
those weak spots.
Pierre Etchemaïté
> Not sure what the max should be for the minor heap increase, but based
> on this benchmark increasing size of minor heap never slows down the
> program. Even when size of minor heap exceeds what fits in the cache.
> I guess there is another microbenchmark that can show the opposite
> though.
Another consequence of increasing minor heap size is the latency
introduced by minor heap collections. That said, when does it become
the worse garbage collector latency? (... when heap compaction is
disabled?)
And of course, for many programs such latency matters a lot less than
thoughput...
Christophe Raffalli
if promoted (that if visited twice by the minor GC) at least once by the
major GC (when freed because
major GC is mark and sweep) ... So a heap allocated block is traversed
0, 1, 3 or more time by one of the GC ...
So increasing the minor heap size strictly decreases the number of time
a word is examined by one of the GC ...
I think that having two heaps is mainly (only ?) for one reason:
collecting some memory sooner (and therefore using less memory for dead
objects).
This may speed up the program because saving memory decreases cache
misses, but normaly it slows down the
program. But it is still better than one generation GCed more frequently ...
The latency is not directly related to minor collection ... If there was
only the major heap,
OCaml could still to a bit of major-gc cycle at each collection ... This
is a (good) design choice of OCaml
to have a small minor heap that can be collected in a short time by a
simple and fast "stop and copy"
algorithm.
The main problem of the bin-tree benchmark is when you know that an
object will live long enough to be promoted, you can not ask OCaml to
allocate it in the major heap directly. This is why increasing the minor
heap to have all object collected at the first GC works.
A syntax like Node#(x,d,y) to allocate a variant in the major heap
directly together with similar syntax for
tuples, records, ... could be quite useful in OCaml in general for known
long living objects ... An the is the Ancient modules for object that
will live even longer.
Cheers,
Christophe
Benedikt Meurer
Benedikt
[1] http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.144.3640
oli...@first.in-berlin.de
[...]
> The main problem was that other languages have bigger standard
> libraries, whereas OCaml has a very small one (just what is needed
> to compile the compiler, actually). In many problems, you could
> benefit from using a very simple shared-memory library (in
> mandelbrot, the ocaml multicore solution has to copy the image in a
> socket between processes, whereas it could just be in a shared
> memory segment),
..so you work on a shared-mem module?!
> and in general, many solutions could benefit from
> specialised data structures that are provided in other languages by
> their standard libraries, and from some system calls that are
> currently not in the Unix library.
[...]
During the last some releases a lot more unix syscalls were added
and that's fine of course).
Which calls are you missing there?
Ciao,
Oliver
oli...@first.in-berlin.de
> Hello,
> > To the extent that this rule is the same for all languages and that most
> > languages on the shootout are also garbage collected, I think OCaml's
> > problem with this benchmark do point at a weakness of the current
> > GC code.
> This is untrue ... the bintree example, is just bad in OCaml because the
> default
> value of the minor heap size is the correct value for reactive programs
> where you want fast minor GC slice, because they interrupt the program ...
And if your program contains both kinds of functionality?
What possible solution would you recommend?
Ciao,
Oliver
Eray Ozkural
benedik...@googlemail.com> wrote:
> Speaking of the OCaml GC in general, wouldn't it make sense to replace the
> current generational collector with a collector framework that requires less
> copying in the common case. For example, dividing the heap into two parts,
> "large object space" and "small object space", where LOS is managed by
> mark-sweep/mark-compact (could use the current major heap) and SOS is
> managed by a recent mark-copy/mark-region collector (Immix [1] comes to mind
> here). That way objects would no longer need to be copied between
> generations, and using Immix may also reduce cache misses and improve
> locality of small/medium sized objects (with proper evacuation heuristics).
> This should be doable with a moderate amount of work, and should require no
> incompatible changes, while opening the door for concurrent/parallel garbage
> collection (this would require incompatible changes then, replacing
> caml_young_ptr/caml_young_limit with TLS vars, etc.).
>
> Benedikt
>
> [1] http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.144.3640
>
I was suggesting dealing with small fixed-size objects differently in
another post. This doesn't sound like a bad combination, nice idea :)
Best,
--
Eray Ozkural, PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara
http://groups.yahoo.com/group/ai-philosophy
http://myspace.com/arizanesil http://myspace.com/malfunct
Jon Harrop
a long-lived structure was single large block but, in this case, the
long-lived structure is a tree composed of many small heap-allocated blocks
and, therefore, they would undergo the same wasteful "allocate in young only
to be copied to old" pathological behaviour?
Surely what you really want the ability to spot when a full minor heap
contains mostly live objects and, therefore, make the whole minor heap part
of the major heap, allocate yourself a new minor heap and clear the
remembered sets. IIRC, G1 does something like this.
On a related note, When designing HLVM I thought that a mallocs function
that allocated many small blocks of the same size such that they could be
freed individually would be helpful. Another option might be to preallocate
larger numbers of blocks at the same time under the assumption that a thread
allocating many small surviving blocks would continue to do so, as a kind of
pool hybrid.
Cheers,
Jon.
Benedikt Meurer
On Nov 28, 2010, at 20:40 , Jon Harrop wrote:
> I don’t understand why this would help here though. Wouldn’t that help when a long-lived structure was single large block but, in this case, the long-lived structure is a tree composed of many small heap-allocated blocks and, therefore, they would undergo the same wasteful “allocate in young only to be copied to old” pathological behaviour?
There is no "young" and no "old" in this scheme. There are two different heaps, one for large objects, one for small (probably 2-8k max object size, so not really small in the sense of OCaml). The small area is managed by Immix, which avoids copying of long-lived objects if they are allocated together with other long-lived objects (likely), or if not evacuates a set of probably related objects to a single chunk (this is somewhat dependent on the evacuation strategy, which will be differnt for OCaml compared to Java), further improving locality. There are simple heuristics to ensure that an object is not evacuated too often (it is already unlikely with the base algorithm), so there will be a lot less copying. One difficulty remains however: the pause times. It would probably be necessary to adopt Immix to a (semi-)incremental style to get the same pause times as with the current GC.
Benedikt
Jon Harrop
high allocation rates of short-lived objects? Been a while since I read the
Immix paper...
Cheers,
Jon.
> -----Original Message-----
> From: caml-lis...@yquem.inria.fr [mailto:caml-list-
> bou...@yquem.inria.fr] On Behalf Of Benedikt Meurer
> Sent: 28 November 2010 20:00
> To: caml...@yquem.inria.fr
> Subject: Re: [Caml-list] OCaml GC [was Is OCaml fast?]
>
>
Christophe Raffalli
> On Sat, Nov 27, 2010 at 04:58:55PM +0100, Christophe Raffalli wrote:
>> Hello,
>>> To the extent that this rule is the same for all languages and that most
>>> languages on the shootout are also garbage collected, I think OCaml's
>>> problem with this benchmark do point at a weakness of the current
>>> GC code.
>> This is untrue ... the bintree example, is just bad in OCaml because the
>> default
>> value of the minor heap size is the correct value for reactive programs
>> where you want fast minor GC slice, because they interrupt the program ...
> [...]
>
> And if your program contains both kinds of functionality?
> What possible solution would you recommend?
Changing the value of the minor heap size at runtime ...
There is no pb with this ...
>
>
> Ciao,
> Oliver
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
--
Jon Harrop
Cheers,
Jon.
> -----Original Message-----
> From: caml-lis...@yquem.inria.fr [mailto:caml-list-
oli...@first.in-berlin.de
Benedikt Meurer
On Nov 29, 2010, at 00:34 , Jon Harrop wrote:
> I see. Yes, that sounds like a great idea. How well does Immix cope with
> high allocation rates of short-lived objects? Been a while since I read the
> Immix paper...
In theory this should be handled more efficiently compared to a generational collector (given an appropriate evacuation heuristic), since there is less copying and short-lived objects tend to reuse the "hot" memory of objects that died previously (similar to the minor heap now), while survivers benefit from better compaction (without the need for a seperate mark-compact phase as implemented in the current major GC).
The timings for Java are impressive, but the allocation rates in Java tend to be different from those typically found in functional programs. There was also a GSoC project to implement Immix for GHC [1], but they seem to have used Immix in a generational setting, with mixed results (my knowledge of the GHC GC is limited).
> Cheers,
> Jon.
greets,
Benedikt
[1] http://hackage.haskell.org/trac/ghc/wiki/Commentary/Rts/Storage/GC/Immix