[Caml-list] fundata1 -- Karmic Social Capital Benchmark and Shootout

[Alexy Khrabrov]

I am happy to announce fundata1 -- the largest-ever program per RAM allocation in Haskell, originally implemented in Clojure and then OCaml and Haskell for social network modeling.

http://github.com/alexy/fundata1

It has now become the first large-scale social networking benchmark with a real dynamic social graph built from the actual Twitter gardenhose, with the data OK'd by Twitter and supplied along with the benchmark.

I wrote three reference implementations, all on github as well. Clojure and OCaml are quite basic, while Haskell community had a chance to optimize its data structures and in fact fix a GC integer overflow while working on it. You're welcome to fork and improve all of these implementations, and supply others!

There's a Google Group,

http://groups.google.com/group/fundata/

to discuss the shootout. There's also a blog about it and other functional things at

http://functional.tv/

Let the fun begin!

-- Alexy Khrabrov
firstname.lastnameATgmaildotcom

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs

[bluestorm]

I was mildly curious and directly went for performance results.
A few bits of information for those that dont want to explore the website
themselves :

- it's a really big data set; on the website you're advised to
export OCAMLRUNPARAM='h=5G;s=1G'

- the Haskell implementation was developped first, tuned, and now takes 17
minutes to run. The OCaml implementation is a simple port of the Haskell
implementation (with the data structures adapted), and it takes 15 minute to
run. A younger Clojure implementation is at 30 minutes for now.

My hasty conclusion : the OCaml GC and the Hasthbl implementation scale
well.