[Caml-list] true parallelism / threads

[Atmam Ta]

Hi,

I am trying to evaluate ocaml for a project involving large scale numerical
calculations. We would need parallel processing, i.e. a library that
distributes jobs accross multiple processors within a machine and accross
multiple PCs.
Speed and easy programability are important. I have tried to search this
issue first, but the postings I found were usually negative and 4-5 years
old. On the other hand, I see a number of libraries in the Hump that by now
might be taking care of these things.

My question is: is ocaml good for parallel processing / hreaded computation,
are there (mature) libraries or tools that let developers make use of
multicore and multimachine environments?

cheers,
Atmam

[Yoann Padioleau]

Atmam Ta <atma...@gmail.com> writes:

MPI ...
http://pauillac.inria.fr/~xleroy/software.html#ocamlmpi

>

Then it's quite easy to define your own helpers on top of that.
Here is for example my poor's man google map-reduce in ocaml:
http://aryx.kicks-ass.org/~pad/darcs/commons/distribution.ml

> cheers,
> Atmam
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs

[Hezekiah M. Carty]

2009/2/20 Atmam Ta <atma...@gmail.com>:

There are several libraries available which seem to be reasonably
usable in their current state.
Distributed processing across multiple machines:
- OCAMLMPI - http://pauillac.inria.fr/~xleroy/software.html
- OCamlP3l - http://camlp3l.inria.fr/eng.htm
- BSML - http://frederic.loulergue.eu/research/bsmllib/bsml-0.4beta.html

Fork-based parallelism for exploiting multiple cores/processors locally:
- Prelude.ml - http://github.com/kig/preludeml/tree/master

There is also JoCaml (http://jocaml.inria.fr/), which is an extension
of OCaml itself. JoCaml has examples for various distributed
processing methods.

Hez

--
Hezekiah M. Carty
Graduate Research Assistant
University of Maryland
Department of Atmospheric and Oceanic Science

[Will M. Farr]

Atmam,

I've had some luck using OCaml with MPI (using the OCamlMPI library at
http://caml.inria.fr/cgi-bin/hump.en.cgi?contrib=401 ). That may not
satisfy your needs as far as multi-core goes, but perhaps it will. I
can't speak to the speed of the interface (my operations were
compute-bound on the individual processors, not communication bound,
so any OCaml overhead on the MPI communication was lost in the noise),
but it was definitely easy to use. At the extreme easy-to-use end,
you can simply send arbitrary OCaml values over the MPI channels; for
more performance, you can use the functions specific to common types
(float arrays, int arrays, etc) to speed up the operations.

As far as single-core OCaml speed goes, I find that it is always
within a factor of 2 of C for straight-line loops (i.e. matrix-vector
multiply, etc), and usually *much* faster whenever more complicated
data structures are involved (maps, binary trees, etc), unless you
really sweat blood with the C implementation.

Hope this helps!

Will

2009/2/20 Atmam Ta <atma...@gmail.com>:

[Markus Mottl]

2009/2/20 Atmam Ta <atma...@gmail.com>:

> My question is: is ocaml good for parallel processing / hreaded computation,
> are there (mature) libraries or tools that let developers make use of
> multicore and multimachine environments?

For heavy-duty linear algebra you might want to use Lacaml:

http://ocaml.info/home/ocaml_sources.html#lacaml

It interfaces almost all functions in BLAS and LAPACK and allows
executing multiple computations in several threads in parallel on
multi-core machines. If you combine this with some tool for
distributed computation (e.g. MPI-based, etc.), you should get what
you need.

Regards,
Markus

--
Markus Mottl http://www.ocaml.info markus...@gmail.com

[Gerd Stolpmann]

ocamlnet contains a mature sunrpc implementation, and a framework for
multi-processing. It is used in professional cluster environments, e.g.
by the Wink people searcher.

See here for a commented example:
http://blog.camlcity.org/blog/parallelmm.html

Gerd
--
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany
ge...@gerd-stolpmann.de http://www.gerd-stolpmann.de
Phone: +49-6151-153855 Fax: +49-6151-997714
------------------------------------------------------------