[Caml-list] Native dynlink on 3.11: a request for packagers
Dario Teixeira
As you are probably aware, the upcoming 3.11 release includes
Alain Frisch's native dynlink patches. AFAIK, you cannot
directly dynlink a cmxa or cmx file like you would do in
bytecode with cma or cmo files. Beforehand, a native code
binary must be made into a plugin via ocamlopt's -shared
option. The suggested convention [1] is to give these
plugins a cmxs extension:
ocamlopt -shared -linkall -o foobar.cmxs foobar.cmxa
These native code plugins are a prerequisite if one intends
to run Ocsigen in native mode. Therefore, the life of Ocsigen
users would be greatly simplified if library writers and
packagers (GODI, Debian, Fedora, etc) would add this little
extra step of generating cmxs for each cmxa. This request
only makes sense for 3.11, of course.
Complementary, META files should also include extra directives
for plugins. For example:
archive(byte) = "foobar.cma"
archive(native) = "foobar.cmxa"
archive(plugin,byte) = "foobar.cma"
archive(plugin,native) = "foobar.cmxs"
This request can of course become an OSR. My question is if the
herein contained instructions will still be valid verbatim for
3.11 final. Will they? Also, please check the Ocsigen Wiki for
more info on this subject [2].
Best regards,
Dario Teixeira
[1] http://alain.frisch.fr/natdynlink.html
[2] http://www.ocsigen.org/trac/wiki/NativeCodeVersion
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Richard Jones
compiling OCaml, I've certainly come to appreciate findlib more than
ever.
Rich.
--
Richard Jones
Red Hat
Dario Teixeira
> Does ocamlfind support any of this? Since I'm now also
> cross-compiling OCaml, I've certainly come to appreciate
> findlib more than ever.
Indeed it does. The current version of Ocsigen already
makes use of this feature. However, because very few
(none, actually, other than Ocsigen's) packages currently
ship with cmxs files, an Ocsigen user who wants to play
with native code is forced to manually generate them.
(What impelled me to write the request was precisely
having just gone through the process of producing the
cmxs for a bunch of Ocamlnet and PXP libraries...)
Cheers,
Dario
Jon Harrop
> Hi,
>
> > Does ocamlfind support any of this? Since I'm now also
> > cross-compiling OCaml, I've certainly come to appreciate
> > findlib more than ever.
>
> Indeed it does. The current version of Ocsigen already
> makes use of this feature. However, because very few
> (none, actually, other than Ocsigen's) packages currently
> ship with cmxs files, an Ocsigen user who wants to play
> with native code is forced to manually generate them.
> (What impelled me to write the request was precisely
> having just gone through the process of producing the
> cmxs for a bunch of Ocamlnet and PXP libraries...)
This begs the question: what is the advantage of a .cmxa over a .cmxs, and
what are the downsides of linking everything dynamically?
--
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e
David Allsopp
> On Friday 21 November 2008 23:45:34 Dario Teixeira wrote:
> > Hi,
> >
> > > Does ocamlfind support any of this? Since I'm now also
> > > cross-compiling OCaml, I've certainly come to appreciate
> > > findlib more than ever.
> >
> > Indeed it does. The current version of Ocsigen already
> > makes use of this feature. However, because very few
> > (none, actually, other than Ocsigen's) packages currently
> > ship with cmxs files, an Ocsigen user who wants to play
> > with native code is forced to manually generate them.
> > (What impelled me to write the request was precisely
> > having just gone through the process of producing the
> > cmxs for a bunch of Ocamlnet and PXP libraries...)
>
> This begs the question: what is the advantage of a .cmxa over a .cmxs,
> and
> what are the downsides of linking everything dynamically?
That's reducible to the standard debate of static vs dynamic linking in any language, surely? The only "impact" on performance presumably is the re-writing of the symbol tables which flexlink/dlopen does as the .cmxs file is loaded but that must be negligible. However, isn't ocamlopt able to do some cross-module optimisation (which is why .cmx files are usually shipped as well as .cmxa) - this presumably isn't possible when linking dynamically...
Short-term, native dynlink doesn't work on all platforms (AFAIK?) so .cmxa is not dead, whatever your linking preference.
David
Stefano Zacchiroli
> Therefore, the life of Ocsigen users would be greatly simplified if
> library writers and packagers (GODI, Debian, Fedora, etc) would add
> this little extra step of generating cmxs for each cmxa. This
> request only makes sense for 3.11, of course.
Debian-side, this is already planned. The work-flow will be as usual:
patch the Makefiles of upstream not (yet) using dynlinking, and send
patches upstream. Of course it will take some time, and library
authors helping out making new releases supporting that out of the box
will be really appreciated.
Cheers.
--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu à tous ceux que j'aime
Daniel Bünzli
> No facilities are provided to access value names defined by the
> unit. Therefore, the unit must register itself its entry points with
> the main program, e.g. by modifying tables of functions.
Thus just generating a cmxs for a module won't allow you to use it
dynamically. You'll need support from the module no ?
Daniel
Gerd Stolpmann
Am Freitag, den 21.11.2008, 15:45 -0800 schrieb Dario Teixeira:
> Hi,
>
> > Does ocamlfind support any of this? Since I'm now also
> > cross-compiling OCaml, I've certainly come to appreciate
> > findlib more than ever.
>
> Indeed it does. The current version of Ocsigen already
> makes use of this feature.
Ahem. I'm not aware of this feature... Can you please elaborate to what
you refer?
Gerd
> However, because very few
> (none, actually, other than Ocsigen's) packages currently
> ship with cmxs files, an Ocsigen user who wants to play
> with native code is forced to manually generate them.
> (What impelled me to write the request was precisely
> having just gone through the process of producing the
> cmxs for a bunch of Ocamlnet and PXP libraries...)
>
> Cheers,
> Dario
>
>
>
>
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
--
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany
ge...@gerd-stolpmann.de http://www.gerd-stolpmann.de
Phone: +49-6151-153855 Fax: +49-6151-997714
------------------------------------------------------------
Alain Frisch
> Thus just generating a cmxs for a module won't allow you to use it
> dynamically. You'll need support from the module no ?
Well, if your program is only composed of .cmxs plugins (libraries +
your own program), then you can have a generic driver that simply load a
list of .cmxs files specified e.g. on its command line.
The point is that a plugin can refer to symbols defined in other plugins
already dynlinked.
-- Alain
Dario Teixeira
> > Indeed it does. The current version of Ocsigen
> > already makes use of this feature.
>
> Ahem. I'm not aware of this feature... Can you please
> elaborate to what you refer?
AFAIK, you can annotate findlib's META files with custom
tags. Ocsigen expects dynamically loaded plugins to have
a "plugin" tag, like this:
archive(native) = "foobar.cmxa"
archive(plugin,native) = "foobar.cmxs"
Ocsigen's module loader can thus leverage all of Findlib's
existing facilities (such as dependency handling) also for
dynamically loading plugins.
(The Ocsigen guys can correct/expand on this better than
anyone, of course).
Daniel Bünzli
Le 21 nov. 08 à 20:29, Dario Teixeira a écrit :
> AFAIK, you cannot directly dynlink a cmxa or cmx file like you would
> do in
> bytecode with cma or cmo files. Beforehand, a native code
> binary must be made into a plugin via ocamlopt's -shared
> option.
To make it easy to do this for a module or a library would it be
possible to add a new default %.cmxs target in ocamlbuild before 3.11
is released (or is it already too late) ?
Best,
Daniel
P.S. Tell me if you prefer to have that in the bug tracker.
Dario Teixeira
> To make it easy to do this for a module or a library would
> it be possible to add a new default %.cmxs target in
> ocamlbuild before 3.11 is released (or is it already too
> late) ?
Good point. It shouldn't be too late, since the changes are
minimal and 3.11 is only at RC1 status...
Cheers,
Dario
Stefano Zacchiroli
> To make it easy to do this for a module or a library would it be
> possible to add a new default %.cmxs target in ocamlbuild before
> 3.11 is released (or is it already too late) ?
Good idea. ... even though I doubt in practice it would impact
significantly on the speed of %.cmxs spreading, as the projects using
ocamlbuild to build are not that many (yet).
> P.S. Tell me if you prefer to have that in the bug tracker.
It is probably better, even only to increase the visibility and
traceability of the request.
Cheers.
--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'č ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu ŕ tous ceux que j'aime