[Caml-list] Dynamic linking
Alain Frisch
I'm pleased to announce the existence of an experimental branch in
OCaml's CVS. Main changes:
* Dynamic loading of OCaml code in native code programs, through
the Dynlink module.
* Simplified dynamic linking of C code under Windows.
More info: http://alain.frisch.fr/natdynlink.html
Enjoy and send some feedback!
Alain
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Joel Reymont
On Jun 15, 2007, at 5:03 PM, Alain Frisch wrote:
> * Dynamic loading of OCaml code in native code programs, through
> the Dynlink module.
When will this make it into the main tree?
Thanks, Joel
--
http://topdog.cc - EasyLanguage to C# translator
http://wagerlabs.com - Blog
Brian Hurt
>Hello,
>
>I'm pleased to announce the existence of an experimental branch in
>OCaml's CVS. Main changes:
>
>* Dynamic loading of OCaml code in native code programs, through
> the Dynlink module.
>
>* Simplified dynamic linking of C code under Windows.
>
>
>More info: http://alain.frisch.fr/natdynlink.html
>
>Enjoy and send some feedback!
>
>Alain
>
>
>
A) Thanks, this is usefull.
B) How extensive were the changes to the Ocaml source code? How likely
is it that this will get included in some future version of the main trunk?
Brian
Alain Frisch
> B) How extensive were the changes to the Ocaml source code?
For Linux x86, the change is rather small and straightforward.
For Windows ports, it is more tricky, because a Windows DLL cannot
reference symbols from the main program. I wrote the FlexDLL tool to
simulate a POSIX-ish dlopen API, so that the patch to OCaml itself
mostly reduces to removing code or sharing Makefiles with Unix. All the
trouble is encapsulated into FlexDLL (which is itself a quite small
piece of code, but a tricky one).
>From Linux AMD64, the ABI/dynamic linker does not allow to dynamically
load position dependant code, so I had to modify the last stage of
ocamlopt's backend in order to produce PIC code (when the option -dlcode
is given; note that this is independant from the existing -fpic option,
details on demand). Again, this is a rather small and unintrusive change.
You can use CVS to get a diff between the ocaml3100 tag and the
natdynlink branch (but it won't show you new files; you can always "cvs
checkout" both and do a local diff).
> How likely
> is it that this will get included in some future version of the main trunk?
It is likely to be included in OCaml 3.11, not before. (Except for the
native toplevel which will probably not be included.) Of course,
feedback can only help for the integration.
-- Alain
Jon Harrop
This is incredible. Thank you so much!
On Friday 15 June 2007 19:24:20 Alain Frisch wrote:
> ... (Except for the native toplevel which will probably not be included.)
Is that "native" as in we're going to get an OCaml top-level with native-code
performance? If so, this is hugely important for me...
--
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
OCaml for Scientists
http://www.ffconsultancy.com/products/ocaml_for_scientists/?e
Alain Frisch
> On Friday 15 June 2007 19:24:20 Alain Frisch wrote:
>> ... (Except for the native toplevel which will probably not be included.)
>
> Is that "native" as in we're going to get an OCaml top-level with native-code
> performance? If so, this is hugely important for me...
What kind of application do you have in mind?
It's "native" as in "we have an OCaml top-level with native-code
performance and a large latency". Each phrase is compiled with
ocamlopt's backend to an assembler file, which is then assembled, linked
into a shared library, dl-opened and then run. (Shared libraries will
never be closed so your OS must be strong enough to support that.)
Performances will likely be equal to ocamlopt's generated code (in
particular, we have cross-phrase optimizations).
Feel free to play with it ("make ocamlnat") and tell us whether this is
adequate for what you want to do.
Alain
skaller
> Hello,
>
> I'm pleased to announce the existence of an experimental branch in
> OCaml's CVS. Main changes:
>
> * Dynamic loading of OCaml code in native code programs, through
> the Dynlink module.
>
> * Simplified dynamic linking of C code under Windows.
>
>
> More info: http://alain.frisch.fr/natdynlink.html
>
> Enjoy and send some feedback!
A question, probably not directed at Alain: will there be support
for AMD64/64 bit Ocaml for Windows XP64/VC2005 +/- Vista?
Windows ports are all 32 bit at the moment aren't they?
Also a question: how does the Windows native (MSVC based) Ocaml
code find C libraries? The VC2003 style DLL-hell approach is now
deprecated on Windows platforms: you have to use Assemblies,
that is, ship applications with manifests and related components
which are installed in system caches to get around DLL-hell.
The resulting system is horrible to prepare for vendors I think,
I haven't figured it out, but it does support multi-version and
multi-arch, and seems more sophisticated the Linux versioning
technology.
The bottom line, however, is you simply cannot link to a DLL
by just plonking it in the PATH anymore.
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
Quôc Peyrot
On Jun 15, 2007, at 8:24 PM, Alain Frisch wrote:
> Brian Hurt wrote:
>> B) How extensive were the changes to the Ocaml source code?
>
> For Linux x86, the change is rather small and straightforward.
Is it working (or going to work) on PPC (Mac OS X)?
--
Best Regards,
Quôc
Christophe TROESTLER
>
> A question, probably not directed at Alain: will there be support
> for AMD64/64 bit Ocaml for Windows XP64/VC2005 +/- Vista?
> Windows ports are all 32 bit at the moment aren't they?
Extract from README.win32 :
Native MS Native MinGW Cygwin
64 bits? Win32 or Win64 Win32 only Win32 only
Third-party software required
- for base bytecode system none none none
- for ocamlc -custom MSVC Cygwin Cygwin
- for native-code generation MSVC+MASM Cygwin Cygwin
Speed of bytecode interpreter 70% 100% 100%
Replay debugger no no yes
The Unix library partial partial full
The Threads library yes yes yes
The Graphics library yes yes no
Restrictions on generated executables? none none yes (*)
ChriS
skaller
> On Sat, 16 Jun 2007, skaller <ska...@users.sourceforge.net> wrote:
> >
> > A question, probably not directed at Alain: will there be support
> > for AMD64/64 bit Ocaml for Windows XP64/VC2005 +/- Vista?
> > Windows ports are all 32 bit at the moment aren't they?
>
> Extract from README.win32 :
>
> Native MS Native MinGW Cygwin
>
> 64 bits? Win32 or Win64 Win32 only Win32 only
Ooops, OK! Sweet!
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
_______________________________________________
Alain Frisch
>> For Linux x86, the change is rather small and straightforward.
>
> Is it working (or going to work) on PPC (Mac OS X)?
It is not currently working. The code generator must be adapted to
produce PIC code. I've started to do that but it doesn't work and I
won't be able to finish it. If someone is interested to help, please
tell me.
-- Alain
Jon Harrop
The native top-level is the bit that interests me the most, as this combines
interactivity with OCaml's superb native-code performance.
I just benchmarked the new top-level and it runs my ray tracer 17x faster:
$ time ocaml unix.cma ray.ml >image.pgm
real 1m11.656s
user 2m21.106s
sys 0m0.331s
$ time ocamlnat unix.cmxa ray.ml >image.pgm
real 0m4.294s
user 0m6.265s
sys 0m0.521s
Interactive compilation to native code was one of the main benefits of F# over
OCaml. So I, for one, would dearly love to see this in the mainstream OCaml!
--
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
OCaml for Scientists
http://www.ffconsultancy.com/products/ocaml_for_scientists/?e
_______________________________________________
skaller
> On Friday 15 June 2007 19:24:20 Alain Frisch wrote:
> > Except for the native toplevel which will probably not be included.
>
> The native top-level is the bit that interests me the most, as this combines
> interactivity with OCaml's superb native-code performance.
I do not understand how the toplevel is even remotely useful:
it isn't "interactive" in any real sense of the word.
Natively you can't even edit lines, it's very lame.
Even with ledit, what you can do is very limited.
I fail to see how this is any better than preparing
a file and compiling it -- that way you get a proper
editor. If you're using emacs or some other souped up
editor it works much like an interactive IDE anyhow,
only better..
So exactly what is the advantage of the top level?
Yes, I use it occasionally for testing 1-2 line
fragments, mainly the typing, but i just can't
envisage how a ray tracer could be use productively
in such an environment, as compared to just making
a suitable file and compiling and running it.
It IS a bit of a pain, setting up the build environment
to do that, but I can't imagine it's that hard to automate
with something like emacs, or one of the make systems.
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
_______________________________________________
Christophe TROESTLER
>
> So exactly what is the advantage of the top level?
>
> Yes, I use it occasionally for testing 1-2 line
> fragments, mainly the typing, but i just can't
> envisage how a ray tracer could be use productively
> in such an environment, as compared to just making
> a suitable file and compiling and running it.
I generally also type code in a file and use C-x C-e if I need to get
an immediate feedback. However, native performance can be useful in
order (say) to perform interactive data analysis and use pure ocaml
libraries (as opposed to C wrappers). Data analysis requires some
kind of fast feeback because you generally only discover what you want
to do during the analysis itself...
ChriS
Philippe Wang
> On Sat, 2007-06-16 at 18:33 +0100, Jon Harrop wrote:
>
>> On Friday 15 June 2007 19:24:20 Alain Frisch wrote:
>>
>>> Except for the native toplevel which will probably not be included.
>>>
>> The native top-level is the bit that interests me the most, as this combines
>> interactivity with OCaml's superb native-code performance.
>>
>
> I do not understand how the toplevel is even remotely useful:
> it isn't "interactive" in any real sense of the word.
>
A native toplevel can be (very) useful when using ocaml as a scripting
language.
I like to write (web) scripts with ocaml when I don't want to spend a
lot of time on searching for trivial security holes (i.e. code
injection) with languages such as bash or perl.
And compiling a file and using the binaries is a risk of loosing the
source...
I know, this may sound weird...
--
Philippe Wang
Alain Frisch
It is now possible to try the natdynlink branch with GODI:
- check that you are on section 3.10 (GODI_SECTION = 3.10 in your
godi.conf);
- "update the list of available packages"
- in the configuration screen for godi-ocaml-src, set these variables:
OCAML_CVS_CHECKOUT = yes
OCAML_CVS_REVISION = natdynlink
- ask GODI to rebuild the two packages godi-ocaml-src and godi-ocaml.
Jon Harrop
> Enjoy and send some feedback!
I love the native-code top-level!
Any chance you could get MetaOCaml running on AMD64?
--
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
The OCaml Journal
http://www.ffconsultancy.com/products/ocaml_journal/?e