[Caml-list] Native dynlink and reloading modules

[Richard W.M. Jones]

I'm a bit surprised to find that native dynlink doesn't work in the
same way as bytecode dynlink in respect to reloading the same module.
(See attached test program)

In bytecode dynlink, reloading (ie. Dynlink.loadfile) the same module
causes the new module code to override the old module code:

$ ./dynlink_test
testing bytecode ...
a
b

But in native dynlink, the new module is silently discarded and the
old code appears to run:

$ ./dynlink_test
testing native ...
a
a

I would classify this as a bug, but I'm not quite sure what is
expected to happen. Is there some other way to override a module as
in bytecode?

$ ocaml -version
The Objective Caml toplevel, version 3.12.1

Rich.

--
Richard Jones
Red Hat

--
Caml-list mailing list. Subscription management and archives:
https://sympa-roc.inria.fr/wws/info/caml-list
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs

[Alain Frisch]

On 03/22/2012 12:47 PM, Richard W.M. Jones wrote:
>
> I'm a bit surprised to find that native dynlink doesn't work in the
> same way as bytecode dynlink in respect to reloading the same module.
> (See attached test program)
>
> In bytecode dynlink, reloading (ie. Dynlink.loadfile) the same module
> causes the new module code to override the old module code:
>
> $ ./dynlink_test
> testing bytecode ...
> a
> b
>
> But in native dynlink, the new module is silently discarded and the
> old code appears to run:
>
> $ ./dynlink_test
> testing native ...
> a
> a
>
> I would classify this as a bug, but I'm not quite sure what is
> expected to happen. Is there some other way to override a module as
> in bytecode?

natdynlink currenlty depends on the OS dynamic linker, we cannot control
the semantics so precisely (you can try loadfile_private, but I'm not
sure it would solve your issue). Note that doing Dynlink.loadfile
several times on the same module name can break the soundness of the
type system (if you change the implementation while keeping the same
interface):

http://caml.inria.fr/mantis/view.php?id=4229


So it's probably a bad idea anyway.

-- Alain

[Richard W.M. Jones]

On Thu, Mar 22, 2012 at 01:51:57PM +0100, Alain Frisch wrote:
> On 03/22/2012 12:47 PM, Richard W.M. Jones wrote:
> >
> >I'm a bit surprised to find that native dynlink doesn't work in the
> >same way as bytecode dynlink in respect to reloading the same module.
> >(See attached test program)
> >
> >In bytecode dynlink, reloading (ie. Dynlink.loadfile) the same module
> >causes the new module code to override the old module code:
> >
> > $ ./dynlink_test
> > testing bytecode ...
> > a
> > b
> >
> >But in native dynlink, the new module is silently discarded and the
> >old code appears to run:
> >
> > $ ./dynlink_test
> > testing native ...
> > a
> > a
> >
> >I would classify this as a bug, but I'm not quite sure what is
> >expected to happen. Is there some other way to override a module as
> >in bytecode?
>
> natdynlink currenlty depends on the OS dynamic linker, we cannot
> control the semantics so precisely (you can try loadfile_private,
> but I'm not sure it would solve your issue).

This is all a bit, umm, unexpected.

Stepping back, the problem I'm trying to solve is how to reload a
configuration-like file into a daemon without restarting the daemon.
The "configuration-like file" is the whenjobs jobs script[1], and the
daemon is the whenjobs daemon[2].

Any ideas on how best to go about this? Note that the whenjobs jobs
script is intentionally a Turing-complete OCaml program, so converting
it to another format is probably not going to be practical.

Rich.

[1] examples:
http://git.annexia.org/?p=whenjobs.git;a=blob;f=tests/jobs/t201_ocaml_set_variable.ml;hb=HEAD
http://people.redhat.com/~rjones/whenjobs/whenjobs.txt

[2] http://git.annexia.org/?p=whenjobs.git;a=blob;f=daemon/daemon.ml;h=bc4f51a2f1177e5be2692a233d7f2b850f9a55bc;hb=HEAD#l293

--
Richard Jones
Red Hat

[Pierre Chambart]

I did not really read your code, but it seams to me that you are doing
the compilation in your tool.
So you can choose the name of the module. Isn't it possible to change
the name of the file each
time you compile/reload it ?
--
Pierre

[Richard W.M. Jones]

On Thu, Mar 22, 2012 at 08:12:08PM +0000, Richard W.M. Jones wrote:

> On Thu, Mar 22, 2012 at 06:14:03PM +0100, Pierre Chambart wrote:
> > On 03/22/2012 05:42 PM, Richard W.M. Jones wrote:
> > > Any ideas on how best to go about this? Note that the whenjobs jobs
> > > script is intentionally a Turing-complete OCaml program, so converting
> > > it to another format is probably not going to be practical.
> > >
> > I did not really read your code, but it seams to me that you are doing
> > the compilation in your tool.
> > So you can choose the name of the module. Isn't it possible to change
> > the name of the file each
> > time you compile/reload it ?
>

> Yes, I guess something involving -pack to pack all the files into
> a randomly-named submodule, could be the way to go.

Just to follow up on this. I implemented this by packing all my files
into a randomly-named module. ie. My files are compiled using the
-pack/-for-pack options as submodules of 'Jobs__<time>' where <time>
is the current time_t. The top (randomly named) module is then loaded.

This works fine in both bytecode and native code.

The code is here:

http://git.annexia.org/?p=whenjobs.git;a=commitdiff;h=de72662854c3db9365296dd45cade2253910be7f

Rich.

--
Richard Jones
Red Hat