Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Re: [Caml-list] Wanted: your feedback on the hierarchy of OCaml Batteries Included

25 views
Skip to first unread message

David Teller

unread,
Nov 18, 2008, 6:18:40 AM11/18/08
to Richard Jones, OCaml
This raises two questions:
1) how important is it to allow third-party modules to extend the
namespace?
2) how important is it to offer a uniform package structure (where
levels are always separated by '.' rather than some level by '.' and
some by '_')?

For the moment, we have considered point 1 not very important and point
2 a little more. There are several reasons to disregard point 1. Among
these, clarity of origin (as in "is this module endorsed by Batteries or
not?") and documentation issues (as in "gosh, this module pretends to be
part of [Data] but I can't find the documentation anywhere in the
documentation of Batteries, wtf?").

Do you believe that we should have chosen otherwise?

Cheers,
David

On Tue, 2008-11-18 at 10:06 +0000, Richard Jones wrote:
> Your biggest problem is using dot ('.') instead of underscore ('_').
> Using a dot means that the System namespace cannot be extended by
> external packages. If you use an underscore then an external package
> can extend the namespace (eg. by providing System_Newpackage)
>
> Rich.
>
--
David Teller-Rajchenbach
Security of Distributed Systems
http://www.univ-orleans.fr/lifo/Members/David.Teller
Angry researcher: French Universities need reforms, but the LRU act brings liquidations.

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs

David Teller

unread,
Nov 18, 2008, 6:22:24 AM11/18/08
to Zheng Li, OCaml, Richard Jones
I thought the linker only linked in symbols which were actually used?

On Tue, 2008-11-18 at 11:21 +0100, Zheng Li wrote:
> > Your biggest problem is using dot ('.') instead of underscore ('_').
> > Using a dot means that the System namespace cannot be extended by
> > external packages. If you use an underscore then an external package
> > can extend the namespace (eg. by providing System_Newpackage)
>

> And, doesn't that forces all sub modules to be linked into the final
> executables even if we only use one of them?

Daniel Bünzli

unread,
Nov 18, 2008, 6:35:39 AM11/18/08
to OCaml List

Le 18 nov. 08 à 11:29, Erkki Seppala a écrit :

> For example I prefer using the least amount of opening of modules,
> to make it easier to see where the values come from

Same here. This is why I'm a little bit sceptical about this hierarchy.

With the current standard library if I suddenly want to use
Int32.of_int, I know I just need to type Int32.of_int in my source.
With your proposal I need to remember that it is in Data.Numeric and
go at the beginning of my file to open it or write
Data.Numeric.Int32.of_int, to me this brings bureaucracy without any
benefit. And lack of bureaucracy is one of the reasons I like ocaml
(and dislike java for example).

Besides Hierarchies are anyway limited in their descriptive power and
one day you'll find something that will fit in two places, Rope is
already an example being both Data.Persistent and Data.Text.

Thus my proposal would be to _present_ them as a hierarchy (but even
here a mean to tag/browse the modules with/by keywords would do a
better job) but keep the actual module structure of Batteries as flat
as possible, everything just under the toplevel Batteries. When I code
I really don't want to have to think about all these open directives
that essentially bring nothing.

Best,

Daniel

Thomas Gazagnaire

unread,
Nov 18, 2008, 6:47:46 AM11/18/08
to Daniel Bünzli, OCaml List
>
> With the current standard library if I suddenly want to use Int32.of_int, I
> know I just need to type Int32.of_int in my source. With your proposal I
> need to remember that it is in Data.Numeric and go at the beginning of my
> file to open it or write Data.Numeric.Int32.of_int, to me this brings
> bureaucracy without any benefit. And lack of bureaucracy is one of the
> reasons I like ocaml (and dislike java for example).
>
> Besides Hierarchies are anyway limited in their descriptive power and one
> day you'll find something that will fit in two places, Rope is already an
> example being both Data.Persistent and Data.Text.
>

I use modules in the same way, mostly to be able to grep Int32.of_int in my
code when needed (as greping for of_int only would make the result less
precise).


> Thus my proposal would be to _present_ them as a hierarchy (but even here a
> mean to tag/browse the modules with/by keywords would do a better job) but
> keep the actual module structure of Batteries as flat as possible,
> everything just under the toplevel Batteries. When I code I really don't
> want to have to think about all these open directives that essentially bring
> nothing.
>

tag system for modules is a good idea, and I would like to add that type
search for functions (which is already done by ocamlbrowser) is also nice.
--
Thomas

David Teller

unread,
Nov 18, 2008, 7:15:43 AM11/18/08
to Daniel Bünzli, OCaml List
On Tue, 2008-11-18 at 12:34 +0100, Daniel Bünzli wrote:
>Besides Hierarchies are anyway limited in their descriptive power and
>one day you'll find something that will fit in two places, Rope is
>already an example being both Data.Persistent and Data.Text.

That's correct, there are plenty of modules which could fit in different
places. For the moment, we decided that every module should appear only
in one place. However, we could easily change this -- in fact, to allow
this, we only need to alter our documentation generator.

> Thus my proposal would be to _present_ them as a hierarchy (but even
> here a mean to tag/browse the modules with/by keywords would do a
> better job) but keep the actual module structure of Batteries as flat
> as possible, everything just under the toplevel Batteries. When I code
> I really don't want to have to think about all these open directives
> that essentially bring nothing.

Browsing by keywords sounds like an interesting idea. I'm adding this to
our TODO list. Of course, the next step will be to actually add these
keywords and that's going to be much longer if we intend to tag all
values.

However, we disagree on the necessity of a hierarchy. There are two good
reasons why the base library of OCaml doesn't have a hierarchy (almost):
it's small and there are almost no redundancies between modules. Neither
is true for Batteries.

For an example of this redundancy, consider threads. For the moment, we
have five thread-related modules: [Threads], [Mutex], [RMutex],
[Condition] and [Event]. These modules, which are essentially the same
modules as those of the base library, are all submodules of
[Control.Concurrency.Threads]. Now, I personally like
[Control.Concurrency] but I agree that this is debatable. The reason why
we group these modules into [Threads] is because sooner or later, we
are going to have four or five other thread-related modules called
[Threads], [Mutex], [Condition], [Event] and perhaps [RMutex]. These
modules will get into [Control.Concurrency.CoThreads]. They won't
replace the first batch, they will exist side-by-side. Of course, we
could trim the hierarchy and remove [Control.Concurrency] -- trimming
the hierarchy is the main reason for launching this thread,
incidentally. But, to keep things ordered, we will still need modules
[Threads.Threads], [Threads.Mutex], [Threads.RMutex]...
[CoThreads.Threads], [CoThreads.Mutex]... and, well, that's a hierarchy
already.

coThreads is not an exceptional case, mind you. We may end up with two
definitions of [Graphics], several data structures with the same name
but different purposes, etc.

There's also the issue of labels and other partial redefinitions of
modules. The OCaml base library defines [Array]/[ArrayLabels],
[List]/[ListLabels], [Map]/[MoreLabels.MapLabels] etc. In Batteries
Included, we define [Array], [Array.Labels], [List], [List.Labels],
which clutters less the list of modules and makes for something more
consistent, especially since [FooLabel] is not the only kind of "module
[Foo] with a variant": we also have [Array.ExceptionLess], for
operations without exceptions, and [Array.Cap] for read-only/write-only
arrays. Other variants may still appear.

Do you see any better way of managing the complexity of all this?

Cheers,
David


--
David Teller-Rajchenbach
Security of Distributed Systems
http://www.univ-orleans.fr/lifo/Members/David.Teller
Angry researcher: French Universities need reforms, but the LRU act brings liquidations.

_______________________________________________

Richard Jones

unread,
Nov 18, 2008, 7:22:58 AM11/18/08
to David Teller, OCaml
On Tue, Nov 18, 2008 at 12:17:28PM +0100, David Teller wrote:
> This raises two questions:
> 1) how important is it to allow third-party modules to extend the
> namespace?
> 2) how important is it to offer a uniform package structure (where
> levels are always separated by '.' rather than some level by '.' and
> some by '_')?
>
> For the moment, we have considered point 1 not very important and point
> 2 a little more. There are several reasons to disregard point 1. Among
> these, clarity of origin (as in "is this module endorsed by Batteries or
> not?") and documentation issues (as in "gosh, this module pretends to be
> part of [Data] but I can't find the documentation anywhere in the
> documentation of Batteries, wtf?").
>
> Do you believe that we should have chosen otherwise?

Easy - look at CPAN[1]. If you want to scale a project you have to
make decisions that allow a distributed network of people to
cooperate, without needing too much central coordination. CPAN is a
great example of this loose coupling because packages make their own
decision about naming (albeit they can become "official" later - but
they won't need to rename unless there is an actual naming conflict).

If the problem is documentation or provenance of packages, then add a
mechanism to solve that problem. Perl also solves this through an
existing, lightweight, distributed mechanism (a standard location to
install man-pages, and a standard man-page format and man-page
generating mechanism -- POD).

Rich.

[1] http://www.cpan.org/

--
Richard Jones
Red Hat

Richard Jones

unread,
Nov 18, 2008, 7:32:41 AM11/18/08
to David Teller, Daniel Bünzli, OCaml List
On Tue, Nov 18, 2008 at 01:15:39PM +0100, David Teller wrote:
> Do you see any better way of managing the complexity of all this?

I'm still not getting where the benefit of having this hierarchy is,
except that it adds a Java-like complexity and will create
hard-to-manage churn if a module ever moves.

API changes are handled really badly in OCaml, ironically because of
the lack of a textual preprocessor. You can't just write this every
time lablgtk / calendar / latest culprit decides to change their API:

#ifdef LABLGTK < 210
let icon = GMisc.image () in
icon#set_stock icon_type ~size:size;
icon
#else
let icon = GMisc.image () in
icon#set_stock `DIALOG_ERROR;
icon#set_icon_size `DIALOG;
icon
#endif

(Well, you can run -pp cpp, but that breaks other stuff)

Rich.

--
Richard Jones
Red Hat

_______________________________________________

David Teller

unread,
Nov 18, 2008, 7:40:08 AM11/18/08
to Daniel Bünzli, OCaml List
On Tue, 2008-11-18 at 12:34 +0100, Daniel Bünzli wrote:
> Le 18 nov. 08 à 11:29, Erkki Seppala a écrit :
>
> > For example I prefer using the least amount of opening of modules,
> > to make it easier to see where the values come from
>
> Same here. This is why I'm a little bit sceptical about this hierarchy.
>
> With the current standard library if I suddenly want to use
> Int32.of_int, I know I just need to type Int32.of_int in my source.
> With your proposal I need to remember that it is in Data.Numeric and
> go at the beginning of my file to open it or write
> Data.Numeric.Int32.of_int, to me this brings bureaucracy without any
> benefit. And lack of bureaucracy is one of the reasons I like ocaml
> (and dislike java for example).

I forgot to answer that part.

In Batteries, for the moment, we decided to keep the module names of the
base library as shortcuts to our new modules. Consequently, you can
still write your [Int32.of_int] in addition to our new [Int32.print],
etc. The old modules are still available as submodules of [Legacy], if
needed.

Should you wish to flatten the complete hierarchy, assuming that it's
possible and that there are no collisions on names, that's also
something which you can do quite easily. We even provide some syntactic
sugar for this. It's just the matter of writing a file my_batteries.ml
along the lines of

module Array = Data.Mutable.Array
module List = Data.Persistent.List
..
module PosixThreads = Control.Concurrency.Threads.Threads
module PosixMutex = Control.Concurrency.Threads.Mutex
module CoThreads = Control.Concurrency.CoThreads.Threads
..
module ArrayExn = Data.Mutable.Array include ExceptionLess
(*syntactic sugar*)
module ArrayLabels = Data.Mutable.Array include Labels
module ArrayCapExn = Data.Mutable.Array.Cap include ExceptionLess
module ArrayCapLabels= Data.Mutable.Array.Cap include Labels
..

I personally don't like name [ArrayCapLabels] but I can't think of any
better name to represent this once we have removed any hierarchy.

I personally prefer the hierarchy but, once again, the majority may
disagree. So if you believe this is better, the next logical step would
be to design a full and consistent list of modules including all the
modules which already appear in the current version of Batteries, and
with some space left for OCamlnet, OCamlnae, Reins, Camomile, ULex,
Camlp4, CoThreads and a few others. I truly mean it, if you can provide
us with something you consider more comfortable and as future-proof, we
may adopt it.

Cheers,
David

--
David Teller-Rajchenbach
Security of Distributed Systems
http://www.univ-orleans.fr/lifo/Members/David.Teller
Angry researcher: French Universities need reforms, but the LRU act brings liquidations.

_______________________________________________

Zheng Li

unread,
Nov 18, 2008, 7:51:05 AM11/18/08
to David Teller, OCaml, Richard Jones
David Teller wrote:
> I thought the linker only linked in symbols which were actually used?

You really should check.

I have not yet looked too much into the source, but if the
batteries_core.ml is one of them to be referenced anyway, I'm afraid all
modules (not just parents/siblings) will be linked.

Try to compile the following source into executable:

----
open Batteries.Data.Persistent.List

let _ = iter
----

You will end up with being asked for numerous unrelated modules during
the linking phrase, or you can use the recommended "ocamlfind
batteries/ocamlc" shortcut. Either way, an executable of +50 times
bigger in size (i.e. +1M for the 2 lines) than using the standard List
will be produced.

--
Zheng

> On Tue, 2008-11-18 at 11:21 +0100, Zheng Li wrote:
>>> Your biggest problem is using dot ('.') instead of underscore ('_').
>>> Using a dot means that the System namespace cannot be extended by
>>> external packages. If you use an underscore then an external package
>>> can extend the namespace (eg. by providing System_Newpackage)
>> And, doesn't that forces all sub modules to be linked into the final
>> executables even if we only use one of them?
>

_______________________________________________

David Teller

unread,
Nov 18, 2008, 7:51:43 AM11/18/08
to Benedikt Grundmann, OCaml
Ok, that's an interesting point. Now, we just need to all agree on one
standard :)

On Tue, 2008-11-18 at 12:28 +0000, Benedikt Grundmann wrote:
> > Do you see any better way of managing the complexity of all this?

> Yes don't introduce it at all, make a decision to use or not use labels
> and stick with it. Similarly make a decision to use or not use exceptions
> as the "default", suffix / rename alternative functions as appropriate
> (consistently). Consistency is a big win. Not only as it speeds you up
> when you read/modify other people's code it also reduces the amount
> of decisions you have to do when writing new code.
>
> http://ocaml.janestreet.com/?q=node/28
>
> Cheers,
>
> Bene

David Teller

unread,
Nov 18, 2008, 7:52:23 AM11/18/08
to Richard Jones, OCaml
On Tue, 2008-11-18 at 12:22 +0000, Richard Jones wrote:
> > Do you believe that we should have chosen otherwise?
>
> Easy - look at CPAN[1]. If you want to scale a project you have to
> make decisions that allow a distributed network of people to
> cooperate, without needing too much central coordination. CPAN is a
> great example of this loose coupling because packages make their own
> decision about naming (albeit they can become "official" later - but
> they won't need to rename unless there is an actual naming conflict).

Interesting point. So far, the approach of Batteries has certainly been
different, in large part because we don't want everything to end up part
of the Batteries hierarchy (or, well, lack thereof). Of course, this is
in contradiction with our sometimes imperialistic tendencies, so we may
be guilty of schizophrenia.

Perhaps we should organise a poll on this subject.

> If the problem is documentation or provenance of packages, then add a
> mechanism to solve that problem. Perl also solves this through an
> existing, lightweight, distributed mechanism (a standard location to
> install man-pages, and a standard man-page format and man-page
> generating mechanism -- POD).

I'm not sure the man-page format quite scales up to the kind of
hyperlinked complexity we have in Batteries for the moment. But yes, I
agree, we can certainly work something out. In fact, we could say that
we've started on this track, albeit perhaps not with such grand
ambitions.

Thanks for the idea,
David

P.S.: I've pointedly ignored your perch on POD :) In my mind, that's a
very different topic. For the moment, we'll stick with ocamldoc.

--
David Teller-Rajchenbach
Security of Distributed Systems
http://www.univ-orleans.fr/lifo/Members/David.Teller
Angry researcher: French Universities need reforms, but the LRU act
brings liquidations.

_______________________________________________

David Teller

unread,
Nov 18, 2008, 7:58:00 AM11/18/08
to Richard Jones, Daniel Bünzli, OCaml List
On Tue, 2008-11-18 at 12:32 +0000, Richard Jones wrote:
> API changes are handled really badly in OCaml, ironically because of
> the lack of a textual preprocessor. You can't just write this every
> time lablgtk / calendar / latest culprit decides to change their API:
>
> #ifdef LABLGTK < 210
> let icon = GMisc.image () in
> icon#set_stock icon_type ~size:size;
> icon
> #else
> let icon = GMisc.image () in
> icon#set_stock `DIALOG_ERROR;
> icon#set_icon_size `DIALOG;
> icon
> #endif

Side-note: That's certainly something we could add to Batteries, if
needed. Camlp4 is pretty-much necessary to use Batteries anyway and
Camlp4 already defines IFDEF, INCLUDE, etc. We would just need to
complete that DSL perhaps to accept any valid OCaml expression and call
the ocaml interpreter to evaluate these expressions.

Cheers,
David

--
David Teller-Rajchenbach
Security of Distributed Systems
http://www.univ-orleans.fr/lifo/Members/David.Teller
Angry researcher: French Universities need reforms, but the LRU act brings liquidations.

_______________________________________________

Daniel Bünzli

unread,
Nov 18, 2008, 8:26:12 AM11/18/08
to OCaml List

Le 18 nov. 08 à 13:15, David Teller a écrit :

> But, to keep things ordered, we will still need modules
> [Threads.Threads], [Threads.Mutex], [Threads.RMutex]...
> [CoThreads.Threads], [CoThreads.Mutex]... and, well, that's a
> hierarchy
> already.

If you include in batteries an external package that has its own
hierarchy and is designed to be opened I don't mind having that
hierarchy. In that case you can just add the new toplevel entry
CoThread. And if I want to use CoThread, I just open CoThreads, not
Control.Concurrency.CoThreads. Just try to keep it as flat as
possible, don't try to force modules in an ad-hoc hierarchical
taxonomy to try to sort out modules. I don't care if the toplevel list
of modules is three hundred pages long if there is an efficient mean
to access their documentation (like tags). I do however care a lot if
it becomes bureaucratic to be able to _use_ a module in my code.


Le 18 nov. 08 à 13:22, Richard Jones a écrit :

> Easy - look at CPAN[1]. If you want to scale a project you have to
> make decisions that allow a distributed network of people to
> cooperate, without needing too much central coordination.

But (unfortunately, sorry to repeat that) Batteries is not a CPAN like
initiative. It aims at giving a library of modules/syntax extensions
selected by the library maintainers, as such it is inherently
centralized and I don't think that questions (1) or (2) are actually
pertinent for the project.

Best,

Daniel

Dario Teixeira

unread,
Nov 18, 2008, 8:32:06 AM11/18/08
to OCaml List
Hi,

> I personally prefer the hierarchy but, once again, the majority
> may disagree. So if you believe this is better, the next logical
> step would be to design a full and consistent list of modules
> including all the modules which already appear in the current
> version of Batteries, and with some space left for OCamlnet,
> OCamlnae, Reins, Camomile, ULex, Camlp4, CoThreads and a few
> others. I truly mean it, if you can provide us with something
> you consider more comfortable and as future-proof, we may adopt it.

Paraphrasing Einstein, I think the hierarchy should be as flat
as possible, but no flatter. For example, I see no reason to
materialise in the hierarchy the separation between persistent
and mutable data structures. The should be a documentation
issue. However, and as you noted, there are cases where some
hierarchisation may remove namespace clutter and allow for
better code reuse.

Cheers,
Dario Teixeira

Alain Frisch

unread,
Nov 18, 2008, 9:11:26 AM11/18/08
to David Teller, OCaml, Richard Jones, Zheng Li
David Teller wrote:
> I thought the linker only linked in symbols which were actually used?

No, it is not the case.

The only automatic mechanism for code pruning is at the level of
individual modules embedded in a library. As soon as you pack, you
obtain a monolithic module which can only be linked as a whole.

-- Alain

David Teller

unread,
Nov 18, 2008, 9:23:43 AM11/18/08
to Dario Teixeira, OCaml List
On Tue, 2008-11-18 at 05:31 -0800, Dario Teixeira wrote:
> Paraphrasing Einstein, I think the hierarchy should be as flat
> as possible, but no flatter. For example, I see no reason to
> materialise in the hierarchy the separation between persistent
> and mutable data structures. The should be a documentation
> issue. However, and as you noted, there are cases where some
> hierarchisation may remove namespace clutter and allow for
> better code reuse.

Duly noted. As you may see on our candidate replacement hierarchy, we
intend to merge Data.Persistent and Data.Mutable into Data.Containers.

Whether we flatten further remains open to debate.

Thanks,
David

--
David Teller-Rajchenbach
Security of Distributed Systems
http://www.univ-orleans.fr/lifo/Members/David.Teller
Angry researcher: French Universities need reforms, but the LRU act
brings liquidations.

_______________________________________________

Stefano Zacchiroli

unread,
Nov 18, 2008, 9:40:42 AM11/18/08
to caml...@yquem.inria.fr
On Tue, Nov 18, 2008 at 03:23:33PM +0100, David Teller wrote:
> On Tue, 2008-11-18 at 05:31 -0800, Dario Teixeira wrote:
> > Paraphrasing Einstein, I think the hierarchy should be as flat
> > as possible, but no flatter. For example, I see no reason to
> > materialise in the hierarchy the separation between persistent
> > and mutable data structures. The should be a documentation
> > issue. However, and as you noted, there are cases where some
> > hierarchisation may remove namespace clutter and allow for
> > better code reuse.
>
> Duly noted. As you may see on our candidate replacement hierarchy, we
> intend to merge Data.Persistent and Data.Mutable into Data.Containers.

More generally, I would like to advertise a bit more the proposed
*replacement* hierarchy reported at the bottom of David's blog post
[1]; do a text search for "One possible replacement" and start reading
from there.

Several problems with the current hierarchy which have been pointed
out in this thread were notice by ourselves as well, and are already,
at least partly, solved by the proposed new hierarchy.

Cheers.

[1] http://dutherenverseauborddelatable.wordpress.com/2008/11/18/batteries-hierarchy/

--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'č ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu ą tous ceux que j'aime

David Teller

unread,
Nov 18, 2008, 9:47:07 AM11/18/08
to Alain Frisch, OCaml, Richard Jones, Zheng Li
Ok, good to know. Since we're packing anyway, there's nothing we can do
yet. However, we've already planned to work on a dynamically linked
version of Batteries. Just not for release 1.0

So back to square 1 on this argument.

Thanks Alain & Zheng


On Tue, 2008-11-18 at 15:10 +0100, Alain Frisch wrote:
> David Teller wrote:
> > I thought the linker only linked in symbols which were actually used?
>
> No, it is not the case.
>
> The only automatic mechanism for code pruning is at the level of
> individual modules embedded in a library. As soon as you pack, you
> obtain a monolithic module which can only be linked as a whole.
>
> -- Alain
>
>

--
David Teller-Rajchenbach
Security of Distributed Systems
http://www.univ-orleans.fr/lifo/Members/David.Teller
Angry researcher: French Universities need reforms, but the LRU act brings liquidations.

_______________________________________________

David Teller

unread,
Nov 18, 2008, 9:47:07 AM11/18/08
to Daniel Bünzli, OCaml List
On Tue, 2008-11-18 at 14:24 +0100, Daniel Bünzli wrote:
> Le 18 nov. 08 à 13:15, David Teller a écrit :
>
> > But, to keep things ordered, we will still need modules
> > [Threads.Threads], [Threads.Mutex], [Threads.RMutex]...
> > [CoThreads.Threads], [CoThreads.Mutex]... and, well, that's a
> > hierarchy
> > already.
>
> If you include in batteries an external package that has its own
> hierarchy and is designed to be opened I don't mind having that
> hierarchy.
>
> In that case you can just add the new toplevel entry
> CoThread. And if I want to use CoThread, I just open CoThreads, not
> Control.Concurrency.CoThreads. Just try to keep it as flat as
> possible, don't try to force modules in an ad-hoc hierarchical
> taxonomy to try to sort out modules. I don't care if the toplevel list
> of modules is three hundred pages long if there is an efficient mean
> to access their documentation (like tags). I do however care a lot if
> it becomes bureaucratic to be able to _use_ a module in my code.

I concur that tags make a considerable difference.

But let us return to threads for one second. There is a very good reason
to have two distinct modules [Threads] and [CoThreads] with 4-5
submodules each: functors. Assuming [Threads] and [CoThreads] implement
the same interface -- which they do -- I can write a module which takes
as argument either [Threads], [CoThreads] or [WhateverThreads] and
produces a pseudo-concurrent/truly concurrent/whatever implementation of
an algorithm. The same thing could apply to latin-1 strings vs. Unicode
strings (this is essentially what happens in Camomile).

Now, there are certainly several possibilities.

Here's one which doesn't involve a deep hierarchy:
* [Thread], [Mutex], [Concurrent], [Event] remain top-level modules
* [Threads] is also a top-level module, which contains aliases to
[Thread], [Mutex], [Concurrent], [Event]
* [CoThreads] is also a top-level module, which contains its own
implementations of [Thread], [Mutex], [Concurrent], [Event]


We could do the same for strings
* [String], [Char], [Rope], [UChar] remain top-level modules
* we introduce a new module [Strings] containing [String] and [Char]
* we introduce another new module [UStrings] containing an alias
[String] to [Rope] and an alias [Char] to [UChar]

And for numbers
* [Float], [Int], [SafeInt], [BigInt] and hypothetical [SafeFloat] and
[BigFloat] (don't ask me what a BigFloat is supposed to be) remain
top-level modules
* we introduce a new module [Numeric] containing [Float] and [Int]
* we introduce a new module [SafeNumeric] containing [SafeFloat] aliased
as [Float], [SafeInt] aliased as [Int]
* we introduce a new module [BigNumeric] containing [BigFloat] aliased
as [Float], [BigInt] aliased as [Int]

etc.

To me, this seems like the only way to combine no hierarchy and
modularity. However, I have the nasty feeling that this is going to end
up messy, cluttered and otherwise both unmaintainable and unusable
(despite tags).

>
> Le 18 nov. 08 à 13:22, Richard Jones a écrit :
>
> > Easy - look at CPAN[1]. If you want to scale a project you have to
> > make decisions that allow a distributed network of people to
> > cooperate, without needing too much central coordination.
>
> But (unfortunately, sorry to repeat that) Batteries is not a CPAN like
> initiative. It aims at giving a library of modules/syntax extensions
> selected by the library maintainers, as such it is inherently
> centralized and I don't think that questions (1) or (2) are actually
> pertinent for the project.

No, we're not CPAN. If someone wishes to build a CPAN, please feel free
to do it. That may actually be easier to do once Batteries 1.0 has
landed. However, Richard's remark remains interesting. So perhaps
redesigning Batteries to have an open namespace structure is a good
idea.

Cheers,
David

--
David Teller-Rajchenbach
Security of Distributed Systems
http://www.univ-orleans.fr/lifo/Members/David.Teller
Angry researcher: French Universities need reforms, but the LRU act brings liquidations.

_______________________________________________

Richard Jones

unread,
Nov 18, 2008, 10:20:42 AM11/18/08
to David Teller, OCaml
On Tue, Nov 18, 2008 at 01:49:09PM +0100, David Teller wrote:
> P.S.: I've pointedly ignored your perch on POD :) In my mind, that's a
> very different topic. For the moment, we'll stick with ocamldoc.

I've used POD selectively even in OCaml projects, mainly because it is
by far the easiest way to generate man pages. OCamldoc is great for
developer documentation (APIs etc) but POD is super-simple for making
manual pages.

cf man page:
http://hg.et.redhat.com/virt/applications/virt-top--devel/?f=5b38082d8aa4;file=virt-top/virt-top.pod
vs ocamldoc documentation:
http://hg.et.redhat.com/virt/applications/ocaml-libvirt--devel/?f=893899664388;file=libvirt/libvirt.mli

One place where POD really stands out, and could be replicated by
camlp4, is for standalone programs that combine argument parsing,
usage and man page all in one place. In many cases you can keep the
option parsing, implementation of the option, and documentation for
the option right next to each other.

http://perldoc.perl.org/Getopt/Long.html#Documentation-and-help-texts

Rich.

--
Richard Jones
Red Hat

_______________________________________________

Richard Jones

unread,
Nov 18, 2008, 1:59:22 PM11/18/08
to Jon Harrop, caml...@yquem.inria.fr
On Tue, Nov 18, 2008 at 06:17:23PM +0000, Jon Harrop wrote:
> I don't follow. Can you not use "include" to extend an existing module:
>
> # module Array = struct
> include Array

You're missing the point which is scalability - how to deal with
distributed parties who are loosely coordinated. The above scheme
allows one person to extend the Array module, but not two people,
unless they coordinate with each other about which order they extend
it (or both have incompatible extensions).

Jon Harrop

unread,
Nov 18, 2008, 2:15:42 PM11/18/08
to caml...@yquem.inria.fr, Richard Jones
On Tuesday 18 November 2008 18:59:14 Richard Jones wrote:
> On Tue, Nov 18, 2008 at 06:17:23PM +0000, Jon Harrop wrote:
> > I don't follow. Can you not use "include" to extend an existing module:
> >
> > # module Array = struct
> > include Array
>
> You're missing the point which is scalability - how to deal with
> distributed parties who are loosely coordinated. The above scheme
> allows one person to extend the Array module, but not two people,
> unless they coordinate with each other about which order they extend
> it (or both have incompatible extensions).

If the library creator did not use functors or classes to make their design
reusable then the only solution for the user is to include all of the
implementations they require:

module Array = struct
include RichardsArray
include JonsArray
end

Given the lack of libraries available for OCaml anyway, this seems like a very
minor concern to me.

--
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e

Richard Jones

unread,
Nov 18, 2008, 2:22:49 PM11/18/08
to Jon Harrop, caml...@yquem.inria.fr
On Tue, Nov 18, 2008 at 08:17:36PM +0000, Jon Harrop wrote:
> If the library creator did not use functors or classes to make their design
> reusable then the only solution for the user is to include all of the
> implementations they require:

You're talking about something completely different.

In Perl they have:

Net
Net::Amazon
Net::BitTorrent
Net::FTPServer
(and a million others[1])

The proposal is to have a hierarchy of OCaml modules, of this sort:

Net
Net.Amazon
Net.BitTorrent
Net.FTPServer
(and a million more)

which doesn't scale. However, using '_' as a separator scales because
distributed, loosely coordinated parties can add new modules ad hoc to
such a namespace.

Rich.

[1] http://www.cpan.org/modules/by-module/Net/

--
Richard Jones
Red Hat

_______________________________________________

Daniel Bünzli

unread,
Nov 18, 2008, 2:52:00 PM11/18/08
to OCaml List

Le 18 nov. 08 à 20:22, Richard Jones a écrit :

> The proposal is to have a hierarchy of OCaml modules, of this sort:
>
> Net
> Net.Amazon
> Net.BitTorrent
> Net.FTPServer
> (and a million more)
>
> which doesn't scale.

If there is nothing in the Net module (and ignoring the linking issue)
you can actually achieve that by using -pack. Just redo the pack on
the client whenever it installs a new package in the namespace. No ?

Best,

Daniel

Jon Harrop

unread,
Nov 18, 2008, 4:40:54 PM11/18/08
to Nicolas Pouillard, Caml_mailing list
On Tuesday 18 November 2008 17:51:21 Nicolas Pouillard wrote:
> Excerpts from Jon Harrop's message of Tue Nov 18 19:17:23 +0100 2008:

> > # module Array = struct
> > include Array
> > let empty = [||]
> > end;;
> > module Array :
> > sig
> > external length : 'a array -> int = "%array_length"
> > ...
> > val empty : 'a array
> > end
>
> Yes but that's the same than saying you can change a value:
>
> let x = 42
> let x = x + 1
>
> So you make a new module but don't extend it.

In what way is that unsatisfactory?

--
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e

_______________________________________________

Richard Jones

unread,
Nov 18, 2008, 4:50:25 PM11/18/08
to Daniel Bünzli, OCaml List
On Tue, Nov 18, 2008 at 08:50:51PM +0100, Daniel Bünzli wrote:
> Le 18 nov. 08 ŕ 20:22, Richard Jones a écrit :

> >The proposal is to have a hierarchy of OCaml modules, of this sort:
> >
> > Net
> > Net.Amazon
> > Net.BitTorrent
> > Net.FTPServer
> > (and a million more)
> >
> >which doesn't scale.
>
> If there is nothing in the Net module (and ignoring the linking issue)
> you can actually achieve that by using -pack. Just redo the pack on
> the client whenever it installs a new package in the namespace. No ?

No because Net isn't necessarily an empty module, nor does it
magically pull in all the modules underneath it (which would be
impossible because the Net::* space is constantly changing).

Rich.

--
Richard Jones
Red Hat

_______________________________________________

Alain Frisch

unread,
Nov 18, 2008, 5:07:48 PM11/18/08
to caml...@yquem.inria.fr
On 11/18/2008 7:17 PM, Jon Harrop wrote:
> I don't follow. Can you not use "include" to extend an existing module:
>
> # module Array = struct
> include Array
> let empty = [||]
> end;;
> module Array :
> sig
> external length : 'a array -> int = "%array_length"
> ...
> val empty : 'a array
> end

In addition to this being non-modular, this extension scheme does not
work well with hiararchy as it forces you to mention all the siblings of
the ancestors of the module you want to extend.

E.g. if you start from:

module M = struct
module M1 = struct
module M11 = struct ... end
module M12 = struct ... end
module M13 = struct ... end
...
end
module M2 = struct
...
end
module M3 = struct
...
end
...
end

and you want to extend M11, you need to write:

module M' = struct
module M1 = struct
module M11 = struct include M.M1.M11 (* extension here *) end
module M12 = M.M1.M12
module M13 = M.M1.M13
...
end
module M2 = M.M2
module M3 = M.M3
...
end


Frankly, I don't think that having a nice and well-organized hierarchy
of modules really matters. Things like having uniform interfaces,
consistent idioms and compatible types across libraries seem much more
important to me. Anyway, if a hierarchy is desired, I fail to see any
advantage of using "." instead of e.g. "_" (easily extensible + does not
force you to link everything).

-- Alain

Jon Harrop

unread,
Nov 18, 2008, 5:27:43 PM11/18/08
to caml...@yquem.inria.fr
On Tuesday 18 November 2008 09:56:18 David Teller wrote:
> Now, we've decided that our current hierarchy is perhaps somewhat clumsy
> and that it may benefit from some reworking. Before we proceed, we'd
> like some feedback from the community...

I only have one major concern: you say "with the large number of modules
involved, we would need a hierarchy of modules" but the number of modules
involved is tiny (a few dozen in OCaml compared to tens or even hundreds of
thousands in any industrial-strength language) because OCaml has very few
libraries. Yet your module hierarchies are already enormous and often require
a longer sequence of modules to reach simple functionality than is required
in a comparatively-huge library like .NET.

To me, the most striking example is printf which is just printf in F#,
Printf.printf in OCaml and is now Text.Printf.printf in OCaml+Batteries.
Surely this is a step in the wrong direction?

--
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e

_______________________________________________

Jon Harrop

unread,
Nov 18, 2008, 5:46:58 PM11/18/08
to caml...@yquem.inria.fr
On Tuesday 18 November 2008 22:07:33 Alain Frisch wrote:
> and you want to extend M11, you need to write:
>
> module M' = struct
> module M1 = struct
> module M11 = struct include M.M1.M11 (* extension here *) end
> module M12 = M.M1.M12
> module M13 = M.M1.M13
> ...
> end
> module M2 = M.M2
> module M3 = M.M3
> ...
> end

Ah, yes. Otherwise you get "Multiple definition of the module name ...".

Perhaps that could be solved with extensive Camlp4 hacking to rename the
previous modules (even coming from an "include") to avoid the clash?

> Frankly, I don't think that having a nice and well-organized hierarchy
> of modules really matters. Things like having uniform interfaces,
> consistent idioms and compatible types across libraries seem much more
> important to me.

Indeed. I think the current system would withstand an order of magnitude more
(popular) libraries. I'd also recommend the SML Basis library and F# for
inspiration: they both contain some great designs.

> Anyway, if a hierarchy is desired, I fail to see any advantage of using "."
> instead of e.g. "_" (easily extensible + does not force you to link
> everything).

That brings its own problems, of course. You no longer have a real hierarchy
so you cannot do anything at a given depth in the hierarchy, e.g. apply
mid-level module to a functor.

No doubt people will want both so we'll end up with an ad-hox mix of "."
and "_" separators. In that case, I'd prefer to flatten every "_" (assuming
names didn't clash).

--
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e

_______________________________________________

Alain Frisch

unread,
Nov 18, 2008, 6:14:44 PM11/18/08
to Jon Harrop, caml...@yquem.inria.fr
On 11/19/2008 12:49 AM, Jon Harrop wrote:
> Perhaps that could be solved with extensive Camlp4 hacking to rename the
> previous modules (even coming from an "include") to avoid the clash?

I don't think so. It seems you need type information.

> That brings its own problems, of course. You no longer have a real hierarchy
> so you cannot do anything at a given depth in the hierarchy, e.g. apply
> mid-level module to a functor.

Jon, come on. Is it a joke or do you seriously think that not being able
to apply a functor to a subtree of the hierarchy matters in any way in
this context?


-- Alain

Yaron Minsky

unread,
Nov 18, 2008, 10:09:40 PM11/18/08
to Alain Frisch, Zheng Li, David Teller, Richard Jones, OCaml
Do you have a sense of how hard this would be to fix? It would be lovely to
have some reasonable namespace control without having to take chunks of the
namespace as all-or-nothing pieces....

y

Till Varoquaux

unread,
Nov 18, 2008, 10:47:36 PM11/18/08
to ymi...@gmail.com, OCaml, Zheng Li, David Teller, Richard Jones, Alain Frisch
Whilst not really answering your question I'd like to point out mixin
modules [1]. This is a way to provide extensible modules, thus getting
you a lot of you'd want from a namespace system. I would guess these
would actually be harder to implement than namespaces; both seem to
require compiler support to feel not too hackish. I'm guessing
namespace support could be done in way that'd be fairly orthogonal to
the rest of the typing pass. Maintaining the patch would then be a
problem by itself....

If Batteries included or Core end up being big, coherent collections
of libraries maybe there will be more of incentive to add namespacing
in Ocaml.

Till

[1] http://www.lama.univ-savoie.fr/~hirschowitz/phd/

2008/11/18 Yaron Minsky <ymi...@gmail.com>:

David Teller

unread,
Nov 19, 2008, 1:30:02 AM11/19/08
to Jon Harrop, caml...@yquem.inria.fr

On Tue, 2008-11-18 at 23:30 +0000, Jon Harrop wrote:
> On Tuesday 18 November 2008 09:56:18 David Teller wrote:
> I only have one major concern: you say "with the large number of modules
> involved, we would need a hierarchy of modules" but the number of modules
> involved is tiny (a few dozen in OCaml compared to tens or even hundreds of
> thousands in any industrial-strength language) because OCaml has very few
> libraries. Yet your module hierarchies are already enormous and often require
> a longer sequence of modules to reach simple functionality than is required
> in a comparatively-huge library like .NET.

Well, we're trying to be future-proof. Don't you think we should?

> To me, the most striking example is printf which is just printf in F#,
> Printf.printf in OCaml and is now Text.Printf.printf in OCaml+Batteries.
> Surely this is a step in the wrong direction?

Well, if you it's just the matter of [printf], we can add it to
[Batteries.Standard] to import it in the standard namespace. The biggest
question is how many things we want imported in that standard namespace.
Or you could start your files with [open Text.Printf] or [module P =
Text.Printf] or any similar combination.

Oh, and, [Printf.printf] works, too. This is one of the modules which
have a shortcut to their path in the hierarchy, to mirror the base
library.

Cheers,
David
--
David Teller-Rajchenbach
Security of Distributed Systems
http://www.univ-orleans.fr/lifo/Members/David.Teller
Angry researcher: French Universities need reforms, but the LRU act brings liquidations.

_______________________________________________

Jon Harrop

unread,
Nov 19, 2008, 2:33:53 AM11/19/08
to caml...@yquem.inria.fr
On Wednesday 19 November 2008 06:29:52 David Teller wrote:
> On Tue, 2008-11-18 at 23:30 +0000, Jon Harrop wrote:
> > I only have one major concern: you say "with the large number of modules
> > involved, we would need a hierarchy of modules" but the number of modules
> > involved is tiny (a few dozen in OCaml compared to tens or even hundreds
> > of thousands in any industrial-strength language) because OCaml has very
> > few libraries. Yet your module hierarchies are already enormous and often
> > require a longer sequence of modules to reach simple functionality than
> > is required in a comparatively-huge library like .NET.
>
> Well, we're trying to be future-proof.

Sure.

> Don't you think we should?

No. :-)

I think it is extremely unlikely that OCaml will get many more libraries so I
do not think it is worth spending much time designing infrastructure to cope
with that eventuality.

> > To me, the most striking example is printf which is just printf in F#,
> > Printf.printf in OCaml and is now Text.Printf.printf in OCaml+Batteries.
> > Surely this is a step in the wrong direction?
>
> Well, if you it's just the matter of [printf], we can add it to
> [Batteries.Standard] to import it in the standard namespace. The biggest
> question is how many things we want imported in that standard namespace.
> Or you could start your files with [open Text.Printf] or [module P =
> Text.Printf] or any similar combination.

You could but it will still deter newbies:

open Text.Printf
printf "Hello world!\n"

etc.

> Oh, and, [Printf.printf] works, too. This is one of the modules which
> have a shortcut to their path in the hierarchy, to mirror the base
> library.

Sure. I would certainly vote for flattening out the hierarchy as much as
possible though. For example, I would keep containers in List, Array etc. and
not nest them in Data or Containers or Collections.

--
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e

_______________________________________________

Paolo Donadeo

unread,
Nov 19, 2008, 4:46:35 AM11/19/08
to OCaml mailing list
Couldn't we take inspiration from the Python standard library [1]?
Python hasn't namespace but is provided with a module system similar
to OCaml *and* the standard library is really impressive.

And nobody can say Python is a bureaucratic language like Java :-)


[1] http://www.python.org/doc/2.5.2/lib/lib.html


--
Paolo
~
~
:wq

Stefano Zacchiroli

unread,
Nov 19, 2008, 5:57:39 AM11/19/08
to OCaml
On Tue, Nov 18, 2008 at 10:06:21PM -0500, Yaron Minsky wrote:
> Do you have a sense of how hard this would be to fix? It would be
> lovely to have some reasonable namespace control without having to
> take chunks of the namespace as all-or-nothing pieces....

AOL on this kind of feature request.

With OCaml we have been prone for years with potential module
namespace conflicts. Those conflicts have been thought to be solved by
-pack, which nowadays is finally easy to use via ocamlbuild.

Still, if using it thoroughly we are back to the square 1 of having to
link everything together, then it is not a viable solution for large
libraries.

Cheers.

--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'č ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu ą tous ceux que j'aime

_______________________________________________

Stefano Zacchiroli

unread,
Nov 19, 2008, 8:28:58 AM11/19/08
to caml...@yquem.inria.fr
On Tue, Nov 18, 2008 at 10:06:25AM +0000, Richard Jones wrote:
> Your biggest problem is using dot ('.') instead of underscore ('_').
> Using a dot means that the System namespace cannot be extended by
> external packages. If you use an underscore then an external package
> can extend the namespace (eg. by providing System_Newpackage)

As can be evinced from David's reply, this "biggest problem" comes
from the fact that extension by the means of external libraries was
not a design goal :) But you're right in stating that once one have a
hierarchical organized library, the temptation of adding your own
libraries (you is a third-party contributor here) into it is high.

Still, I'm not sure this problem should be addressed at the Batteries
level. To some extent we have just inherited limitations of the module
system, like the fact that packed module cannot be separately compiled
(or the annoying link all together issue). Frankly I wouldn't like to
give up the hierarchy in favor of "_" for this limitation, before at
least trying to understand if in the future the limitations can be
lifted.

There might be an alternative though:

- use "_" and rely on some syntax extensions to retain the advantages
of the hierarchy, like the fact that you can open a "partial module
path". We can for example use "__" as a hierarchy separator and have
"open" treat that sequence as a "."

No idea how hard that would be, or how many implications there will
be; bluestorm?

Cheers.


--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/

Dietro un grande uomo c'è ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu à tous ceux que j'aime

Stefano Zacchiroli

unread,
Nov 19, 2008, 8:37:04 AM11/19/08
to caml...@yquem.inria.fr
On Tue, Nov 18, 2008 at 12:34:28PM +0100, Daniel Bünzli wrote:
>> For example I prefer using the least amount of opening of modules, to
>> make it easier to see where the values come from
> Same here. This is why I'm a little bit sceptical about this hierarchy.

Well, the problem of not knowing where a value comes from is more a
tool problem, than an argument against hierarchies. The compiler knows
where a value comes from, it is only hard to get this information
back. IIRC one of this year OSP addressed precisely that problem.

> Besides Hierarchies are anyway limited in their descriptive power
> and one day you'll find something that will fit in two places, Rope
> is already an example being both Data.Persistent and Data.Text.

Yes, but that's not a good reason to give up hierarchies
completely. The advantage of hierarchies is to have less top-level
roots, which reduce the likelihood of clashes with external
libraries. Even though in some cases you might need to choose among
two different places, it is rarely the case in practice.

Also remember that the Batteries hierarchy was not meant to allocate
*all* existing libraries into a common hierarchy, that interpretation
came from Rich's comment. So the real question is, according to what
you currently see in the hierarchy, do you like it or not? Do you
something placed in weird places?

Stefano Zacchiroli

unread,
Nov 19, 2008, 8:38:15 AM11/19/08
to caml...@yquem.inria.fr
On Tue, Nov 18, 2008 at 12:32:31PM +0000, Richard Jones wrote:
> On Tue, Nov 18, 2008 at 01:15:39PM +0100, David Teller wrote:
> > Do you see any better way of managing the complexity of all this?
>
> I'm still not getting where the benefit of having this hierarchy is,
> except that it adds a Java-like complexity and will create
> hard-to-manage churn if a module ever moves.

Regarding the advantages see my previous post, where I put some
motivations. Regarding the difficulties of moving modules around, how
harder is than moving a module around when you have no hierarchy?

--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/

Dietro un grande uomo c'č ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu ą tous ceux que j'aime

Stefano Zacchiroli

unread,
Nov 19, 2008, 8:48:17 AM11/19/08
to caml...@yquem.inria.fr
On Tue, Nov 18, 2008 at 09:50:14PM +0000, Richard Jones wrote:
> > If there is nothing in the Net module (and ignoring the linking issue)
> > you can actually achieve that by using -pack. Just redo the pack on
> > the client whenever it installs a new package in the namespace. No ?

> No because Net isn't necessarily an empty module, nor does it
> magically pull in all the modules underneath it (which would be
> impossible because the Net::* space is constantly changing).

Still, the idea of doing the pack on the client-side is an interesting
one. That way users can install libraries with some kind of metadata
(maybe incorporated in META files) which tell where do they fit into
the hierarchy, and the hierarchy is re-assembled for the client.

BUT

that would horribly break down with checksum assumptions, I believe.

--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'č ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu ą tous ceux que j'aime

_______________________________________________

Daniel Bünzli

unread,
Nov 19, 2008, 9:30:10 AM11/19/08
to OCaml List
> Yes, but that's not a good reason to give up hierarchies completely.
> The advantage of hierarchies is to have less top-level roots, which
> reduce the likelihood of clashes with external libraries.

I think that the name clash problem is overblown. Really.

Would it arise concretly I prefer developer cooperation rather than
have the problem solved beforehand by forcing a bureaucracy on me (and
even the hierarchy cannot prevent the problem completely).

Besides in batteries the maintainers control everything that is below
the Batteries module so it is their duty to avoid clashes in their
name space and would a clash with an external library B occur I can
use Batteries.B to refer to the battery one.

Someone mentionned python's library, if it corresponds to this [1],
then I see no hierarchy there (OTOH nobody tells me that python users
are actually screaming for a hierarchy on their list).

Best,

Daniel

[1] http://docs.python.org/library/

Paolo Donadeo

unread,
Nov 19, 2008, 9:55:38 AM11/19/08
to OCaml mailing list
> Someone mentionned python's library, if it corresponds to this [1], then I
> see no hierarchy there (OTOH nobody tells me that python users are actually
> screaming for a hierarchy on their list).

The Python library hierarchy is very flat, for example all markup
tools are presented into a tree of packages [1] for user convenience.

Nobody actually complains about this situation.


[1] http://docs.python.org/library/markup.html

--
Paolo
~
~
:wq

_______________________________________________

Richard Jones

unread,
Nov 19, 2008, 12:38:01 PM11/19/08
to Stefano Zacchiroli, caml...@yquem.inria.fr
On Wed, Nov 19, 2008 at 02:38:05PM +0100, Stefano Zacchiroli wrote:
> On Tue, Nov 18, 2008 at 12:32:31PM +0000, Richard Jones wrote:
> > On Tue, Nov 18, 2008 at 01:15:39PM +0100, David Teller wrote:
> > > Do you see any better way of managing the complexity of all this?
> >
> > I'm still not getting where the benefit of having this hierarchy is,
> > except that it adds a Java-like complexity and will create
> > hard-to-manage churn if a module ever moves.
>
> Regarding the advantages see my previous post, where I put some
> motivations. Regarding the difficulties of moving modules around, how
> harder is than moving a module around when you have no hierarchy?

Well I guess what I _meant_ to say was that if your modules aren't in
a hierarchy to start with, then you won't be tempted to move them
around the hierarchy :-)

Rich.

--
Richard Jones
Red Hat

_______________________________________________

Stéphane Glondu

unread,
Nov 19, 2008, 1:05:28 PM11/19/08
to OCaml
Stefano Zacchiroli wrote:
> Still, if using it thoroughly we are back to the square 1 of having to
> link everything together, then it is not a viable solution for large
> libraries.

..or maybe it's time to have shared libraries (≠ plugins).


Cheers,

--
Stéphane

Stéphane Glondu

unread,
Nov 19, 2008, 2:02:22 PM11/19/08
to caml...@yquem.inria.fr
Stefano Zacchiroli wrote:
> Still, the idea of doing the pack on the client-side is an interesting
> one. [...]

>
> BUT
>
> that would horribly break down with checksum assumptions, I believe.

It would also mean someone's executables (contents and size) will depend
on the libraries installed during the compilation, which is even worse.

--
Stéphane

Maxence Guesdon

unread,
Nov 19, 2008, 3:11:34 PM11/19/08
to Paolo Donadeo, OCaml mailing list
On Wed, 19 Nov 2008 10:46:24 +0100
"Paolo Donadeo" <p.do...@gmail.com> wrote:

> Couldn't we take inspiration from the Python standard library [1]?
> Python hasn't namespace but is provided with a module system similar
> to OCaml *and* the standard library is really impressive.
>
> And nobody can say Python is a bureaucratic language like Java :-)
>
>
> [1] http://www.python.org/doc/2.5.2/lib/lib.html

Hello,

I think a hierarchy in the documentation is very useful: it helps beginners
understand the big picture about all available modules and help developers
find the functions they look for. But in the code, I think it is really
more convenient to only have one level of "standard" modules.

By the way, I, too, always prefix idents with the module they come from,
(like in List.length) and only "open" a module to use fields and
constructors, so using Data.Containers.List.length or whatever is longer
than "List.length" is not an option. Having a shortcut "List" for
Data.Containers.List does not convince me, because the problem will remain
with modules with no shortcut.

So hierarchy in documentation: yes; in the code: no.

Regards,

--
Maxence Guesdon http://yquem.inria.fr/~guesdon/
Service Expérimentation et Développements https://devel.inria.fr/rocq/
INRIA Paris-Rocquencourt http://www.inria.fr/rocquencourt/

Stefano Zacchiroli

unread,
Nov 19, 2008, 7:15:57 PM11/19/08
to OCaml
On Wed, Nov 19, 2008 at 07:05:17PM +0100, Stéphane Glondu wrote:
> Stefano Zacchiroli wrote:
> > Still, if using it thoroughly we are back to the square 1 of having to
> > link everything together, then it is not a viable solution for large
> > libraries.
>
> ...or maybe it's time to have shared libraries (≠ plugins).

ACK, though that is orthogonal. Even with shared libraries I don't
want to have link relationships with modules whose symbols I don't
use, no matter how the symbols where -pack ed together.

Cheers.

--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/

Dietro un grande uomo c'è ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu à tous ceux que j'aime

Nicolas Pouillard

unread,
Nov 20, 2008, 4:29:09 AM11/20/08
to Maxence Guesdon, Caml_mailing list, Paolo Donadeo
Excerpts from Maxence Guesdon's message of Wed Nov 19 21:11:24 +0100 2008:

> On Wed, 19 Nov 2008 10:46:24 +0100
> "Paolo Donadeo" <p.do...@gmail.com> wrote:
>
> > Couldn't we take inspiration from the Python standard library [1]?
> > Python hasn't namespace but is provided with a module system similar
> > to OCaml *and* the standard library is really impressive.
> >
> > And nobody can say Python is a bureaucratic language like Java :-)
> >
> >
> > [1] http://www.python.org/doc/2.5.2/lib/lib.html
>
> Hello,
>
> I think a hierarchy in the documentation is very useful: it helps beginners
> understand the big picture about all available modules and help developers
> find the functions they look for. But in the code, I think it is really
> more convenient to only have one level of "standard" modules.
>
> By the way, I, too, always prefix idents with the module they come from,
> (like in List.length) and only "open" a module to use fields and
> constructors, so using Data.Containers.List.length or whatever is longer
> than "List.length" is not an option. Having a shortcut "List" for
> Data.Containers.List does not convince me, because the problem will remain
> with modules with no shortcut.
>
> So hierarchy in documentation: yes; in the code: no.

No one (I guess) would recommend you to use fully qualified paths as in
Data.Containers.List.length of course. Data.Containers.List.length is the
external name, made to be well organized not to be quick to type, the way
to use it to open it *OR* to define an internal name for it :

module L = Data.Containers.List

And then use L.length, L.map...

I know that the choice of name qualification, opening, or local modules is
controversial, however I would like to point out that the external name don't
need to be the same than the internal name.

Cheers,

--
Nicolas Pouillard aka Ertai

Richard Jones

unread,
Nov 20, 2008, 5:33:13 AM11/20/08
to Nicolas Pouillard, Maxence Guesdon, Caml_mailing list, Paolo Donadeo
On Thu, Nov 20, 2008 at 10:28:07AM +0100, Nicolas Pouillard wrote:
> No one (I guess) would recommend you to use fully qualified paths as in
> Data.Containers.List.length of course. Data.Containers.List.length is the
> external name, made to be well organized not to be quick to type, the way
> to use it to open it *OR* to define an internal name for it :
>
> module L = Data.Containers.List
>
> And then use L.length, L.map...

I've lost the plot on what problem are we trying to solve .. except
for the original one which is "Windows users are too stupid to use a
packaging system, so let's give them everything in a single
installer". But surely having everyone using privately named modules
is a bad idea? The private names chosen won't be consistent, and they
require a reference back to the top of the code to find out which
module they are really using. Encouraging developers to open modules
is also usually a bad idea, except in very limited circumstances
(hello Printf).

Rich.

--
Richard Jones
Red Hat

_______________________________________________

Stefano Zacchiroli

unread,
Nov 20, 2008, 5:49:26 AM11/20/08
to caml...@yquem.inria.fr
On Thu, Nov 20, 2008 at 10:33:03AM +0000, Richard Jones wrote:
> Encouraging developers to open modules is also usually a bad idea,
> except in very limited circumstances (hello Printf).

Why? You and others failed me to convince of this. Or, better, I'm
sure there are problems with that, but they just show deficiencies
inherited from other parts of the language.

Problem 1) once you open you loose the information where an identifier
comes from. True, but it is a tool deficiency, not an intrinsic
deficiency.

AFAIU Ocamlwizard addresses that [1].

Problem 2) "open Module" is too broad, hence it "splice in" the
current scope all identifiers of Module. I do agree that the
solution of locally defined modules is not a satisfactory solution
for that problem, for the reason mentioned (cumbersome syntax, and
non-uniform choice for the local module names).

The most straightforward solution to this problem to me looks like
providing a syntax equivalent like "from Module import foo, bar"
which selectively imports only some identifiers from a given module.

Cheers.

[1] http://osp.janestcapital.com/files/ocamlwizard.pdf

--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/

Dietro un grande uomo c'č ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu ą tous ceux que j'aime

David Allsopp

unread,
Nov 20, 2008, 6:30:17 AM11/20/08
to Stefano Zacchiroli, caml...@yquem.inria.fr
On 20 November 2008 10:49, Stefano Zacchiroli wrote:
> On Thu, Nov 20, 2008 at 10:33:03AM +0000, Richard Jones wrote:
> > Encouraging developers to open modules is also usually a bad idea,
> > except in very limited circumstances (hello Printf).
>
> Why? You and others failed me to convince of this. Or, better, I'm
> sure there are problems with that, but they just show deficiencies
> inherited from other parts of the language.

Consider

open Array;;
open List;;

(* Hundreds of lines of code *)

length [];;

The code is now is brittle in terms of the order of the open statements at
the top of the file and will fail to compile if they're swapped. Of course,
if you don't care about that kind of subtle refactoring error then open is
completely fine. Personally, I find that kind of brittleness irritating -
and it also has the potential to waste a huge amount of time if you have to
refactor the code.

Whether you find code less readable with or without module names is of
course a matter taste and IIRC, OCaml 3.11 .annot files contain the
necessary information to expand them so there could be a nice editor plugin
to expand or remove module paths...

> > The most straightforward solution to this problem to me looks like
> > providing a syntax equivalent like "from Module import foo, bar"
> > which selectively imports only some identifiers from a given module.

Which, for values only, is of course a trivial camlp4 extension... and could
be generalised to include type declarations and so on with only a little
more work. The .NET languages have a syntax for selectively importing
classes from a namespace rather than the entire namespace (and it's
different from Java's in that you can rename the class while you do it).


David

Daniel Bünzli

unread,
Nov 20, 2008, 6:32:57 AM11/20/08
to OCaml List

Le 20 nov. 08 à 11:49, Stefano Zacchiroli a écrit :

> Problem 1) once you open you loose the information where an identifier
> comes from. True, but it is a tool deficiency, not an intrinsic
> deficiency.

I disagree. Having to invoke a tool to know where an identifier comes
from when I read code involves one more (superfluous IMHO) action. Not
to mention that I do sometimes print code on real paper to read it.

I want to be able to read code without the need of invoking tools
every two lines, thus I try to follow this policy :

- Any non prefixed identifier in a file is defined in that file.

This makes reading and navigating through the code much more easier.
The less there are implicit definitions in my code, the better.

I do not object using open for external libraries that pack some
related _modules_ in a _single level_ hierarchy but I clearly see no
benefit of having to open things to use the standard library
(especially to use something as ubiquituous as lists).

> Problem 2) "open Module" is too broad, hence it "splice in" the
> current scope all identifiers of Module.

This problem doesn't occur if you pack only closely related _modules_
in the module you open.

Best,

Daniel

Richard Jones

unread,
Nov 20, 2008, 6:41:14 AM11/20/08
to Stefano Zacchiroli, caml...@yquem.inria.fr
On Thu, Nov 20, 2008 at 11:49:14AM +0100, Stefano Zacchiroli wrote:
> On Thu, Nov 20, 2008 at 10:33:03AM +0000, Richard Jones wrote:
> > Encouraging developers to open modules is also usually a bad idea,
> > except in very limited circumstances (hello Printf).
>
> Why? You and others failed me to convince of this. Or, better, I'm
> sure there are problems with that, but they just show deficiencies
> inherited from other parts of the language.
>
> Problem 1) once you open you loose the information where an identifier
> comes from. True, but it is a tool deficiency, not an intrinsic
> deficiency.
> AFAIU Ocamlwizard addresses that [1].

Well, it is a tool deficiency, but the fact is that it's a deficiency
we have, and until someone writes the tuareg extension for
ocamlwizard, we'll continue to have this deficiency. (Better not
forget vi users, ocamde users, Eclipse, etc.) 'Course, that fixes the
editor part, but there's still all the other places where OCaml code
can be displayed - eg. on web pages, in version control systems, in
books, .. - so better extend ocamlwizard to those areas too.

[...]


> The most straightforward solution to this problem to me looks like
> providing a syntax equivalent like "from Module import foo, bar"
> which selectively imports only some identifiers from a given module.

Again, Perl gets this mostly right, in that the module developer can
define "modes of use" of the module. In Perl something like:

use CGI qw(:standard);

causes all the "standard" symbols to be imported (as defined by the
module author). Or you can import just the symbols you want. As
befits a dynamic language, the implementation is completely flexible
-- at runtime the module sees the literal string parameter and can
decide to export any combination of symbols it likes based on the
string parameter. An example of how flexible and idiomatic that can
be is shown here:

http://search.cpan.org/dist/CGI.pm/CGI.pm#SPECIAL_FORMS_FOR_IMPORTING_HTML-TAG_FUNCTIONS

Rich.

--
Richard Jones
Red Hat

_______________________________________________

Richard Jones

unread,
Nov 20, 2008, 6:49:08 AM11/20/08
to David Allsopp, caml...@yquem.inria.fr
On Thu, Nov 20, 2008 at 11:29:44AM -0000, David Allsopp wrote:
> Consider
>
> open Array;;
> open List;;
>
> (* Hundreds of lines of code *)
>
> length [];;

Oh god yes, I was bitten by almost this just a few days ago, except my
code was:

open Printf
open Format

(*...*)
printf "I'm trying to debug something\n"

(The file was quite literally not more than 15 lines long, but it took
me a good hour to work out why that printf wasn't printing anything ...)

Rich.

--
Richard Jones
Red Hat

_______________________________________________

Nicolas Pouillard

unread,
Nov 20, 2008, 8:00:46 AM11/20/08
to Richard Jones, Maxence Guesdon, Caml_mailing list, Paolo Donadeo
Excerpts from Richard Jones's message of Thu Nov 20 11:33:03 +0100 2008:

> On Thu, Nov 20, 2008 at 10:28:07AM +0100, Nicolas Pouillard wrote:
> > No one (I guess) would recommend you to use fully qualified paths as in
> > Data.Containers.List.length of course. Data.Containers.List.length is the
> > external name, made to be well organized not to be quick to type, the way
> > to use it to open it *OR* to define an internal name for it :
> >
> > module L = Data.Containers.List
> >
> > And then use L.length, L.map...
>
> I've lost the plot on what problem are we trying to solve .. except
> for the original one which is "Windows users are too stupid to use a
> packaging system, so let's give them everything in a single
> installer". But surely having everyone using privately named modules
> is a bad idea? The private names chosen won't be consistent, and they
> require a reference back to the top of the code to find out which
> module they are really using. Encouraging developers to open modules
> is also usually a bad idea, except in very limited circumstances
> (hello Printf).

Actually having to look at the top of each file (and only the top), is
my favorite option. That's in fact exactly what I already do.

--
Nicolas Pouillard aka Ertai

_______________________________________________

Nicolas Pouillard

unread,
Nov 20, 2008, 8:02:55 AM11/20/08
to David Allsopp, Caml_mailing list
Excerpts from David Allsopp's message of Thu Nov 20 11:29:44 UTC 2008:

> On 20 November 2008 10:49, Stefano Zacchiroli wrote:
> > On Thu, Nov 20, 2008 at 10:33:03AM +0000, Richard Jones wrote:
> > > Encouraging developers to open modules is also usually a bad idea,
> > > except in very limited circumstances (hello Printf).
> >
> > Why? You and others failed me to convince of this. Or, better, I'm
> > sure there are problems with that, but they just show deficiencies
> > inherited from other parts of the language.
>
> Consider
>
> open Array;;
> open List;;
>
> (* Hundreds of lines of code *)
>
> length [];;

That's not a good example, because I consider the shadowing above as
an ambiguity on certain identifiers. Don't confuse short names and ambiguous
names.

> The code is now is brittle in terms of the order of the open statements at
> the top of the file and will fail to compile if they're swapped. Of course,
> if you don't care about that kind of subtle refactoring error then open is
> completely fine. Personally, I find that kind of brittleness irritating -
> and it also has the potential to waste a huge amount of time if you have to
> refactor the code.
>
> Whether you find code less readable with or without module names is of
> course a matter taste and IIRC, OCaml 3.11 .annot files contain the
> necessary information to expand them so there could be a nice editor plugin
> to expand or remove module paths...
>
> > > The most straightforward solution to this problem to me looks like
> > > providing a syntax equivalent like "from Module import foo, bar"
> > > which selectively imports only some identifiers from a given module.
>
> Which, for values only, is of course a trivial camlp4 extension... and could
> be generalised to include type declarations and so on with only a little
> more work. The .NET languages have a syntax for selectively importing
> classes from a namespace rather than the entire namespace (and it's
> different from Java's in that you can rename the class while you do it).

I have a trivial camlp4 extension for that actually if someone is interested,
feel free to ask.

--
Nicolas Pouillard aka Ertai

_______________________________________________

Nicolas Pouillard

unread,
Nov 20, 2008, 8:43:16 AM11/20/08
to David Allsopp, Stefano Zacchiroli, Caml_mailing list
Excerpts from Nicolas Pouillard's message of Thu Nov 20 14:01:53 +0100 2008:

Given the crowd of people wanting it, I've added a link [1] to the camlp4 wiki [2] :)

[1] http://aloxe.inria.fr/darcs/pa_import
[2] http://brion.inria.fr/gallium/index.php/Camlp4_contributions

Ashish Agarwal

unread,
Nov 20, 2008, 9:46:41 AM11/20/08
to Caml List
> Consider
>
> open Array;;
> open List;;

I doubt anyone is recommending this. The module design dictates, to some
extent, whether the module should be opened. Array and List clearly should
not since they have commonly used function names. However, the proposed
Data.Containers certainly should be opened. There is no confusion about
private names in this case. If I am using Batteries, it will be clear which
module "List" refers to. The bureaucracy of writing open statements at the
top of every file would get cumbersome, but that can be avoided by the
proposed short-circuiting.

Stefano Zacchiroli

unread,
Nov 20, 2008, 11:44:22 AM11/20/08
to Caml_mailing list
On Thu, Nov 20, 2008 at 02:41:12PM +0100, Nicolas Pouillard wrote:
> Given the crowd of people wanting it, I've added a link [1] to the
> camlp4 wiki [2] :)

Erm, given the crowd of people wanting it, what about including it in
legacy camlp4? :-)

--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/

Dietro un grande uomo c'è ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu à tous ceux que j'aime

Stefano Zacchiroli

unread,
Nov 20, 2008, 12:55:10 PM11/20/08
to caml...@yquem.inria.fr
On Thu, Nov 20, 2008 at 11:29:44AM -0000, David Allsopp wrote:
> > Why? You and others failed me to convince of this. Or, better, I'm
> > sure there are problems with that, but they just show deficiencies
> > inherited from other parts of the language.
>
> Consider
>
> open Array;;
> open List;;

You are stretching quite a lot what is being proposed. You are
deliberately taking two similar modules (two data structure modules,
which also happen to be really similar ADTs), and opening them
together. Doing that is asking for trouble, the only way to avoid that
is forbidding entirely open, which nobody else has proposed either.

What is being proposed wrt the Batteries hierarchy is to open module
paths, which in most (maybe even all) of the cases just contain other
modules. The potential clashes are related to module pairs, and wont
exhibit the problem you shown, because the clashing modules will not
be union-ed together. One will win over the other.

> > > The most straightforward solution to this problem to me looks like
> > > providing a syntax equivalent like "from Module import foo, bar"
> > > which selectively imports only some identifiers from a given module.
>
> Which, for values only, is of course a trivial camlp4
> extension... and could be generalised to include type declarations

Yes, which is one of the reason while I was proposing it. Still, you
did not to comment on whether such an extension (which we already
have, thanks to Nicolas) would be a satisfying solution for the open
issue, which is basically my position.

Cheers.

--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'č ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu ą tous ceux que j'aime

_______________________________________________

Stefano Zacchiroli

unread,
Nov 20, 2008, 12:56:50 PM11/20/08
to caml...@yquem.inria.fr
On Thu, Nov 20, 2008 at 11:48:57AM +0000, Richard Jones wrote:
> open Printf
> open Format

Let me stress once more that the Batteries hierarchy is not
advertising anything like that. What is "advertising" (modulo syntax
extensions helping out) is to open partial module paths which just
contain other modules. I believe that scenario to be sensibly
different than yours, and less prone to clashes.

Cheers.

--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'č ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu ą tous ceux que j'aime

_______________________________________________

David Teller

unread,
Nov 20, 2008, 4:12:25 PM11/20/08
to OCaml
Dear list,

Feedback from active members of the list (and a few other shy people
who seem to prefer answering off-list:)) seems to indicate that
Batteries shouldn't have a general hierarchies of modules but rather a
flat list of modules with a few submodules here and there, along with a
documentation allowing navigation by topics. While that's not my
personal judgement, I'm willing to go along.

So here's a reworked map of the library, along with a few placeholders
to get an idea of where upcoming modules will fit. Text version follows
and html version available on-line:
http://dutherenverseauborddelatable.wordpress.com/2008/11/20/ocaml-batteries-included-the-hierarchy-reloaded/
While I personally find this solution a little clumsier than the
previous hierarchy, ymmv. Again, feedback is appreciated.


If anyone is willing to work on a solution for linking documentation
from third-party libraries into one transparent source, as suggested by
Richard Jones, please contact me. I'm sure it is feasible, with a
(un)healthy dose of JavaScript, but I'm not sure that current members of
Batteries have enough brainpower available to work on this on top of
Batteries.

Cheers,
David

Batteries (pack)
1. Standard (automatically opened)
2. Legacy
A. Arg
B. Array
C. ...
3. Future (things that should become standard eventually)
A. Lexers
I. C
II. OCaml

===== I. Control =====
4. Exceptions
5. Return
6. Monad (Interfaces for monadic operations )
==== I.1. Concurrency ====
7. Concurrency (Interfaces for concurrency operations)
=== I.1.i. Built-in threads ===
8. Condition
9. Event
10. Mutex
11. RMutex
12. Thread
13. Threads (A module containing aliases to Condition, Event...)
=== I.1.ii. coThreads ===
14. CoCondition
15. CoEvent
16. CoMutex
17. CoRMutex
18. CoThread
19. CoThreads (as Threads but with implementations coming from
coThreads)
=== I.1.iii. Shared memory ===
20. Shm_* (Placeholders)
===== II. IO =====
21. IO
A. BigEndian
22. Codec (common interfaces for compressors/decompressors)
23. GZip
24. Bz2
25. Zip
26. Transcode (Unicode transcoding)
===== III. Mutable containers =====
27. Array
A. Cap
I. ExceptionLess
II. Labels
B. ExceptionLess
a. Labels
28. Bigarray
A. Array1
B. Array2
a. Array3
29. Dllist
30. Dynarray
31. Enum
A. ExceptionLess
a. Labels
32. Global
33. Hashtbl
A. Make
I. ExceptionLess
i. Labels


===== IV. Persistent containers ======


34. Lazy
35. List
A. ExceptionLess
B. Labels
36. Map
A. Make
I. ExceptionLess
II. Labels
37. Option
A. Labels
38. PMap
39. PSet
40. RefList
A. Index
41. Queue
42. Ref
43. Set
A. Make
I. ExceptionLess
II. Labels
44. Stack
45. Stream
===== V. Data =====
46. Unit
==== V.1. Logical ====
47. Bool
48. BitSet
==== V.2. Numeric ====
49. Numeric (Interfaces for number-related stuff)
50. Big_int
51. Common
52. Complex
53. Float
54. Int
55. Int32
56. Int64
57. Native_int
58. Num
59. Safe_float (placeholder)
60. Safe_int
==== V.3 Textual data ====
61. Text (Definition of text-related interfaces)
62. Buffer
63. Char
64. UTF8
65. Rope
66. UChar
67. String
68. StringText (A module containing aliases to String and modified
Char)
69. RopeText (As StringText but with implementations from Rope and
UChar
70. UTF8Text (As StringText but with implementations from UTF8 and)
UChar
A. Labels

===== V. Distribution-related stuff =====
71. Packages
72. Compilers
===== VI. Internals =====
73. Gc
74. Modules
75. Oo
A. Private
76. Weak
A. Make

===== VIII. Network (placeholders) =====
77. URL
78. Netencoding
A. Base64
B. QuotedPrintable
a. Q
b. URL
A. Html

==== VIII.1. Http ====
79. Http
80. Http_client
81. Cgi_*
82. Httpd_*
83. MIME
==== VIII.2. Ftp ====
84. Ftp_client
==== VIII.3. Mail ====
85. Netmail
86. Pop
87. Sendmail
88. Smtp
==== VIII.4. Generic server ====
89. Netplex_*
==== VIII.5. RPC ====
90. Rpc_*
==== VIII.6. Languages ====
91. Genlex
92. Lexing
93. CharParser
94. UCharParser
95. ParserCo
A. Source
96. Parsing
97. Format
98. Printf
99. Str
100. PCRE (placeholder)
101. Scanf
A. Scanning
102. SExpr
===== IX. System =====
103. Arg
104. File
105. OptParse
A. Opt
a. OptParser
b. StdOpt
106. Path
107. Shell
108. Unix
A. Labels
109. Equeue
X. Unclassified
110. Digest
111. Random
A. State
112. Date (placeholder)

On Tue, 2008-11-18 at 10:56 +0100, David Teller wrote:
> For this purpose, I have posted a
> tree of the current hierarchy on my blog [1].
>
> [1]
>
http://dutherenverseauborddelatable.wordpress.com/2008/11/18/batteries-hierarchy/


--
David Teller-Rajchenbach
Security of Distributed Systems
http://www.univ-orleans.fr/lifo/Members/David.Teller
Angry researcher: French Universities need reforms, but the LRU act brings liquidations.

Daniel Bünzli

unread,
Nov 20, 2008, 6:19:27 PM11/20/08
to OCaml

Le 20 nov. 08 à 22:12, David Teller a écrit :

> If anyone is willing to work on a solution for linking documentation
> from third-party libraries into one transparent source, as suggested
> by Richard Jones, please contact me.

I'm not sure I understand what you want to acheive. If it is only a
documentation issue cannot that be done with ocamldoc's -dump and -
load ?

> Batteries (pack)
> 1. Standard (automatically opened)

Is this Pervasives ? If it is I think the latter name is more
descriptive.

> 13. Threads (A module containing aliases to Condition, Event...)

> 19. CoThreads (as Threads but with implementations coming from
> coThreads)

If Threads and CoThreads are really semantically compatible I think
that your idea of only having everything in Threads and CoThread is
better and sufficient (i.e. top-level Condition, CoCondition, etc.
should be dropped). Advise the users to open Threads/Cothreads to use
the modules (or functorize their code on Concurrency). This allows to
quickly switch from one implementation to the other by changing the
toplevel open directive. With the current proposal users may be
tempted to use Condition directly, and what happens if some have used
Condition and others CoCondition in their modules and we suddenly try
to use them toghether ?

> While I personally find this solution a little clumsier than the
> previous hierarchy, ymmv.

Of course when you look it as a long list it does, but that's a
presentation issue. This proposal is much more convenient to use in
your code and that's what eventually matters (at least to me). Thanks
for the new proposal.

Best,

Daniel

Eliot Handelman

unread,
Nov 20, 2008, 9:56:12 PM11/20/08
to Caml_mailing list

In order to catch array access violations, it seems necessary to do
something like this:

exception Array_access of int

let test i =
try
[||].(i)
with
Invalid_argument "index out of bounds" -> raise (Array_access i)


The problem is that this test is dependent on a literal string match of
"index out of bounds." If
I accidentally write something like "index out out bounds" (an extra
space between index & out)
then, of course, the above code will not raise Array_access, and indeed
it may be quite hard for me to
discover what went wrong. This seems inconsistent with the idea of
strong typing.


Is there some guarantee in the language that in future releases, the
"index out of bounds" message will remain
exactly what it currently is? My problem is not confined to this
particular case, but rather
with all exceptions that have to be matched on string literals, in
particular because I'm a
rather poor typist. What's the common practice here? Would it not be
better
for Ocaml to have built-in exceptions (not string literals) for things
of this importance?

best,

-- eliot

Daniel Bünzli

unread,
Nov 21, 2008, 2:40:54 AM11/21/08
to OCaml List

Le 21 nov. 08 à 03:56, Eliot Handelman a écrit :

> In order to catch array access violations,

Don't do that. This was already raised on the list but I cannot find
the reference anymore. One argument that comes to mind is if one day
you need optimal performance you won't be able to compile with -
unsafe. Invalid_argument exceptions are programming errors and usually
you should not try to catch them [1].

> My problem is not confined to this particular case

I agree, the problem also exists with Failure and I already
encountered it (full Buffer). In that case the safest route is to wrap
with a handler the greatest body of code that you know can only raise
Failure because of the particular condition you try to catch and raise
your own exception on any Failure (see for example the module Buffer
at the very end of this file [2]).

> Would it not be better for Ocaml to have built-in exceptions (not
> string literals) for things of this importance?


Of course it would.

Best,

Daniel

[1] http://caml.inria.fr/pub/ml-archives/caml-list/2007/10/e6683fd700e87f214c757ecaaa4f8ede.fr.html
[2] http://erratique.ch/software/xmlm/repo/src/xmlm.ml

David Teller

unread,
Nov 21, 2008, 4:35:39 AM11/21/08
to Daniel Bünzli, OCaml
On Fri, 2008-11-21 at 00:18 +0100, Daniel Bünzli wrote:
> Le 20 nov. 08 à 22:12, David Teller a écrit :
>
> > If anyone is willing to work on a solution for linking documentation
> > from third-party libraries into one transparent source, as suggested
> > by Richard Jones, please contact me.
>
> I'm not sure I understand what you want to acheive. If it is only a
> documentation issue cannot that be done with ocamldoc's -dump and -
> load ?

No, it's not. You cannot ask everyone to regenerate all the
documentation of every single package they have as often as they install
new packages. The problem is linking already-generated documentation
post-facto.

> > Batteries (pack)
> > 1. Standard (automatically opened)
>
> Is this Pervasives ? If it is I think the latter name is more
> descriptive.

It is the replacement for [Pervasives], indeed. And I'm pretty sure
that, for beginners, [Pervasives] is more confusing than [Standard].
Since it's automatically opened anyway, most people won't need to know
the name.

> > 13. Threads (A module containing aliases to Condition, Event...)
> > 19. CoThreads (as Threads but with implementations coming from
> > coThreads)
>
> If Threads and CoThreads are really semantically compatible I think
> that your idea of only having everything in Threads and CoThread is
> better and sufficient (i.e. top-level Condition, CoCondition, etc.
> should be dropped). Advise the users to open Threads/Cothreads to use
> the modules (or functorize their code on Concurrency). This allows to
> quickly switch from one implementation to the other by changing the
> toplevel open directive. With the current proposal users may be
> tempted to use Condition directly, and what happens if some have used
> Condition and others CoCondition in their modules and we suddenly try
> to use them toghether ?

Well, that was my argument for hierarchies. Stop stealing my
arguments :)

More seriously, sure.

> > While I personally find this solution a little clumsier than the
> > previous hierarchy, ymmv.
>
> Of course when you look it as a long list it does, but that's a
> presentation issue. This proposal is much more convenient to use in
> your code and that's what eventually matters (at least to me). Thanks
> for the new proposal.

Well, I've started working on a new generation of documentation
generation should make navigation by topics feasible. I'll try and have
a prototype within 1-2 weeks.

> Best,
>
> Daniel

Cheers,
David


--
David Teller-Rajchenbach
Security of Distributed Systems
http://www.univ-orleans.fr/lifo/Members/David.Teller
Angry researcher: French Universities need reforms, but the LRU act brings liquidations.

_______________________________________________

Christophe TROESTLER

unread,
Nov 21, 2008, 4:53:12 AM11/21/08
to el...@colba.net, OCaml Mailing List
On Thu, 20 Nov 2008 21:56:32 -0500, Eliot Handelman wrote:
>
> let test i =
> try
> [||].(i)
> with
> Invalid_argument "index out of bounds" -> raise (Array_access i)
>
>
> The problem is that this test is dependent on a literal string match of
> "index out of bounds." If
> I accidentally write something like "index out out bounds" (an extra
> space between index & out)

These strings are for user information, you should write

let test i =
try
[||].(i)
with

Invalid_argument _ -> raise (Array_access i)

IMHO, in this case, you should rather make sure you do not perform
accesses outside the array bounds.

Cheers,
ChriS

Stefano Zacchiroli

unread,
Nov 23, 2008, 5:32:14 AM11/23/08
to caml...@yquem.inria.fr
On Wed, Nov 19, 2008 at 05:37:51PM +0000, Richard Jones wrote:
> > Regarding the advantages see my previous post, where I put some
> > motivations. Regarding the difficulties of moving modules around, how
> > harder is than moving a module around when you have no hierarchy?
> Well I guess what I _meant_ to say was that if your modules aren't in
> a hierarchy to start with, then you won't be tempted to move them
> around the hierarchy :-)

.. which sounds like cheating :)

--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'č ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu ą tous ceux que j'aime

_______________________________________________

Stefano Zacchiroli

unread,
Nov 23, 2008, 5:37:38 AM11/23/08
to caml...@yquem.inria.fr
On Thu, Nov 20, 2008 at 12:31:47PM +0100, Daniel Bünzli wrote:
> I disagree. Having to invoke a tool to know where an identifier comes
> from when I read code involves one more (superfluous IMHO) action. Not
> to mention that I do sometimes print code on real paper to read it.

Fair enough.

Once more though, the annoyance you are pointing out seems to me to be
solved by a syntax extension (which we now have) which enables you to
selectively open specific identifiers from a module, instead of only
enabling to open all of it.

> I do not object using open for external libraries that pack some
> related _modules_ in a _single level_ hierarchy but I clearly see no
> benefit of having to open things to use the standard library
> (especially to use something as ubiquituous as lists).

Well, given that Batteries is de facto extending the standard library
to include external library, the distinction between stdandard and
external it is kind of blurred in this case.

Cheers.

--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'č ..| . |. Et ne m'en veux pas si je te tutoie

sempre uno zaino ...........| ..: |.... Je dis tu ŕ tous ceux que j'aime

Stefano Zacchiroli

unread,
Nov 23, 2008, 5:38:29 AM11/23/08
to caml...@yquem.inria.fr
On Thu, Nov 20, 2008 at 11:41:03AM +0000, Richard Jones wrote:
> > The most straightforward solution to this problem to me looks like
> > providing a syntax equivalent like "from Module import foo, bar"
> > which selectively imports only some identifiers from a given module.
>
> Again, Perl gets this mostly right, in that the module developer can
> define "modes of use" of the module. In Perl something like:
>
> use CGI qw(:standard);

That's interesting, but requires support in all involved library. Do
you know how CPAN authors are required to annotate what is "standard"
and what is not?

Cheers.

--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'č ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu ą tous ceux que j'aime

_______________________________________________

Richard Jones

unread,
Nov 23, 2008, 6:01:44 AM11/23/08
to Stefano Zacchiroli, caml...@yquem.inria.fr
On Sun, Nov 23, 2008 at 11:38:17AM +0100, Stefano Zacchiroli wrote:
> On Thu, Nov 20, 2008 at 11:41:03AM +0000, Richard Jones wrote:
> > > The most straightforward solution to this problem to me looks like
> > > providing a syntax equivalent like "from Module import foo, bar"
> > > which selectively imports only some identifiers from a given module.
> >
> > Again, Perl gets this mostly right, in that the module developer can
> > define "modes of use" of the module. In Perl something like:
> >
> > use CGI qw(:standard);
>
> That's interesting, but requires support in all involved library. Do
> you know how CPAN authors are required to annotate what is "standard"
> and what is not?

It's left to the module authors to define how these labels work,
although there are a lot of idiomatic uses.

Rich.

--
Richard Jones
Red Hat

_______________________________________________

Benedikt Grundmann

unread,
Dec 19, 2008, 6:00:21 AM12/19/08
to David Teller, OCaml
Somehow I forgot reply back when you posted this reply. And I was just
reminded when I read this:

"Batteries is meant to serve the following purposes:
[snip]
provide consistent abstractions and APIs for otherwise independent libraries.
"

on

http://wiki.cocan.org/events/europe/ocamlmeetinggrenoble2009

How can you expect to provide consistent abstractions if you are
not willing to make those decisions?

Cheers,

Bene

2008/11/18 David Teller <David....@univ-orleans.fr>:
> Ok, that's an interesting point. Now, we just need to all agree on one
> standard :)
>
> On Tue, 2008-11-18 at 12:28 +0000, Benedikt Grundmann wrote:
>> > Do you see any better way of managing the complexity of all this?
>> Yes don't introduce it at all, make a decision to use or not use labels
>> and stick with it. Similarly make a decision to use or not use exceptions
>> as the "default", suffix / rename alternative functions as appropriate
>> (consistently). Consistency is a big win. Not only as it speeds you up
>> when you read/modify other people's code it also reduces the amount
>> of decisions you have to do when writing new code.
>>
>> http://ocaml.janestreet.com/?q=node/28
>>
>> Cheers,
>>
>> Bene


>>
> --
> David Teller-Rajchenbach
> Security of Distributed Systems
> http://www.univ-orleans.fr/lifo/Members/David.Teller
> Angry researcher: French Universities need reforms, but the LRU act brings liquidations.
>
>

--
Calvin: I try to make everyone's day a little more
surreal.

(From Calvin & Hobbes)

David Teller

unread,
Jan 5, 2009, 3:20:28 PM1/5/09
to Benedikt Grundmann, OCaml
Hi Benedikt,

You're right, we should make this kind of decision. For the moment, we
are focusing on different issues (e.g. standardising I/O, enumerations,
module names, etc), in an effort to obtain a base relatively fast,
something which could be tested both with existing code and new
applications. It is our hope that this will yield enough interest for
people to comment and discuss policies regarding exceptions, labels,
etc.

Cheers,
David

On Fri, 2008-12-19 at 11:00 +0000, Benedikt Grundmann wrote:
> Somehow I forgot reply back when you posted this reply. And I was just
> reminded when I read this:
>
> "Batteries is meant to serve the following purposes:
> [snip]
> provide consistent abstractions and APIs for otherwise independent libraries.
> "
>
> on
>
> http://wiki.cocan.org/events/europe/ocamlmeetinggrenoble2009
>
> How can you expect to provide consistent abstractions if you are
> not willing to make those decisions?
>
> Cheers,
>
> Bene

--

David Teller-Rajchenbach
Security of Distributed Systems
http://www.univ-orleans.fr/lifo/Members/David.Teller

Latest News of French Research: System being liquidated. Researchers
angry.

0 new messages