[Caml-list] [ANN] OCaml Reins 0.1 - Persistent Data Structure Library
Mike Furr
I'm happy to announce the first source release of the OCaml Reins data
structure library available at:
http://ocaml-reins.sourceforge.net
This project started as an "OCaml Summer Project" and is now continuing
its development on sourceforge. The library already contains several
implementations of persistent data structures and will continue to grow
(possibly adding ephemeral data structures at some point if there's
interest).
Features of this release include:
* List data types:
o Single linked lists (compatible with the standard library type)
o O(1) catenable lists
o Acyclic double linked lists
o Random access lists with O(1) hd/cons/tl and O(log n)
lookup/update
* Double ended queues
* Sets/Maps with both polymorphic and monomorphic keys/values
o AVL
o Red/Black
o Big-endian Patricia
o Splay
* Heaps:
o Binomial
o Skew Binomial
* Zipper style cursor interfaces
* Persistent, bi-directional, cursor based iterators (currently only
for lists and sets)
* All standard types hoisted into the module level (Int, Bool, etc...)
* A collection of functor combinators to minimize boilerplate
(e.g., constructing compare or to_string functions)
* Quickcheck testing framework
o Each structure provides a gen function that can generate a random
instance of itself
* Completely safe code. No -unsafe or references to Obj.*
* Consistent function signatures. For instance, all fold functions
take the accumulator in the same position.
* All operations use no more than O(log n) stack space (except for a
few operations on splay trees which currently have O(log n) expected
time, but O(n) worst case)
Cheers,
-Mike
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Daniel Bünzli
Is it possible to pick a single module/datastructure an embed it in
ones project ? Or is this a "cathedral like" library ?
And what's the licence ?
Best and thanks for sharing the code,
Daniel
Mike Furr
> Is it possible to pick a single module/datastructure an embed it in ones
> project ? Or is this a "cathedral like" library ?
I've deliberately tried to unify many of the conventions that people
have used to allow users to easily change between data structure
implementations. A major goal of the project is that you don't have to
rely on downloading some random .ml/.mli from someone's webpage and
embed it your project. Instead you just use the reins library and only
have to worry about updating one piece of software for updates/ bug fixes.
As for being a cathedral vs bazaar, I am very open to (and hoping to
receive!) contributions from others. There have been several
interesting data structures developed by members of the OCaml community
and I would happily give any developer access to the project that wished
to contribute their work.
> And what's the licence ?
Oops, thanks for reminding me. The code is under the LGPL v2.1
Cheers,
-Mike
Daniel Bünzli
Le 25 sept. 07 à 21:30, Mike Furr a écrit :
> I've deliberately tried to unify many of the conventions that
> people have used to allow users to easily change between data
> structure implementations. A major goal of the project is that you
> don't have to rely on downloading some random .ml/.mli from
> someone's webpage and embed it your project. Instead you just use
> the reins library and only have to worry about updating one piece
> of software for updates/ bug fixes.
While I really like having a central repository and unification of
conventions I do like cherry-picking and embedding my modules.
Nevertheless I understand and respect your point, maybe it is a
matter of opinion/taste. But I'll explain why I like picking.
There are libraries that need several tightly dependent modules to
provide their service and they should be distributed as libraries
since use of one module implies use of the others -- note however
that sometimes this is by misdesign e.g. everything can actually be
easily and conveniently packed in a single module or some parts can
be functorized away to split the distribution.
But there are also focused services that can be provided as single,
pure caml, modules and these should IMHO be distributed as single
modules. I try -- as much as possible -- to embed and statically link
dependencies in my projects. Of course if there is a bug in one
dependency it means more work for me. But the thing is that by doing
so I put the burden and complexity of building my project on me and
not on my users. If I do not do so, this burden may be aleviated (or
not ?) by using a package management system but this solution messes
the environment of my users.
There are a lot of useful reusable modules in many projects like
extlib, ocamlnae, etc, however they are all tied to their
"cathedral" [1] even though these ties are sometimes very small and
could be functorized away (or dropped altogheter). Now if I _just_
want to use the Unzip module of extlib, the Cf_tai64 module of
ocamlnae and your Patricia set. Why do I have to install these three
whole libraries whereas I could simply integrate these three modules
in my project ?
Having to install them not only puts a burden on potential users, but
it also puts a burden on me when I suddenly start to hack my project
in another environment which has not all the dependencies installed.
In the end I think module integration makes me loose less time
because once the build system is setup I have to think about it much
less at later times. The alternative solution of embedding the whole
libraries is the start of bloat. It also hides the real dependencies
which makes it harder for new commers to approach the code base.
I think developers of reusable components should try to strive for
*focused*, independent modules, even if it means functorization and
convention. Eventually it makes it easier for developers to integrate
these modules and exchange specific parts if some module poses a
problem or better alternatives popup.
And while certainly more could be done to try to simplify this style
of development (maybe something along the lines of caml-get), I have
to say that now that we have been provided with ocamlbuild, the task
of integrating pure caml modules is really a breeze -- even if you
want to relegate dependencies to a specific directory.
Again I'm not for embedding everything in projects, sometimes it is
unrealistic. But for exemple I'd systematically do this for data
structures, unless of course they are in the standard distribution.
Best and thanks again for the work,
Daniel
[1] btw this was my point about the cathedral, many modules that you
have to take as a one-stop-shop-it-all even if you care only for one,
not the development style.
Sylvain Le Gall
>
> Le 25 sept. 07 à 21:30, Mike Furr a écrit :
>
> it also puts a burden on me when I suddenly start to hack my project
> in another environment which has not all the dependencies installed.
> In the end I think module integration makes me loose less time
> because once the build system is setup I have to think about it much
> less at later times. The alternative solution of embedding the whole
> libraries is the start of bloat. It also hides the real dependencies
> which makes it harder for new commers to approach the code base.
>
I think you should take a look at godi! (which allow you to
download/install all your dependencies at once).
There is also linux distribution which have a good support of ocaml
languages (debian/redhat...)
I hope that a debian package will be released soon!
Anyway, embeding any external code into your project is a nightmare for
security/maintenance in the long term... I would avoid this solution if
i want things that doesn't have problem.
Regards,
Sylvain Le Gall
David Teller
library ? Right now, I'm thinking about ExtLib, OcamlNet, ulex,
CamlImages, and LablGTK, but others might be nice, too. I believe a
stronger standard library would make miracles for beginners.
Cheers,
David
On Tue, 2007-09-25 at 14:53 -0400, Mike Furr wrote:
> I'm happy to announce the first source release of the OCaml Reins data
> structure library available at:
>
> http://ocaml-reins.sourceforge.net
>
> This project started as an "OCaml Summer Project" and is now continuing
> its development on sourceforge. The library already contains several
> implementations of persistent data structures and will continue to grow
> (possibly adding ephemeral data structures at some point if there's
> interest).
_______________________________________________
skaller
> Le 25 sept. 07 à 21:30, Mike Furr a écrit :
> Having to install them not only puts a burden on potential users, but
> it also puts a burden on me when I suddenly start to hack my project
> in another environment which has not all the dependencies installed.
indeed, reins requires findlib, Omake, and Ounit.. Omake doesn't
configure on my system (it can't find unixdll.so, which seems
to be required only to build the docs).
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
skaller
> On 25-09-2007, Daniel Bünzli <daniel....@epfl.ch> wrote:
> Anyway, embeding any external code into your project is a nightmare for
> security/maintenance in the long term... I would avoid this solution if
> i want things that doesn't have problem.
Embedding source is generally better, unfortunately,
if you're distributing a product for other people to use.
This is because
1. people use crappy build systems
2. have unnecessary dependencies
3. don't maintain their code 'reliably' and/or don't provide
repository (write) access
Apart from Python and Ocaml itself (and an unfortunate need for
a C++ compiler) my product has no external dependencies,
and it builds on all platforms I know about *as shipped*.
My rule for Ocaml code is simple: either it is shipped in the
standard distro or it is shipped in mine. People have enough
problems getting Ocaml and C++ to work without adding second order
dependencies.
Hopefully (1) will eventually be solved by ocamlbuild, however
it still isn't really reliable/do the right thing, and doesn't
work on Windows. We provide it as an option: if ocamlbuild is
found it is used -- I have to keep turning it off though,
because it isn't compatible with our other build scripts
(fails sanitise checks, probably our _tags etc aren't right yet).
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
_______________________________________________
Maxence Guesdon
Daniel Bünzli <daniel....@epfl.ch> wrote:
>
> Le 25 sept. 07 à 21:30, Mike Furr a écrit :
>
> > I've deliberately tried to unify many of the conventions that
> > people have used to allow users to easily change between data
> > structure implementations. A major goal of the project is that you
> > don't have to rely on downloading some random .ml/.mli from
> > someone's webpage and embed it your project. Instead you just use
> > the reins library and only have to worry about updating one piece
> > of software for updates/ bug fixes.
>
> While I really like having a central repository and unification of
> conventions I do like cherry-picking and embedding my modules.
> Nevertheless I understand and respect your point, maybe it is a
> matter of opinion/taste. But I'll explain why I like picking.
> [...]
Hello,
You might be interested in caml-get:
http://pauillac.inria.fr/~guesdon/camlget.en.html
Regards,
Maxence
Daniel Bünzli
Le 26 sept. 07 ą 01:33, Sylvain Le Gall a écrit :
> Anyway, embeding any external code into your project is a nightmare
> for
> security/maintenance in the long term... I would avoid this
> solution if
> i want things that doesn't have problem.
As I explained I don't think so. In the best case you delegate the
nightmare to the user. Can you explain exactly where the nightmare is
with embedding ? If the distributed modules you integrate are
labelled with a version, I don't see it. The work needed for
maintenance in the long term is exactly the same, except only the
developer has to care. As for security updates in ocaml, you cannot
anyway rely on dynamic linking. Which I see as a good thing, for
applications dynamic linking creates more problems than it solves and
should be avoided most of the time (except of course for system
libraries).
Best,
Daniel
skaller
> On Wed, 26 Sep 2007 00:16:08 +0200
> You might be interested in caml-get:
> http://pauillac.inria.fr/~guesdon/camlget.en.html
>
This is actually a nice idea, however it depends on contributors
documenting with the custom ocamldoc tags. BTW, it isn't clear
how or even if it handles licence issues. BTW2: any thoughts for
ocamlbuild support, i.e. docs on how to make this an automatic
byproduct of building a supported source?
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
_______________________________________________
Daniel Bünzli
Le 26 sept. 07 à 09:03, Maxence Guesdon a écrit :
> You might be interested in caml-get:
Yes, I mentioned it in my email. This is a good idea and something
along this line would be useful. But before I start using it, it
should drop its dependency on cameleon, integrate well with
ocamlbuild (both should be thought and designed toghether) and make
it in the standard distribution.
I also think it should be simplified. I'd take the module as the only
atomic unit. This would avoid things like trying to reimplement the
notion of signature and interface in comments (@cgtype,@cgname). For
me such a tool should *just* allow me to get modules, keep track of
their origin, version and license. The tool should not try to look
into modules or hack them by appending snippets of code to other
modules (I don't really care for the "little" functions, I care for
reusable abstractions). As for the distribution, in the simplest
case, it should be as easy as putting a .ml and .mli file on a web
server with an approriate @cgversion and @cglicense in the comments.
Best,
Daniel
skaller
> Le 26 sept. 07 à 01:33, Sylvain Le Gall a écrit :
> As for security updates in ocaml, you cannot
> anyway rely on dynamic linking. Which I see as a good thing, for
> applications dynamic linking creates more problems than it solves and
> should be avoided most of the time (except of course for system
> libraries).
There is a tradeoff between reliability and ease of maintenance
such that for many larger applications dynamic linking is actually
*more* reliable.
Would you really recommend statically linking your whole desktop
against the Linux kernel?? :))
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
_______________________________________________
Daniel Bünzli
Le 26 sept. 07 à 10:19, skaller a écrit :
> On Wed, 2007-09-26 at 09:22 +0200, Daniel Bünzli wrote:
>> As for security updates in ocaml, you cannot
>> anyway rely on dynamic linking. Which I see as a good thing, for
>> applications dynamic linking creates more problems than it solves and
>> should be avoided most of the time (except of course for system
>> libraries).
>
> There is a tradeoff between reliability and ease of maintenance
> such that for many larger applications dynamic linking is actually
> *more* reliable.
>
> Would you really recommend statically linking your whole desktop
> against the Linux kernel?? :))
Did you read the last parenthesis of my email ? Libraries that are
guaranteed to be provided with a particular version of an os should
be dynamically linked in, but the rest should be mostly static.
Best,
Daniel
Maxence Guesdon
skaller <ska...@users.sourceforge.net> wrote:
> On Wed, 2007-09-26 at 09:03 +0200, Maxence Guesdon wrote:
> > On Wed, 26 Sep 2007 00:16:08 +0200
>
> > You might be interested in caml-get:
> > http://pauillac.inria.fr/~guesdon/camlget.en.html
> >
>
> This is actually a nice idea, however it depends on contributors
> documenting with the custom ocamldoc tags. BTW, it isn't clear
> how or even if it handles licence issues.
It does not handle licences yet. The idea is to add a tag to indicate the
licence name and a (web) reference to it.
> BTW2: any thoughts for
> ocamlbuild support, i.e. docs on how to make this an automatic
> byproduct of building a supported source?
Why not, it's just an ocamldoc command to run.
Regards,
--
Maxence Guesdon http://yquem.inria.fr/~guesdon/
Service Expérimentation et Développements https://devel.inria.fr/rocq/
INRIA Paris-Rocquencourt http://www.inria.fr/rocquencourt/
skaller
> Le 26 sept. 07 à 10:19, skaller a écrit :
> > Would you really recommend statically linking your whole desktop
> > against the Linux kernel?? :))
>
> Did you read the last parenthesis of my email ?
Yes, I am just moving the 'dividing line' you make closer
to reality, i.e. most major application on all platforms
these days use dynamic linking quite extensively ..
and they do it because it is MORE reliable on that
scale.
> Libraries that are
> guaranteed to be provided with a particular version of an os should
> be dynamically linked in, but the rest should be mostly static.
Define 'os'! Do you include X windows as part of the OS?
How about GUI support eg GTK? (Win32 has kernel support for
parts of the GUI stuff). What about OpenGL libs?
You could ask, eg Debian developers how they'd go maintaining
the Debian archive without dynamic linkage ..
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
_______________________________________________
Daniel Bünzli
Le 26 sept. 07 à 10:58, skaller a écrit :
> Define 'os'! Do you include X windows as part of the OS?
> How about GUI support eg GTK? (Win32 has kernel support for
> parts of the GUI stuff). What about OpenGL libs?
Yes that's part of it. Any can be part of it as long as I can tell an
*end-user* my program will work out of the box on a stock install of
say osx 10.4 or Vista or Debian, i.e. without needing to install a
dozen dynlinked libraries.
Daniel
Vityok
It looks like my yet-not-finished Fixed Records Database project has
something in common with Reins...
-------
With best regards,
Victor
Daniel Bünzli
Le 26 sept. 07 à 10:53, Maxence Guesdon a écrit :
> It does not handle licences yet. The idea is to add a tag to
> indicate the
> licence name and a (web) reference to it.
Web links may be too brittle for these kind of things. Why not
support well-known, huge, licenses internally (a copy is embedded in
caml-get to produce it if needed) via a key, let's say '@cglicense
gpl3' and custom licenses via '@cglicense text of the license', (i.e.
the legaleze is embedded in the comment).
Best,
Daniel
Sylvain Le Gall
>
> Le 26 sept. 07 à 01:33, Sylvain Le Gall a écrit :
>
>> Anyway, embeding any external code into your project is a nightmare
>> for
>> security/maintenance in the long term... I would avoid this
>> solution if
>> i want things that doesn't have problem.
>
> As I explained I don't think so. In the best case you delegate the
> nightmare to the user. Can you explain exactly where the nightmare is
> with embedding ? If the distributed modules you integrate are
> labelled with a version, I don't see it. The work needed for
> maintenance in the long term is exactly the same, except only the
> developer has to care. As for security updates in ocaml, you cannot
> anyway rely on dynamic linking. Which I see as a good thing, for
> applications dynamic linking creates more problems than it solves and
> should be avoided most of the time (except of course for system
> libraries).
>
OK, let me make myself clear, about the fact that embedding things is
the worst case. Let consider library libA and program progB, progB
requires libA:
* libA is embedded inside progB:
** libA do a minor bug correction -> you need to re-release progB even if
nothing has changed inside it
** you make a bug correction in libA, as many developers you don't have
time to submit bug upstream -> libA upstream and libA embedded begin to
be different
* libA is not embedded inside progB:
** libA do a minor bug correction -> progB just need to be recompiled
(no release), easy to do automatically with a dependency tracking
** you make a bug correction in libA -> you have no choice than to
report it to upstream libA, because your bug correction will disappear
in next libA release
Unfortunately i have seen projects (like mldonkey) which has embedded a
lot of libraries. The result of this is that you only creates branches,
with no real feedback to upstream author. This is a shame, it
duplicates works of bug correction for libA upstream and progB.
It is funny, because when you see other big languages -- which are
working very well -- like Perl, they all try to avoid embedding libraries!
The only real needs is to have something that automatically
download/build/install dependency!
AND WE HAVE IT: godi!
Or if you want things more distro based: debian...
Off course, as long as you will think that you must embed everything, you
won't need to learn godi (which is very simple) and you won't evolve on
this point.
I really do invite you to try godi/ocamlfind?
These tools are really great and are a solution for the kind of problems
you describe..
Regards,
Sylvain Le Gall
Jim Miller
> It is funny, because when you see other big languages -- which are
> working very well -- like Perl, they all try to avoid embedding libraries!
>
> The only real needs is to have something that automatically
> download/build/install dependency!
>
> AND WE HAVE IT: godi!
>
> Or if you want things more distro based: debian...
This is a fascinating discussion because it hits at one of the core problems
I have with OCaml. Almost all of my work is done on systems that are not
connected to any sort of network and all software that I install on the
systems has to be approved. I have no choice but to burn CD/DVD's with
modules to install.
I personally also do the static linking against any library that doesn't
come standard with the operating system. Typically the ONLY libraries that
I don't statically link are libc (I don't write UI applications so X isn't
an issue).
In the world I work in, we are actually moving away from system shared
libraries because we cannot count on each of the different installations
doing their system maintenance properly. This has led us into DLL hell on
many of the systems we install to. Because we don't have any administrator
access to these machines, and NEVER will, we simply distribute EVERYTHING we
need, sometimes source, sometimes compiled binaries for the platforms we
support.
(Java applications tend to do this)
Godi would be very nice if I could figure out how to create a custom Godi
repository that I could burn onto a CD and then point my Godi installer to
it. I seem to recall going through and looking through Godi a year or so
ago to figure this out but ran out of time and patience. Godi would have to
be able to assemble the repository for me based on a top level set of
requirements (run down the dependency chain) or the situation hasn't
improved at all compared to distributing the modules ourselves.
At some level I think that what this really points to is a lack of a very
large standard library. One of the arguments often made by programmers that
I'm trying to introduce to OCaml is that the standard library sucks compared
to what they're used to. They've gotten used to Java and Ruby and being
able to quickly and easily do sophisticated string operations (the Standard
string doesn't have many of the functions we commonly use so we end up using
Extlib), network operations, (sockets are a start, but just that), and XML
processing. While there are many packages out there to do these things
we've all gotten used to languages that "come with it."
Take it or leave it, just my $.02 from trying to introduce OCaml into an
environment where we write network applications (including but not limited
to web services) that have to run on very tightly controlled systems. (I
know I'm in the huge minority of programmers)
Daniel Bünzli
> ** you make a bug correction in libA, as many developers you don't
> have
> time to submit bug upstream -> libA upstream and libA embedded
> begin to
> be different
I agree this is a problem. But realisticaly if you only bug fix,
interfaces usually do not change. Hence upstream and local don't
really drift away. You should be able to take a new version of libA
and plug it right away.
> * libA is not embedded inside progB:
> ** libA do a minor bug correction -> progB just need to be recompiled
> (no release), easy to do automatically with a dependency tracking
I disagree, the new version of libA may break invariants progB was
relying on. If progB wants to use the new library it should be an
explicit choice of progB's programmer.
> The only real needs is to have something that automatically
> download/build/install dependency! AND WE HAVE IT: godi!
The problem for me is that godi is both locally and remotly
centralized. Actually what I want is a little godi that I manage
myself for each of my projects. Let's be clear I actually also think
that direct source integration is not the best idea, but currently it
is what makes my life the simplest. I wouldn't mind not *directly*
integrate the sources but instead pointers to module downloads if
that allows me to easely regenerate/upgrade the modules.
In some sense I want my projects to be -- modulo automatic downloads
of modules -- without free variables, self-contained, they should be
self-describing. The only untold prerequisite should be the ocaml dev
tools (and even...). The rest should be taken care of automatically.
I think that a tandem ocamlbuild/caml-get could go much more in that
direction, especially because, compared to godi, it would make the
system much more distributed.
> Off course, as long as you will think that you must embed
> everything, you
> won't need to learn godi (which is very simple) and you won't
> evolve on
> this point.
Well I used at some point. But for some reason -- I don't remember
exactly -- it was getting too much in my way and I stopped using it.
Anyway even if you use godi I think that there are many advantages to
make libraries smaller and push them toward atomic modules: they
reveal the real dependencies, they are easier to maintain and it may
be easier to find new maintainers for orphaned modules. A developer
may find it an achievable task to maintain a single module he is
interested in but may shun in front of a cathedral-like library.
Best,
Daniel
Sylvain Le Gall
> --===============1631936878==
> Content-Type: multipart/alternative;
> boundary="----=_Part_6613_111225.1190807134005"
>
> ------=_Part_6613_111225.1190807134005
> Content-Type: text/plain; charset=ISO-8859-1
> Content-Transfer-Encoding: 7bit
> Content-Disposition: inline
>
>>
>> It is funny, because when you see other big languages -- which are
>> working very well -- like Perl, they all try to avoid embedding libraries!
>>
>> The only real needs is to have something that automatically
>> download/build/install dependency!
>>
>> AND WE HAVE IT: godi!
>>
>> Or if you want things more distro based: debian...
>
>
> environment where we write network applications (including but not limited
> to web services) that have to run on very tightly controlled systems. (I
> know I'm in the huge minority of programmers)
>
Second solution: distribution based OCaml -> debian.
For your example, debian is distributing package for PXP (XML),
PagodaCF (network), OCsigen (webservices).... You have most of this in
stable release (Etch) and a lot more in unstable (Sid).
You can burn a CD or DVD of the distribution.
To see all the ocaml library we have:
http://pkg-ocaml-maint.alioth.debian.org/ocaml_src_pkgs.html
You can even take a look at our subversion repository:
http://svn.debian.org/wsvn/pkg-ocaml-maint
Which contains more libraries -- some are not released.
Concerning GODI, i think you can "hack" something to auto download
everything which are referenced in GOD build, considering that getting
the GODI build script is only a matter of doing a subversion checkout.
(taking a short look at GODI, give me, for cryptokit
Makefile:
MASTER_SITES= http://caml.inria.fr/distrib/bazar-ocaml/
distinfo:
SHA1 (cryptokit-1.2.tar.gz) = 95f0bb53dad3e006c97fb573e0bc19c6c47c3ef7
Size (cryptokit-1.2.tar.gz) = 106543 bytes
which should be enough to download cryptokit for GODI... but i agree
there is some works to do.
)
You also have PXP and OCsigen in GODI (but not PagodaCF i think).
I think that standard library should be as small as possible. Most of
the user don't need to have most of the modules provided by it (in fact,
i would have like NOT to have Graphics/LablTK/Unix in ocaml, that will
have made things more simple and more extendable for everyone).
Being able to have an efficient way of distributing library is difficult
but it is what made things powerful.
skaller
> On 26-09-2007, Daniel Bünzli <daniel....@epfl.ch> wrote:
> OK, let me make myself clear, about the fact that embedding things is
> the worst case.
[]
It's called a fork, I have around 5 of them in my package.
I agree it is bad.
> The only real needs is to have something that automatically
> download/build/install dependency!
>
> AND WE HAVE IT: godi!
Doesn't run on Windows.
> Or if you want things more distro based: debian...
Not used on Fedora, incompatible even with Ubuntu (which is
Debian based) and is managed by a bunch of hard working
people -- with an elitist management philosophy that prevents
people contributing.
I have a build system that works on all platforms.
Why aren't you using it?
Bad as it is, a fork is still the only viable option.
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
_______________________________________________
Michael Furr
On Wed, 26 Sep 2007, skaller wrote:
> On Wed, 2007-09-26 at 00:16 +0200, Daniel Bünzli wrote:
>> Le 25 sept. 07 ŕ 21:30, Mike Furr a écrit :
>
>> Having to install them not only puts a burden on potential users, but
>> it also puts a burden on me when I suddenly start to hack my project
>> in another environment which has not all the dependencies installed.
>
> indeed, reins requires findlib, Omake, and Ounit..
Of course these are *build-time* dependencies. These are not required by
anyone using the library once its installed.
-m
Michael Furr
On Wed, 26 Sep 2007, Daniel Bünzli wrote:
> There are a lot of useful reusable modules in many projects like extlib,
> ocamlnae, etc, however they are all tied to their "cathedral" [1] even though
> these ties are sometimes very small and could be functorized away (or dropped
> altogheter). Now if I _just_ want to use the Unzip module of extlib, the
> Cf_tai64 module of ocamlnae and your Patricia set. Why do I have to install
> these three whole libraries whereas I could simply integrate these three
> modules in my project ?
Why should you not? There is no good reason to not use the entire library
instead of just pulling out a single funciton.
I think you need to separate who are *developers* and who are *users*.
Users are those which you should not depend on having an OCaml development
environment intalled. This could be users of a stand-alone app, or
perhaps coders using felix. When you distribute your software to them, it
will be in binary form, and you should take care to ensure this is as easy
as possible (especially if you're on a platform like windows). However,
this does not require you to keep them in your own source tree.
For *developers*, you should have every expectation that they can install
a sufficient environment themselves. No OCaml build system is that bad
(especially compared to other languages) and I expect *every* user of my
library to be able to build its dependences (or, more realistically,
install a package manager to build the deps for them).
> I think developers of reusable components should try to strive for
> *focused*, independent modules, even if it means functorization and
> convention. Eventually it makes it easier for developers to integrate
> these modules and exchange specific parts if some module poses a problem
> or better alternatives popup.
But you should start by raising the bar for re-use, not lowering it. Use
the libraries like you *should* be able to do. If developers find this
too difficult to use, tools will follow. Rely on the bazaar.
> Again I'm not for embedding everything in projects, sometimes it is
> unrealistic. But for exemple I'd systematically do this for data structures,
> unless of course they are in the standard distribution.
There is a significant advantage to combining various data structures into
a single package. For one, there is a lot of room for code sharing. A
lot of the functionality of data structures can be implemented in terms of
a few small functions. (For instance, to_string is easily implemented
with fold). Using mixin modules to extend these to complete
implemetations saves a lot of redundant code. Also, reins runs over 600
unit tests, but there certainly aren't 600 separate tests written in the
source. It uses a functors to apply to the same tests to different data
structures. This ensures that all of the data structures are
observationally equivalent when they implement the same interface (which
is especially useful for ensuring they all throw the same exception, for
instance).
Finally, choosing a single data structure at the beginning of your project
and embedding it in your source tree forces you to always use that data
structure. One of the advantages of Reins is that it includes different
implementations that can be swapped out. If you decide you would rather
using a a list with fast concatenation rather than fast cons, its a one
line change. You don't have to go find the software, build it, ensure it
has the same interface, ensure it behaves the same way, add it to your
source tree, add it to your build system, and then change your uses of it.
Of course, the software is LGPL for a reason. If you want to pull out
parts of it, you are certainly welcome to do so. I just won't recommend
that path to anyone.
Cheers,
-mike
Vincent Aravantinos
Maybe if there were some 'Inria'-made tool (seems camlget is starting
point for that ?) that be settled in the standard ocaml distribution
would help to make everyone agreed, don't you think ? This does not
solve intrinsic problems such as "I prefer dynamically linked" vs "I
prefer statically linked" but at least that would be a tool everyone
could rely on. Then everyone is free to use it or not. It would even
simplify the task for library-writers.
BTW isn't it the same with ocamlbuild ? Before there were Omake and
others. Now it seems to settle some kind of standard.
My 2cents.
Vincent
Sylvain Le Gall
> Ok, there is camlget, ok there is ocamlfind, ok there is godi.
>
> Maybe if there were some 'Inria'-made tool (seems camlget is starting
> point for that ?) that be settled in the standard ocaml distribution
> would help to make everyone agreed, don't you think ? This does not
> solve intrinsic problems such as "I prefer dynamically linked" vs "I
> prefer statically linked" but at least that would be a tool everyone
> could rely on. Then everyone is free to use it or not. It would even
> simplify the task for library-writers.
>
ocamlfind and godi are very good tool on their own. Why do they need to
be INRIA-made tool ?
INRIA made very good compiler. Great! But let other people use this
language to build other tools -- with different idea than INRIA.
> BTW isn't it the same with ocamlbuild ? Before there were Omake and
> others. Now it seems to settle some kind of standard.
>
FYI, i really think having ocamlbuild in ocaml is not a good solution
(as bad as having LablTk). For ages, there was ocamlmakefile -- coming
from INRIA. This was widely used but doesn't become a standard -- there
is no need for it. Anyone, can choose to use his/her/anyone own build
system.
It is really strange, everyone seems to look at OCaml as "languages for
kid" with everything bundle into some kind of nice package... Please be
more realistic, OCaml is a complicated language -- design for
"discriminative hackers". There is a great shift between end-user and
developper (as M. Furr explains). If you want to deliver something
without dependency, you just have to precompile things, tar gzip it in a
nice shell script ".run" and made it available. This will be easy for
end-user. For developpers, compiling their own OCaml program, dealing
with library dependency should be easy !
Regards,
Sylvain Le Gall
Sylvain Le Gall
Forget the best "worst case".
You ship libA -- with a local modification in libB. A developper want to
use libA (original -- no modification) and libB...
Is it possible ?
I think you will have a conflict between the two version (.cmi exists
with different signatures)...
So by shipping libA with modification, you cannot use libA into another
library...
skaller
>
> On Wed, 26 Sep 2007, skaller wrote:
>
> > On Wed, 2007-09-26 at 00:16 +0200, Daniel Bünzli wrote:
> >
> >> Having to install them not only puts a burden on potential users, but
> >> it also puts a burden on me when I suddenly start to hack my project
> >> in another environment which has not all the dependencies installed.
> >
> > indeed, reins requires findlib, Omake, and Ounit..
>
> Of course these are *build-time* dependencies. These are not required by
> anyone using the library once its installed.
This is true .. but you're missing something, which is slightly
surprising considering you're the maintainer and DD manager
of the Debian package for Felix ..
None of my clients will be needing this library to use it,
because they're not (using Felix as) Ocaml programmers.
If I used it, it would be used ONCE by users to build the
Felix executable and discarded. Already, Python, Ocaml
and C++ are needed for this single build.
Anyhow, that's why I would not want to ask MY clients to
have to install these dependencies AND THEN your library:
that adds 4 more dependencies and places where things can
go wrong .. will your build scripts work on Windows?
Will Omakes and findlibs and Ounits build scripts build
transparently on Windows?
My build script do -- they're platform independent.
[Yes, I know Omake is designed specifically to work on Windows
to enable platform independent builds]
skaller
>
> On Wed, 26 Sep 2007, Daniel Bünzli wrote:
> I think you need to separate who are *developers* and who are
> *users*.
> Users are those which you should not depend on having an OCaml
> development
> environment intalled. This could be users of a stand-alone app, or
> perhaps coders using felix. When you distribute your software to
> them, it
> will be in binary form, and you should take care to ensure this is as
> easy
> as possible (especially if you're on a platform like windows).
> However,
> this does not require you to keep them in your own source tree.
But it isn't: Felix is distributed as source code. Only in special
cases such as Debian-Linux are binary package made available.
[Thanks Mike!]
skaller
> On 26-09-2007, Vincent Aravantinos <vincent.a...@yahoo.fr> wrote:
> > Ok, there is camlget, ok there is ocamlfind, ok there is godi.
> >
> > Maybe if there were some 'Inria'-made tool (seems camlget is starting
> > point for that ?) that be settled in the standard ocaml distribution
> > would help to make everyone agreed, don't you think ? This does not
> > solve intrinsic problems such as "I prefer dynamically linked" vs "I
> > prefer statically linked" but at least that would be a tool everyone
> > could rely on. Then everyone is free to use it or not. It would even
> > simplify the task for library-writers.
> >
>
> ocamlfind and godi are very good tool on their own. Why do they need to
> be INRIA-made tool ?
Because that way, everyone who goes to the trouble to install Ocaml
will have them without further problems.
> INRIA made very good compiler. Great! But let other people use this
> language to build other tools -- with different idea than INRIA.
To make that feasible, there needs to be a standard set of tools
and rules for accessing these products. At present, these rules
vary from platform to platform.
Software developers often want to get their software out on
ALL platforms, and that requires a platform independent
package manager and build tool as 'standard'.
> It is really strange, everyone seems to look at OCaml as "languages for
> kid" with everything bundle into some kind of nice package... Please be
> more realistic, OCaml is a complicated language -- design for
> "discriminative hackers". There is a great shift between end-user and
> developper (as M. Furr explains). If you want to deliver something
> without dependency, you just have to precompile things, tar gzip it in a
> nice shell script ".run" and made it available. This will be easy for
> end-user. For developpers, compiling their own OCaml program, dealing
> with library dependency should be easy !
But the problem is I have several OS: Ubuntu-Linux, XP32, XP64
and Cygwin. How do I as the developer actually make the packages
for Redhat/Fedora/Solaris/MacOSX/Windows ....?
How about all the CPU types around? x86, x86_64, ia64, Sparc ...
I am just not rich enough :)
I can make a Python program that builds Ocaml
and C/C++ source code on all platforms though, and have.
I prefer to supply a "Higher Order Function", namely source
code and source code to build source code .. than binary instances
for the same software engineering reasons we use Ocaml in
the first place: polymorphic functions provide better reusability.
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
_______________________________________________
Mike Furr
> This is true .. but you're missing something, which is slightly
> surprising considering you're the maintainer and DD manager
> of the Debian package for Felix ..
I believe the main point here is that you are bending what I consider
best practice for felix because it is still in development and you want
to speed its adoption. I do understand that.
> None of my clients will be needing this library to use it,
> because they're not (using Felix as) Ocaml programmers.
>
> If I used it, it would be used ONCE by users to build the
> Felix executable and discarded. Already, Python, Ocaml
> and C++ are needed for this single build.
You should no more expect them to build the reins library then for the
gcc authors should expect you to build all of the dependencies of
gcc/g++ (of which there appear to be >40 in the debian/control file for
that package). When you distribute the felix binary, all of its ocaml
libraries are statically linked in, so they don't have to download
anything outside of your binary (which is the downside of your choice to
do source releases).
> will your build scripts work on Windows?
> Will Omakes and findlibs and Ounits build scripts build
> transparently on Windows?
I haven't tested it, but I would be very surprised if they didn't.
> My build script do -- they're platform independent.
>
> [Yes, I know Omake is designed specifically to work on Windows
> to enable platform independent builds]
And I am relying on that fact. Good thing I didn't just cut and paste
the relevant omake code to compile my stuff on linux. :-p
-mike
Vincent Aravantinos
Le 26 sept. 07 à 18:42, Sylvain Le Gall a écrit :
> On 26-09-2007, Vincent Aravantinos <vincent.a...@yahoo.fr>
> wrote:
>> Ok, there is camlget, ok there is ocamlfind, ok there is godi.
>>
>> Maybe if there were some 'Inria'-made tool (seems camlget is starting
>> point for that ?) that be settled in the standard ocaml distribution
>> would help to make everyone agreed, don't you think ? This does not
>> solve intrinsic problems such as "I prefer dynamically linked" vs "I
>> prefer statically linked" but at least that would be a tool everyone
>> could rely on. Then everyone is free to use it or not. It would even
>> simplify the task for library-writers.
>>
>
> ocamlfind and godi are very good tool on their own. Why do they
> need to
> be INRIA-made tool ?
To make it easier, that's all some people are reluctant to use non
supported tools (this joins a recent discussion about extending
standard library or keeping other libraries aside). And you can't blame
them for that.
> INRIA made very good compiler. Great! But let other people use this
> language to build other tools -- with different idea than INRIA.
>
>> BTW isn't it the same with ocamlbuild ? Before there were Omake and
>> others. Now it seems to settle some kind of standard.
>>
>
> FYI, i really think having ocamlbuild in ocaml is not a good solution
> (as bad as having LablTk). For ages, there was ocamlmakefile -- coming
> from INRIA. This was widely used but doesn't become a standard --
> there
> is no need for it. Anyone, can choose to use his/her/anyone own build
> system.
Ocamlbuild is here and you're still free to choose...
However it's good to have a default when you are a beginner or don't
want
to mind yourself with "which one should I choose ?".
Those kinds of question are undesirable side-effects that you don't care
when you want to concentrate on thinking the architecture of your code.
> It is really strange, everyone seems to look at OCaml as "languages
> for
> kid" with everything bundle into some kind of nice package...
facilities <> for kids
> Please be
> more realistic, OCaml is a complicated language
No. Saying this is an excuse for people who don't manage to make it
accessible. Ask yourself why ocaml is not as popular as so many other
languages. It is not far most complicated that many other ones. Why
do you think F# is gaining in interest for many people ? It is not
(only) the language...
V.
Daniel Bünzli
Le 26 sept. 07 à 17:32, Michael Furr a écrit :
>> I think developers of reusable components should try to strive for
>> *focused*, independent modules, even if it means functorization
>> and convention. Eventually it makes it easier for developers to
>> integrate these modules and exchange specific parts if some module
>> poses a problem or better alternatives popup.
>
> But you should start by raising the bar for re-use, not lowering it.
I don't know how I should interpret this (I don't understand the
meaning).
For me the bar is too high for sharing. We need more lightweight ways
of doing so, let the real bazar be and eventually good components
will be selected. Maybe if there had been a convenient, agreed bupon,
module-based, *localized* (in the sense not system wide) and
distributed package system you wouldn't be the the xth person to
implement avl trees. For example go look into Camomile there's an
implementation there.
But I agree with you tight coupling can also be beneficial, it's a
trade-off. However maybe (I don't know) reins could have been split
w.r.t to the different kind of semantic data structure (set, list,
map). It depends on the deal of code sharing that actually occurs
between them. The unit tests could have been done as a separate
project that requires these smaller packages. etc. I strongly believe
that smaller components are more manageable in the long term. Because
whenever they break or become orphaned it becomes easier for third-
parties to understand them and maintain them alive.
But hey, I don't want to look like I'm grumling about reins, it sure
looks great and usefull. So I think I better stop this discussion here.
Le 26 sept. 07 à 18:42, Sylvain Le Gall a écrit :
> It is really strange, everyone seems to look at OCaml as "languages
> for
> kid" with everything bundle into some kind of nice package...
> Please be
> more realistic, OCaml is a complicated language -- design for
> "discriminative hackers".
I don't see any reason why complexity should be introduced for the
sake of it. I like simple, elegant, bar-bones, solutions, that's why
I like ocaml and I don't think it is a complicated language. I'm
interested in developing applications not fiddle around with build
and package management systems -- and as such ocamlbuild is a blessing.
Best,
Daniel
Mike Furr
>> But you should start by raising the bar for re-use, not lowering it.
>
> I don't know how I should interpret this (I don't understand the meaning).
I only meant that we should strive to use the "right" solution, even if
that means a little more work for the time being since it will hopefully
improve things in the long run.
> For me the bar is too high for sharing. We need more lightweight ways of
> doing so, let the real bazar be and eventually good components will be
> selected. Maybe if there had been a convenient, agreed bupon,
> module-based, *localized* (in the sense not system wide) and distributed
> package system you wouldn't be the the xth person to implement avl
> trees. For example go look into Camomile there's an implementation there.
Its hard for me to conceptualize how such a distributed system would
work and so I can't really comment if it would in fact be a better
system. However, one problem I wanted to tackle was that while I would
be the xth person to implement avl trees, there will be NO x+1th person.
Reins is one (possible) solution to that.
> But I agree with you tight coupling can also be beneficial, it's a
> trade-off. However maybe (I don't know) reins could have been split
> w.r.t to the different kind of semantic data structure (set, list, map).
> It depends on the deal of code sharing that actually occurs between
> them. The unit tests could have been done as a separate project that
> requires these smaller packages. etc.
Currently, the unit tests are just part of the build and are not
installed. However, since they are parameterized, perhaps I should
distribute them as a second (optional) library so that anyone who
implements a data structure which matches the signatures there can use
them too. Thanks for the idea.
> I strongly believe that smaller
> components are more manageable in the long term. Because whenever they
> break or become orphaned it becomes easier for third-parties to
> understand them and maintain them alive.
As a counter point, smaller components may mean that fewer people are
interested in each component and thus are more likely to become orphaned
as individuals than as part of a larger collection. I think your idea
is interesting, but I haven't seen any anecdotal evidence of it actually
being better than the "cathedral" approach. Pointers/references to such
things would be welcome.
> But hey, I don't want to look like I'm grumling about reins, it sure
> looks great and usefull. So I think I better stop this discussion here.
I don't think you are, and in fact, as library designer/maintainer its
great to get feedback on how people feel about the current status quo of
library management. If someone were to build such a distributed
localized module collection, I would love to see the code from reins in
there. In the time being, I think the cathedral approach should work
quite well.
Thanks for the dialogue... and now back to hacking.
-m
skaller
> skaller wrote:
>
> > This is true .. but you're missing something, which is slightly
> > surprising considering you're the maintainer and DD manager
> > of the Debian package for Felix ..
>
> I believe the main point here is that you are bending what I consider
> best practice for felix because it is still in development and you want
> to speed its adoption. I do understand that.
Actually no: IMHO 'best practice' is that programs are *source code*
and that is the only way they should be distributed. Binaries
should never be installed or distributed: they're just a 'cache'
and should be invisible.
Felix is designed to work that way: you execute script directly,
the same as Perl and Python.
Other systems, such as C, are so utterly plagued by design
faults and portability issues that systems like Debian were
designed to do this caching in a reliable way and distribute
the binaries .. but this is simply a consequence of the poor
engineering of C.
> You should no more expect them to build the reins library then for the
> gcc authors should expect you to build all of the dependencies of
> gcc/g++ (of which there appear to be >40 in the debian/control file for
> that package).
But that's because it is written in C. For Ocaml code
I expect .. and indeed demand .. to build from source.
That is possible because of the superior design of the
Ocaml programming language.
That is how Godi manages things -- and there's a good argument
Debian Ocaml 'binary' packages should ACTUALLY contain source
code which is built on demand: it would be a whole lot more
portable and easier to maintain, since upgrades to Ocaml require
modifying ALL the dang binary libraries, but very rarely
any change to sources.
Binary distros suck because user code not part of the distro
isn't handled. 'Execute source' is the way to go, because
it rebuilds everything, including user code, automatically
as required. IMHO of course :)
> When you distribute the felix binary, all of its ocaml
> libraries are statically linked in, so they don't have to download
> anything outside of your binary (which is the downside of your choice to
> do source releases).
Yes, this is true, however I don't own a compile farm to build
all the binaries.
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
_______________________________________________
skaller
> But hey, I don't want to look like I'm grumling about reins, it sure
> looks great and usefull. So I think I better stop this discussion here.
It's the other way around, for me at least: reins looks so good,
I want to use it .. but feel I can't for reasons entirely
unrelated to the quality of the code itself.
If these issues aren't discussed there will never be any consensus
what to do about it.
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
_______________________________________________
Mike Furr
Several people emailed me about the problems with the LGPL and static
linking. So, I have released a new version (0.1a) whose only change is
that it adds a linking exception to the LGPL in the exact same spirit of
INRIA's own exception.
Cheers,
-Mike
Vincent Aravantinos
Le 26 sept. 07 ą 12:26, Sylvain Le Gall a écrit :
> AND WE HAVE IT: godi!
As a side effect, you re-raised my interest about Godi. But looking
at the main page it seems it doesn't work with ocaml 3.10. Am I
missing something is it under developpment ?
Thanx,
V.
Vincent Aravantinos
Le 27 sept. 07 à 00:38, Vincent Aravantinos a écrit :
>
> Le 26 sept. 07 à 12:26, Sylvain Le Gall a écrit :
>
>> AND WE HAVE IT: godi!
>
> As a side effect, you re-raised my interest about Godi. But looking
> at the main page it seems it doesn't work with ocaml 3.10. Am I
> missing something is it under developpment ?
>
> Thanx,
> V.
Sorry for this, a little check on the godi mailing list informed me
about that. Nice.
Sorry for the noise,
Richard Jones
> Godi would be very nice if I could figure out how to create a custom Godi
> repository that I could burn onto a CD and then point my Godi installer to
> it. I seem to recall going through and looking through Godi a year or so
> ago to figure this out but ran out of time and patience. Godi would have to
> be able to assemble the repository for me based on a top level set of
> requirements (run down the dependency chain) or the situation hasn't
> improved at all compared to distributing the modules ourselves.
I'm not a big fan of yum, but one thing it does well is the
'createrepo' command which lets you effortlessly create a repository
from a collection of RPMs. So RPM + createrepo + a CD burner would do
what you want on Fedora.
I think the bigger problem is that certain Well Known Operating
Systems don't come with any package management to speak of.
Rich.
--
Richard Jones
Red Hat
Nathaniel Gray
>
> indeed, reins requires findlib, Omake, and Ounit.. Omake doesn't
> configure on my system (it can't find unixdll.so, which seems
> to be required only to build the docs).
Assuming there's nothing pathological about your system, I'm sure the
OMake developers would like to hear about and correct this problem.
-n8
--
>>>-- Nathaniel Gray -- Caltech Computer Science ------>
>>>-- Mojave Project -- http://mojave.cs.caltech.edu -->
Adrien
> skaller wrote:
>
> > will your build scripts work on Windows?
> > Will Omakes and findlibs and Ounits build scripts build
> > transparently on Windows?
> I haven't tested it, but I would be very surprised if they didn't.
Under mingw/msys, findlib doesn't work.
In fact, I made some (small) modifications which solves all the
problems I could find.
Anyway, findlib's current version (1.1.2pl1) won't work under windows,
except full cygwin, no matter whether it is mingw/msys or msvc.
Christoph Bauer
> Anyway, findlib's current version (1.1.2pl1) won't work under
> windows, except full cygwin, no matter whether it is
> mingw/msys or msvc.
>
This is not true. On my machine it runs. A binary is included in
ocaml-mingw-maxi. I agree, that the build scripts of findlib could be
improved
for windows.
Christoph Bauer
Adrien
- findlib's configure script explicitely uses a forward slash as a
path delimiter
- the EXEC_SUFFIX variable which enables the use of an extension for
executables (window's ".exe") appeared to simply not work
- don't have the full list available here
Technically nothing too big prevents findlib from working under
windows. The thing is findlib's official current version just doesn't.
---
Adrien Nader
David Allsopp
> - findlib's configure script explicitely uses a forward slash as a
> path delimiter
> - the EXEC_SUFFIX variable which enables the use of an extension for
> executables (window's ".exe") appeared to simply not work
> - don't have the full list available here
>
> Technically nothing too big prevents findlib from working under
> windows. The thing is findlib's official current version just doesn't.
I submitted patches for the official source to Gerd 16 months ago and he
very quickly responded with a test version 1.1.3 in SVN for me - regrettably
it has taken *me* this long to get around to testing it. I don't know what
Gerd's workload is like at the moment, but with luck there may be a new
official version that correctly supports Windows soon!
David