[Caml-list] Bug in ocamlyacc
David Allsopp
> doligez: We won't spend time fixing minor issues in ocamlyacc because the
> future is in menhir. The workaround is obvious anyway.
Does this mean that menhir will be moved to the O'Caml distribution at some
point? I don't think I'm the only person reading this list who doesn't use
Menhir (at the moment) solely because it is not part of the *standard*
O'Caml distribution. Unless Menhir is due to become the official,
distributed and supported replacement for ocamlyacc, then this comment is
not terribly impressive[1]
David
[1] Though this specific problem is utterly trivial to work around, as
stated!
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
skaller
> With regards to the closing comment to Bug Report #4251:
>
> > doligez: We won't spend time fixing minor issues in ocamlyacc because the
> > future is in menhir. The workaround is obvious anyway.
>
> Does this mean that menhir will be moved to the O'Caml distribution at some
> point? I don't think I'm the only person reading this list who doesn't use
> Menhir (at the moment) solely because it is not part of the *standard*
> O'Caml distribution.
I won't use Menhir for that reason either. My product, Felix, is
designed to 'build out of the box' on all platforms with minimal
pre-requisites, which happen to be: you need
(a) Python
(b) Ocaml
(c) A C/C++ compiler
* with development versions of libs
People have enough trouble with just 3 pre-reqs without me
adding another code base/product to the list -- it is particularly
messy on Windows (native) platforms.
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
Diego Olivier FERNANDEZ PONS
> I won't use Menhir for that reason either.[...]
Come on, Skaller. You know as well as everyone that adding a tool that
is not mature enough to the standard distribution is a bad idea.
Mehnir is clearly the "official" candidate for replacement of
CamlYacc, without yet being mandatory. This gives you the time to
port, criticize and ask for improvements. Look the mess that happened
with CamlP4 which didn't follow this pattern.
So instead of being complaining because Mehnir is not yet in the
standard distribution, you should be porting your CamlYacc code to
ensure that when the tools are swapped, Menhir will make your code
simpler, faster and cleaner.
Diego Olivier
ls-ocaml-de...@m-e-leypold.de
Diego Olivier FERNANDEZ PONS <diego.fern...@etu.upmc.fr> writes:
> Bonjour,
>
>> I won't use Menhir for that reason either.[...]
>
> Come on, Skaller. You know as well as everyone that adding a tool that
> is not mature enough to the standard distribution is a bad idea.
> Mehnir is clearly the "official" candidate for replacement of
> CamlYacc, without yet being mandatory. This gives you the time to
> port, criticize and ask for improvements. Look the mess that happened
> with CamlP4 which didn't follow this pattern.
>
> So instead of being complaining because Mehnir is not yet in the
> standard distribution, you should be porting your CamlYacc code to
> ensure that when the tools are swapped, Menhir will make your code
> simpler, faster and cleaner.
Isn't there a contradiction here somewhere? "We won't fix Ocamlyacc,
because Menhir will replace it and we won't put Menhir in the standard
distribution because it is not god enough yet?".
I don't want to make any demands whatsoever here -- I haven't looked
an Menhir yet, nor have I looked at the Ocamlyacc bug in
question. It's only I see a discrepancy here.
Regards -- Markus
Francois Pottier
Hello,
On Tue, Apr 24, 2007 at 03:29:31PM +1000, skaller wrote:
> I won't use Menhir for that reason either. My product, Felix, is
> designed to 'build out of the box' on all platforms with minimal
> pre-requisites, which happen to be: you need
>
> (a) Python
> (b) Ocaml
> (c) A C/C++ compiler
> * with development versions of libs
You could in principle pre-compile your parser on your own machine using
menhir and ship the generated files, so menhir wouldn't be a prerequisite.
There are at least two reasons why Menhir isn't (yet) shipped with ocaml.
First, it produces code which is significantly larger than ocamlyacc's
tables. This might be a problem for some people, so we should add an option to
produce tables instead of code. Second, Menhir is still considered
experimental, and its input language could still evolve. For instance, its
treatment of errors and error recovery might be modified in the future, as
well as its treatment of priorities.
Feel free to make suggestions about Menhir! They are most welcome, even
though, by lack of spare time, Menhir is not moving very fast.
--
François Pottier
Francois...@inria.fr
http://cristal.inria.fr/~fpottier/
skaller
> Bonjour,
>
> > I won't use Menhir for that reason either.[...]
>
> Come on, Skaller. You know as well as everyone that adding a tool that
> is not mature enough to the standard distribution is a bad idea.
Not if it is optional, like the ocamlbuild tool for example.
> Mehnir is clearly the "official" candidate for replacement of
> CamlYacc, without yet being mandatory. This gives you the time to
> port, criticize and ask for improvements. Look the mess that happened
> with CamlP4 which didn't follow this pattern.
There do indeed appear to be some issues with camlp4.
However what was done was the 'right thing' despite that --
IMHO of course.
First: few would have tested it, given the incompatibilities,
it it were not forced onto the market.
Second: Ocaml 3.10 is still in beta, and the author(s) of
camlp4 *needed* the feedback and participation from the community.
Third: the community wanted a better camlp4: this is the price,
and I believe most people using camlp4 are willing to help
and get it working.
Fourth: it had to be done sometime, and I would guess that
there was no technical choice but a painful change.
However with Menhir, the issues aren't so bad because it doesn't
yet need to replace Ocamlyacc, just be available as an alternative.
> So instead of being complaining because Mehnir is not yet in the
> standard distribution,
I'm not complaining, but stating my reasons why I think it
should be
> you should be porting your CamlYacc code to
> ensure that when the tools are swapped, Menhir will make your code
> simpler, faster and cleaner.
. and also making it clear, I hope, why I won't be
using it in my product until it is.
Unfortunately my team also has limited resources.
I already tried Menhir and provided some feedback.
I'm supportive of it. But I cannot convert my
production code to use a third party library
unless I can include the source in my product.
Felix doesn't use any external third party libraries:
it uses the system Ocaml and C++ libraries, and it
uses Cil/Frontc, Elkhound and Tre .. which are included
in the Felix system as source code and built by the
Felix build system.
I might consider doing that with Menhir if the licence
were FFAU like MIT/BSD/Boost/Creative Commons.
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
_______________________________________________
skaller
> You could in principle pre-compile your parser on your own machine using
> menhir and ship the generated files, so menhir wouldn't be a prerequisite.
Yes, that is possible, assuming I can also ship any required libraries.
> There are at least two reasons why Menhir isn't (yet) shipped with ocaml.
> First, it produces code which is significantly larger than ocamlyacc's
> tables. This might be a problem for some people, so we should add an option to
> produce tables instead of code.
OUCH. That's a significant complication.
> Second, Menhir is still considered
> experimental, and its input language could still evolve. For instance, its
> treatment of errors and error recovery might be modified in the future, as
> well as its treatment of priorities.
Ocaml is still experimental. Its input language is evolving all
the time. It wouldn't evolve much if it didn't have a user base.
> Feel free to make suggestions about Menhir! They are most welcome, even
> though, by lack of spare time, Menhir is not moving very fast.
Sure! My thoughts would be: Menhir should really go to GLR.
[Felix has GLR parsing built in :]
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
_______________________________________________
Francois Pottier
Hello,
skaller wrote:
> Yes, that is possible, assuming I can also ship any required libraries.
There are none. Menhir produces stand-alone code.
> Sure! My thoughts would be: Menhir should really go to GLR.
Indeed, it would be interesting to have GLR as an option. I don't really have
time to work on this idea at the moment, but it could be a possibility for a
student programming project.
--
François Pottier
Francois...@inria.fr
http://cristal.inria.fr/~fpottier/
_______________________________________________
Martin Jambon
> On Tue, 2007-04-24 at 12:23 +0200, Diego Olivier FERNANDEZ PONS wrote:
> > Bonjour,
> >
> > > I won't use Menhir for that reason either.[...]
> >
> > Come on, Skaller. You know as well as everyone that adding a tool that
> > is not mature enough to the standard distribution is a bad idea.
>
> Not if it is optional, like the ocamlbuild tool for example.
>
> > Mehnir is clearly the "official" candidate for replacement of
> > CamlYacc, without yet being mandatory. This gives you the time to
> > port, criticize and ask for improvements. Look the mess that happened
> > with CamlP4 which didn't follow this pattern.
>
> There do indeed appear to be some issues with camlp4.
> However what was done was the 'right thing' despite that --
> IMHO of course.
>
> First: few would have tested it, given the incompatibilities,
> it it were not forced onto the market.
>
> Second: Ocaml 3.10 is still in beta, and the author(s) of
> camlp4 *needed* the feedback and participation from the community.
>
> Third: the community wanted a better camlp4: this is the price,
> and I believe most people using camlp4 are willing to help
> and get it working.
I wanted to start this as dedicated thread, but let's do it now.
The current situation with camlp4 3.10-beta is terrible. Not
because the new camlp4 is not good or anything, but because it is not
compatible with the old one and yet replaces it. Replacing the old one by
the new one in the next release of OCaml is practically unmanageable
because:
1) Source duplication is unavoidable
2) Upgrading to the new camlp4 is not automatic
The direct consequences are that it will take time until all syntax
extensions support camlp4/ocaml 3.10, and probably anything new will work
with either ocaml 3.09 or 3.10, but not both.
I started upgrading some of my non-trivial syntax extensions already. It's
ok to translate a file once, but I'll be adding new features and I don't
know if I should continue to support ocaml 3.09 or ocaml 3.10. Even though
I want to support both, I can't because it takes twice the time and it's
pretty boring.
A smooth transition would be much preferrable. I acknowledge that I don't
know how make it possible to use syntax extensions using the old and the
new camlp4 on a same ocaml source file, but for those using only one
extension or several extensions using the same version of camlp4, it would
be convenient if it just works.
It could go like "the old camlp4 is deprecated, and support will be
dropped completely [one year after the first stable-and-documented release
of camlp4 3.10]"
* It means two versions of camlp4 would be distributed and installed with
ocaml 3.10.
* Command names for the new camlp4 would be distinct from the old camlp4.
* Old programs would still compile and work without having to patch
makefiles.
* Upgraded programs could add new features only to the 3.10 version, while
maintaining minimal support (bug fixes) for the 3.09 version, without
forcing the camlp4 programmer to use ocaml 3.09 for testing.
* New programs could support only the new camlp4.
Does that make sense?
Camlp4 is such a great tool, it would be sad to loose some users just
because of compatibility uncertainties.
Martin
--
Martin Jambon
http://martin.jambon.free.fr
Yaron Minsky
>
>
> It could go like "the old camlp4 is deprecated, and support will be
> dropped completely [one year after the first stable-and-documented release
> of camlp4 3.10]"
>
> * It means two versions of camlp4 would be distributed and installed with
> ocaml 3.10.
> * Command names for the new camlp4 would be distinct from the old camlp4.
> * Old programs would still compile and work without having to patch
> makefiles.
> * Upgraded programs could add new features only to the 3.10 version, while
> maintaining minimal support (bug fixes) for the 3.09 version, without
> forcing the camlp4 programmer to use ocaml 3.09 for testing.
> * New programs could support only the new camlp4.
This seems like a great idea.
y
ls-ocaml-de...@m-e-leypold.de
"Yaron Minsky" <ymi...@cs.cornell.edu> writes:
> On 4/24/07, Martin Jambon <martin...@ens-lyon.org> wrote:
>
> It could go like "the old camlp4 is deprecated, and support will be
> dropped completely [one year after the first stable-and-documented release
> of camlp4 3.10]"
>
> * It means two versions of camlp4 would be distributed and installed with
> ocaml 3.10.
> * Command names for the new camlp4 would be distinct from the old camlp4.
Call the new version camlp5 ;-).
Regards -- Markus
skaller
> On Wed, 25 Apr 2007, skaller wrote:
> I wanted to start this as dedicated thread, but let's do it now.
> The current situation with camlp4 3.10-beta is terrible. Not
> because the new camlp4 is not good or anything, but because it is not
> compatible with the old one and yet replaces it. Replacing the old one by
> the new one in the next release of OCaml is practically unmanageable
> because:
>
> 1) Source duplication is unavoidable
> 2) Upgrading to the new camlp4 is not automatic
Curious, but can't you run camlp4 (old) to generate pure Ocaml text?
Then, you have no macros and can re-introduce them.
> A smooth transition would be much preferrable. I acknowledge that I don't
> know how make it possible to use syntax extensions using the old and the
> new camlp4 on a same ocaml source file, but for those using only one
> extension or several extensions using the same version of camlp4, it would
> be convenient if it just works.
An alternative is a translator from old to new syntax, I guess that
would be HARD otherwise it would have been done?
> * It means two versions of camlp4 would be distributed and installed with
> ocaml 3.10.
> * Command names for the new camlp4 would be distinct from the old camlp4.
> * Old programs would still compile and work without having to patch
> makefiles.
> * Upgraded programs could add new features only to the 3.10 version, while
> maintaining minimal support (bug fixes) for the 3.09 version, without
> forcing the camlp4 programmer to use ocaml 3.09 for testing.
> * New programs could support only the new camlp4.
>
> Does that make sense?
I wonder if a pipeline would work, so you can multi-stage
pre-processors?
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
_______________________________________________
Daniel de Rauglaudre
On Tue, Apr 24, 2007 at 01:17:31PM -0700, Martin Jambon wrote:
> The current situation with camlp4 3.10-beta is terrible. Not because
> the new camlp4 is not good or anything, but because it is not compatible
> with the old one and yet replaces it.
Alternative solution : download my version of Camlp4, named "camlp4s" :
http://pauillac.inria.fr/~ddr/camlp4s/
This is Camlp4 before its inclusion into OCaml some years ago, and
upgraded to be (almost) compatible to the most recent versions of
Camlp4, except the forthcoming one 3.10.
Most important changes :
One only source which can be used for several versions of OCaml. Just
need to be recompiled. Can be used for ocaml versions from 3.08.1 to
3.10. (If you use a more older version, I can add it.)
The lexer Plexer has been rewritten. It uses a new syntax pa_lexer.cmo
allowing to make parser of streams of characters with a shorter syntax.
Experimented new directive "#pragma" allowing to evaluate expressions
at compile time (= camlp4 time). Useful, for example, to experiment
syntax extensions with the EXTEND statement without having to put
it in another file.
Started an experiment of a new pretty printing system, the old one
of Camlp4 not being satisfactory, nor Format which has sometimes
problems. The library is named Sformat, and is very simple.
But notice that this module Sformat is not yet used for camlp4s pretty
print, the old system remaining for the moment. I experimented it
separatedly and it seems to work. Interesting future feature : you
will be able to decide, by symmetry, to skip all lines in a pattern
matching between the pattern and the expression if one case need a
newline. Same thing in a "if then else" if the "then" case and the
"else" case are similar ("similarity" is tested with the library
"mldiff").
Its distribution being independant from OCaml distribution, new releases
can be done at any time.
--
Daniel de Rauglaudre
http://pauillac.inria.fr/~ddr/
Markus Mottl
> The current situation with camlp4 3.10-beta is terrible. Not
> because the new camlp4 is not good or anything, but because it is not
> compatible with the old one and yet replaces it. Replacing the old one by
> the new one in the next release of OCaml is practically unmanageable
> because:
>
> 1) Source duplication is unavoidable
> 2) Upgrading to the new camlp4 is not automatic
And:
3) Documentation for the new camlp4 is almost non-existent.
> The direct consequences are that it will take time until all syntax
> extensions support camlp4/ocaml 3.10, and probably anything new will work
> with either ocaml 3.09 or 3.10, but not both.
I am not so concerned about the 3.09 compiler not working with the new
macro syntax, because presumably people will upgrade somewhat quickly
anyway. Unless... they depend on old syntax extensions that have not
been rewritten yet.
Very substantial parts of our code base depend on non-trivial Camlp4
macros, and it is a lot of tedious work to rewrite those. Even though
it's a huge PITA, we could live with the lack of backwards
compatibility as long as there were at least sufficient documentation
to support a quick transition. I am also currently working on a new
non-trivial extension, and haven't started writing the macro part yet
in the hope that documentation will soon be available to improve my
productivity.
It seems reasonable to predict that many people won't (be able to)
upgrade to 3.10 for the reasons explained by Martin. I can't imagine
that INRIA wants to make a release that has a hard time of being
accepted by the community. This would be a waste of INRIA's
resources, too.
I'd therefore strongly suggest that INRIA plan more carefully how to
make the next release. From my point of view, the best way would be
to provide sufficient documentation in advance to allow Camlp4
developers to rewrite their macros in time. It would be a great
advantage, too, if the old Camlp4 supported any new syntax (not
extensions, just the base language; there are probably very few
changes anyway) so that people have at least a short reprieve during
which they can just use an old camlp4 installation before they are
forced to upgrade their macros for future OCaml-releases.
> I started upgrading some of my non-trivial syntax extensions already. It's
> ok to translate a file once, but I'll be adding new features and I don't
> know if I should continue to support ocaml 3.09 or ocaml 3.10. Even though
> I want to support both, I can't because it takes twice the time and it's
> pretty boring.
Indeed...
> It could go like "the old camlp4 is deprecated, and support will be
> dropped completely [one year after the first stable-and-documented release
> of camlp4 3.10]"
Sounds reasonable.
> * It means two versions of camlp4 would be distributed and installed with
> ocaml 3.10.
This would surely be helpful, but I don't think that this is
absolutely necessary. It isn't horribly hard to have two
OCaml-installations, and refer to the old camlp4 where necessary.
> * Upgraded programs could add new features only to the 3.10 version, while
> maintaining minimal support (bug fixes) for the 3.09 version, without
> forcing the camlp4 programmer to use ocaml 3.09 for testing.
Maintaining a second OCaml-installation would certainly be a bigger
problem if the macro code itself refers to a lot of external packages
(this may be rare), which may then need to be maintained in a
separation installation, too.
It would surely be most helpful for the community if both good
documentation were available, and a backwards compatible camlp4
version were provided. If there are limited resources at INRIA that
prevent solving both problems, my preference is that they be invested
in the former rather than the latter...
Regards,
Markus
--
Markus Mottl http://www.ocaml.info markus...@gmail.com
skaller
> > * It means two versions of camlp4 would be distributed and installed with
> > ocaml 3.10.
>
> This would surely be helpful, but I don't think that this is
> absolutely necessary. It isn't horribly hard to have two
> OCaml-installations, and refer to the old camlp4 where necessary.
This is not the case for all users. For example I myself have
no problem building multiple versions of Ocaml on my Ubuntu
Linux box, where I deliberately do NOT use the Debian packages
for Ocaml (in fact I prefer the CVS head).
But I also test my product on Cygwin and Windows-native, and both come
with pre-built Ocaml packages. It isn't quite so easy to build on
Cygwin and I cringe in fear at the idea of having to do a Windows
native build -- esp on XP64.
I also run Ocaml on platforms I don't control: shell accounts
on hosted servers.
Furthermore, my product has a Debian package. There's no easy
option to have two ocamls using the Debian autobuilder.
There's NO WAY I can expect my intended user base to install
two versions of Ocaml. I have to write code that "just works"
with any reasonably up-to-date Ocaml installation.
So actually I don't use any camlp4 macros, and at least part
of the reason has been the lack of any concept of installation
and package management in the Ocaml distribution: actually
getting macros to work with camlp4 is quite hard because you
need to know where the files are. The same with third party
libraries.
And now running with multiple installations of Ocaml will
make that even harder -- I already have to do this on Windows
where I support both Cygwin and Windows native build, and
have to support 64 bit Windows Ocaml if someone manages to
actually build one, as well as 32 bit and probably the
MinGW version too...
So I would like to suggest -- yes, it *is* horribly hard to have
two Ocaml installations.
If this is necessary the Ocaml debian maintenance team at least will
have to change their package management design entirely,
since currently it only permits one Ocaml installation.
So actually I like this plan:
* leave camlp4 as it is
* rename the new system camlp5
* supply both
In particular this would support what I would do: if I need to build
third party libraries I put the source in my product, and if library A
uses p4 and library B uses p5 I can easily modify my build scripts
to support both A and B because both p4 and p5 are available.
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
_______________________________________________
Xavier Leroy
> make the next release. From my point of view, the best way would be
> to provide sufficient documentation in advance to allow Camlp4
> developers to rewrite their macros in time.
I agree with the need for documentation, but not with the timing you
propose. Release 3.10.0 has been delayed quite a bit already, and
1- there is no reason to make users who do not depend on Camlp4 wait longer
for 3.10;
2- having the new Camlp4 officially released as part of 3.10.0 can only
facilitate the porting of Camlp4 macros and extensions.
More generally, we are painfully aware that the transition from 3.09
to 3.10 is a minor upgrade for users who do not depend on Camlp4, but
a major change for Camlp4 users. We plan to maintain 3.09 (by making
bug-fix releases) longer than usual so that users who are stuck with
the old Camlp4 can stay with 3.09 for a while. Nobody is forced to
upgrade to 3.10 immediately.
Releasing both old and new Camlp4 as part of 3.10 is out of the
question, however. That would only delay the inevitable and would
waste a lot of our time to make such a release. The priorities are
1- get 3.10.0 out of the door, and 2- of course, provide more
documentation on the new Camlp4.
- Xavier Leroy
Markus Mottl
> I agree with the need for documentation, but not with the timing you
> propose. Release 3.10.0 has been delayed quite a bit already, and
> 1- there is no reason to make users who do not depend on Camlp4 wait longer
> for 3.10;
I agree that there are probably many more users who do not depend on
Camlp4 than ones that do.
> 2- having the new Camlp4 officially released as part of 3.10.0 can only
> facilitate the porting of Camlp4 macros and extensions.
Well, I wouldn't exactly say that it facilitates it, but it certainly
enforces it ;-)
> Releasing both old and new Camlp4 as part of 3.10 is out of the
> question, however. That would only delay the inevitable and would
> waste a lot of our time to make such a release.
We certainly understand that INRIA has limited resources and cannot
afford the same level of support that a commercial company would
provide.
> The priorities are
> 1- get 3.10.0 out of the door
Finally we'll get stack backtraces for native code, which is an
extremely important feature for production systems. I guess every
OCaml-developer will be happy about that.
> and 2- of course, provide more documentation on the new Camlp4.
Please!
Best regards,
Markus
--
Markus Mottl http://www.ocaml.info markus...@gmail.com
_______________________________________________
brogoff
> There are at least two reasons why Menhir isn't (yet) shipped with ocaml.
> First, it produces code which is significantly larger than ocamlyacc's
> tables. This might be a problem for some people, so we should add an option to
> produce tables instead of code.
Hi,
That would be my preference in terms of prioritization of work for
Menhir, to get it to output table driven code. I tried converting a
decent sized grammar (verilog 95 with some later stuff in it) from
ocamlyacc to menhir a few months ago and it was completely painless,
but I did notice that the compile time and code size for menhir were
much greater.
I like where it's going, and even though I'm not a huge fan of yacc,
it is quite compatible AND has addressed some deficiencies of yacc. That
is a huge bonus for me.
-- Brian
Hendrik Tews
Releasing both old and new Camlp4 as part of 3.10 is out of the
question, however. That would only delay the inevitable and would
waste a lot of our time to make such a release. The priorities are
1- get 3.10.0 out of the door, and 2- of course, provide more
documentation on the new Camlp4.
I believe you got the priorities wrong here. Documentation should
be finished before the release.
Bye,
Hendrik