Is ANTLRworks broken?

[trijezdci]

Hi, I have asked the same before but perhaps the title wasn't interesting enough for anybody to care reading it ...

https://groups.google.com/forum/?fromgroups=#!topic/antlr-discussion/LsmpUrO6hyw

Anyway, I have asked three different people across the globe to download ANTLRworks 1.4.3, download our grammar, check to see if  AW (a) verifies the grammar and (b) runs the generated parser without errors. Three times verified ok, three times the generated parser crashes with Java errors.

So, I have reason to believe that ANTLRworks is currently broken but it would be nice if anybody here could confirm this. Thank you.

[Eric]

Hi,

 

When Ter asked for info about upgrading ANTLRWorks, (1.4.3 at the time), I spent a good week poking and prodding ANTLRWorks to give feedback. I never really used AW before that but wanted a better AW so that was why I did the review. After that I never trusted ANTLRWorks 1.x again. That doesn't mean I haven't used it, I just verify everything that it produces either against a run of the command line or pretty print out the AST and then hand check it.

 

With regards to your problem, you should be using the command line for parsing and verification instead of AW.

 

Hope that helps.

 

We are not ignoring you but asking us to take out at least a half hour of our time when you still need to use the command line is not something we jump at.

 

Please keep asking questions, we don't mind and we don't mind a friendly repeat if no answers or responses are given.

 

Regards, Eric

 

 

--
 
 

[trijezdci]

this is the console output that AW shows after trying to run the generated parser:

http://pastebin.com/wRW9e60a (URL valid for 1 month as of today)

To me this looks like ANTLR/AW generate incorrect java code.

[Eric]

Hi,

 

That looks like a known and reported bug over a year ago. Problem is I can't remember the exact name of the bug and if I remember right, it was fixed by Ter, then re-fixed by Sam, but not pushed, then Ter pulled the fix, and the fix finally made it back into the command line a few months ago. If so, AW will not have the fix as it is built to include a complete copy of ANTLR when built. So Ter would have to rebuild AW with the updated to fix AW.

 

If so, toss AW aside and use the command line. You will never get AW to work with that until AW 1.x is rebuilt and I don't see that happening, ever, now that ANTLRWorks 2.x is the focus.

 

Regards, Eric

--
 
 

[trijezdci]

We actually build our RD parser by hand. The attraction of ANTLRworks was to get visual feedback:

(1) showing conflicts in the grammar (visually)

(2) showing the parse tree as it is created for sample input (visually)

It seems to me that AW is still good for task #1, but no longer good for #2.

#1 is more important than #2, so not the end of the world, but unfortunate nevertheless.

Anyway, thank you for the reply.

[Eric]

Hi,

 

Ahh,

 

As I did a lot of work on this more than year ago and haven't touched it since I will give what I can, but it may be wrong.

 

1. You can build AW yourself and include the most recent version of ANTLR that has the fix. Make sure the most recent version of ANTLR doesn't have the bug before you build it in.

 

2. You can also bring up AW in NetBeans under the debugger and fix the bug yourself.

 

3. Search http://antlr.markmail.org/ for info on the problem, I may have posted a work around, but can't remember. It would be a change to the grammar and not ANTLRWorks.

 

4. You can try earlier versions of AW as each one will have the version of ANTLR in it at the time was built. You may get lucky and find a version that doesn't have the bug but is recent enough so that you don't lose all of the functionality.

 

5. If I remember correctly AW does not do the conflict analysis, ANTLR does it and it can be reported using a command line option, can't remember which one. AW is just really good at making it clear visually, which I agree is a great reason to use AW. You might be able to pull that part of the code out of AW and make a stand alone tool.

 

6. You can get a picture of the parse tree or without using AW. The command line can output a dot file that can then be viewed graphically with a dot viewer. e.g. zgrviewer.

 

http://www.antlr.org/wiki/display/ANTLR3/How+do+I+display+ASTs+graphically

 

In short, most of the heavy lifting of what you want is done in the command line tool, AW just takes the hard to read output and converts it nicely into a graphical display.

 

Regards, Eric

 

 

 

 

--
 
 

[Eric]

Hi,

 

Checked some of my old docs and found this.

 

http://www.antlr.org/works/help/tutorial/ant.html

 

It is amazing what is on the site if you take the time to hunt and peck through every link.

 

Regards, Eric

[Eric]

Hi,

 

The ANTLR option to show nodeterminism is -Xnfastates

 

Regards, Eric

[Eric]

Another option is to hand patch the Java code after ANTLR generates it. This will still allow you to use AW immediately but will take a minute to do a replace before proceeding. A pain but worth the price if you need the graphical display instantly.

 

Also if I remember correctly, the fix was not to ANTLR but to the template passed to StringTemplate to generate the Java code. So debugging the Java for AW or ANTLR will be doing more work than necessary but will require you to understand how StringTemplates work.

 

Regards, Eric

[trijezdci]

Hi,

I since found that ANTLR cannot seem to handle productions rules in an input grammar if they coincide with Java reserved words. All the grammars we tested were of Wirthian languages and they all contain a production rule called case which coincides with a Java reserved word. I changed the production rule to case777 and then the generated parser could be run in the debugger. There is at least one other issue which looks like line ends (\n) in the input source are reported as mismatched tokens but I will have to investigate that more closely first to be able to ask intelligently.

Anyway, if the bug you are talking about is the one above, ANTLR not being able to handle rule names coinciding with Java reserved words, then I would like to ask that a new version of AW is built with the version of ANTLR that has the fix and then published as a new release, please.

Thank you very much.

[Terence Parr]

do you have a rule that is called "case"? Don't do that ;) don't use rule names that are keywords in the target language. v4 checks this but v3 doesn't.
Ter

> --
>
>

[trijezdci]

On Monday, December 3, 2012 6:43:59 AM UTC+1, the_antlr_guy wrote:

do you have a rule that is called "case"?  Don't do that ;)  don't use rule names that are keywords in the target language.

That's rather fragile design if you don't mind me saying so. It means you can have a perfectly working grammar and it may break as a result of using a different target language for example. It also means you may not be able to use the same identifiers that the grammar uses in which the language is published. For instance, all Wirthian languages have been published with grammars that contain a production called "case".

 v4 checks this but v3 doesn't.

I hope it doesn't merely check in the sense of emitting a warning or error, but use name mangling of some kind to generate the identifiers of functions in the target parser.