How to get started with a language=Cpp grammar for ANTLR3

[Ben Haller]

Hi all. I want to generate a lexer/parser in C++ for a custom initialization file format. Given that C++ is not yet in ANTLR4, I'm using ANTLR3 (3.5.2).  I've got a grammar that works great when the language target is Java. As soon as I try to switch it to C++, I get an error. Here's what I mean:

darwin:~/antlr/SLiMScript bhaller $ java org.antlr.Tool SLiMScript2.g
error(10): internal error: SLiMScript2.g : java.lang.IllegalArgumentException: No such group file: org/antlr/codegen/templates/Cpp/AST.stg
org.stringtemplate.v4.STGroupFile.<init>(STGroupFile.java:69)
org.stringtemplate.v4.STGroupFile.<init>(STGroupFile.java:48)
org.antlr.tool.ToolSTGroupFile.<init>(ToolSTGroupFile.java:43)
org.antlr.codegen.CodeGenerator.loadTemplates(CodeGenerator.java:216)
org.antlr.codegen.CodeGenerator.genRecognizer(CodeGenerator.java:277)
org.antlr.Tool.generateRecognizer(Tool.java:674)
org.antlr.Tool.process(Tool.java:487)
org.antlr.Tool.main(Tool.java:98)

The relevant part of the grammar is just this, I think, since the grammar itself is very simple and works fine when Java is the target, but I'll be happy to post the full thing if somebody wants to see it:

grammar SLiMScript2;

options {
 language=Cpp;
 output=AST;
}

The only doc I've found on generating C++ is https://theantlrguy.atlassian.net/wiki/pages/viewpage.action?pageId=2687101 and I'm not getting the clue I need from it. It says "Just add language=Cpp; to the options section" and "ANTLR3 will then create the files", but that does not seem to be the case.

Also, anticipating the next issue I will hit, that page also says "To use generated code, you'll need to include Cpp runtime package in your include path."  Which Cpp runtime package would that be?  All I have is the ANTLR jar file.  Am I supposed to find a folder of C++ support files somewhere for the C++ language target and install them on my system somehow?

I imagine this is a pretty dumb question, but I've been googling and reading stuff all morning trying to figure it out and I can't find anything that tells me how to make any progress at all.  Thanks for any help/examples/etc.

EDIT: I have gotten as far as figuring out that I can download the full buildable package for ANTLR instead of just the jar file, and that runtime files for different language targets are then in the runtime/<language> folder.  However, there is no Cpp folder there, which seems odd since Cpp is supposed to be supported, no?  So I guess I need to download those Cpp files from somewhere else, and I don't know where...?

[Gokulakannan K.S.]

Hi Ben,

    C++ Target doesn't support AST. Since you require the output in AST, it won't work. It only supports a Lexer and Parser. No TreeParser.

Gokul.

[Ben Haller]

Oh, that's unfortunate.  I saw the note that the C++ target doesn't support TreeParser, but I did not realize that that meant it couldn't *generate* an AST either.  I'd be quite happy to write my own tree-walking code, so I don't need a TreeParser; I just need to get an AST out of antlr.  Hmm.  Maybe I'll try to set up to build a tree myself using actions.  Does anybody have any sample code for doing that?

Thanks,

-B.

[Mike Lischke]

Oh, that's unfortunate.  I saw the note that the C++ target doesn't support TreeParser, but I did not realize that that meant it couldn't *generate* an AST either.  I'd be quite happy to write my own tree-walking code, so I don't need a TreeParser; I just need to get an AST out of antlr.  Hmm.  Maybe I'll try to set up to build a tree myself using actions.  Does anybody have any sample code for doing that?

Alternatively you can try to use the C target and add your own C++ wrapper class around that. That's how I did it and that works very well.

Mike

-- 

www.soft-gems.net

[Ben Haller]

Hmm, that's an interesting idea, Mike.  Do you have any code you can share?

Cheers,

-B.

[Mike Lischke]

Hmm, that's an interesting idea, Mike.  Do you have any code you can share?

Yes sure. Download MySQL Workbench *source package* from here: http://dev.mysql.com/downloads/workbench/.

Extract it and look in library/myql.parser/. You will see the generated lexer + parser files (e.g. MySQLParser.c) and it's wrapper (e.g. mysql-parser.cpp). There's also a "simple parser", which is the same as the other one but without building an AST (it's sharing the lexer with the full parser). And there are support classes (syntax checker + scanner) which use the generated C files as well for special tasks.

For error reporting I have set an own function (a static C function that gets the parser C++ class as payload to forward calls to it).

Just ask, if you need more details about this.

Mike

-- 

www.soft-gems.net

[Ben Haller]

  Good stuff, I'll check it out.  Thanks Mike!

Cheers,

-B.