How can I add elements to a list in a tree grammar? (antlr 3.5.2, tree grammar, C target)

[Chris McMaiersen]

Hi everyone,

My current situation is as follows:

I have a grammar G1 (output = AST, language = C) with a block as follows:

id_set :     ID (LIST_SEPERATOR ID)* -> ID+

       ;

Corresponding to that grammar, I have a tree grammar G1Tree with the following entry, corresponding to the id_set rule from above:

id_set

       : ID+

       ;

So far, everything works as expected. What I want to do is to use operator "+=" to collect all the IDs in a list. My main concern is that I want to keep the grammar G1 free of any application-specific code to make it reusable for different purposes, so I tried to modify the tree grammar in the following ways:

id_set

       : (ids += ID)+  //compile error: ‘struct ExprCppTreeWalker_Ctx_struct’ has no member named ‘vectors'

       ;

id_set

       : ids+=ID (ids+=ID)* //compile error: ‘struct ExprCppTreeWalker_Ctx_struct’ has no member named ‘vectors'

       ;

My question is: Is it possible to only augment the tree grammar such that I can collect elements in a list without touching the "original" grammar? If so, how?

Kind regards,

chris

[Jim Idle]

I think that you might need to go back in time for rewriting tree grammars to work. The templates were broken at one point and never got fixed if I remember correctly.

However, do you have the output=AST; 

Option set in your tree grammar? If you do not then I think you might not be able to use the += operator.

In general though, you are better off defining an interface for the external stuff and installing a pointer to this interface then calling indirectly through function pointers.  Then your grammar stays the same and you can install different tree handlers. Accumulating the ID tokens is non-specific but you can also do:

id_set

  : (id = ID { *if->accumulateIdSet($id); })+ { *if->finalizeIdSet(); /* may not be needed? */ ;

Jim

--
You received this message because you are subscribed to the Google Groups "antlr-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to antlr-discussi...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Chris McMaiersen]

Hi,

Am Dienstag, 22. April 2014 03:02:09 UTC+2 schrieb Jim Idle:

I think that you might need to go back in time for rewriting tree grammars to work. The templates were broken at one point and never got fixed if I remember correctly.

And by "go back in time" you mean use an older version of antlr?

 

However, do you have the output=AST; 

Option set in your tree grammar? If you do not then I think you might not be able to use the += operator.

No, the AST option is only set in the grammar for parsing the input text, not in the tree grammar.

[edit] I tried setting the AST option for the tree grammer, it results in the following error:

template error: ASTTreeParser.stg 321:25: anonymous template has 0 arg(s) but mapped across 1 value(s)

I tried antlr 3.5.2 and antlr 3.5.

 

In general though, you are better off defining an interface for the external stuff and installing a pointer to this interface then calling indirectly through function pointers.  Then your grammar stays the same and you can install different tree handlers. Accumulating the ID tokens is non-specific but you can also do: [...]

 I currently struggle a bit because I try to build a clean interface to get all the parsed data into nice C++ classes to avoid passing that nasty C stuff around in my application, but I haven't figured out the best way to do this by now... Any advices would be very welcome! ;-)

Jim

 chris

[Jim Idle]

On Tue, Apr 22, 2014 at 6:26 PM, Chris McMaiersen <cmaier....@gmail.com> wrote:

Hi,

Am Dienstag, 22. April 2014 03:02:09 UTC+2 schrieb Jim Idle:

I think that you might need to go back in time for rewriting tree grammars to work. The templates were broken at one point and never got fixed if I remember correctly.

And by "go back in time" you mean use an older version of antlr?

Yes,

 

 

However, do you have the output=AST; 

Option set in your tree grammar? If you do not then I think you might not be able to use the += operator.

No, the AST option is only set in the grammar for parsing the input text, not in the tree grammar.

Then you will need to turn that on in your tree grammar to be able to use += I think. But as I say, That's probably too much overhead as you are not really re-writing trees. 

 

In general though, you are better off defining an interface for the external stuff and installing a pointer to this interface then calling indirectly through function pointers.  Then your grammar stays the same and you can install different tree handlers. Accumulating the ID tokens is non-specific but you can also do: [...]

 I currently struggle a bit because I try to build a clean interface to get all the parsed data into nice C++ classes to avoid passing that nasty C stuff around in my application, but I haven't figured out the best way to do this by now... Any advices would be very welcome! ;-)

Just copy the style of the interfaces in the C runtime - create a .h file with a struct that defines the interfaces as a list of pointers to functions, then populate a struct with pointers to your C++ methods marked as extern "C", but write everything in C++. The methods you call can build your C++ objects as a model of the program (or whatever it is you are parsing). Pass the pointer to structure in to the parser and make it available globally to your parser actions. Create macros to make the calls to your interface a little cleaner looking. Your action code should not contain any logic at all, just accumulate the token pointers you need and pass them off to your interface functions. This way if you want to reuse the tree walker to do something else with the same tree, you can just pass in a different interface. It isn't as neat as C++ inheritance etc, but it is quick and neat enough - once you have the grammar doing the interface calls you will only touch it if you change the grammar.

Jim

 

Jim

 chris

[Chris McMaiersen]

Hi,

Am Mittwoch, 23. April 2014 03:18:32 UTC+2 schrieb Jim Idle:

Then you will need to turn that on in your tree grammar to be able to use += I think. But as I say, That's probably too much overhead as you are not really re-writing trees. 

After giving some more thought to it I tend to agree...  A tree grammar may be the wrong weapon to deal with my problem.

Just copy the style of the interfaces in the C runtime - create a .h file with a struct that defines the interfaces as a list of pointers to functions, then populate a struct with pointers to your C++ methods marked as extern "C", but write everything in C++. The methods you call can build your C++ objects as a model of the program (or whatever it is you are parsing). Pass the pointer to structure in to the parser and make it available globally to your parser actions. Create macros to make the calls to your interface a little cleaner looking. Your action code should not contain any logic at all, just accumulate the token pointers you need and pass them off to your interface functions. This way if you want to reuse the tree walker to do something else with the same tree, you can just pass in a different interface. It isn't as neat as C++ inheritance etc, but it is quick and neat enough - once you have the grammar doing the interface calls you will only touch it if you change the grammar.

Thank you very much for your advice, that design sounds pretty neat to me! :)  

[miku...@math.tu-berlin.de]

Hi guys,

I apologize for the dull question but I am bit confused how I could create an interface to have a C++ class and invoke parser / tree walker functions in C.

Do you have a simple example a la 'Evaluating Expressions via an AST' from A Quick Tour for the Impatient - ANTLR3 reference?

Thank you in advance!

Patryk

[Chris McMaiersen]

Hi,

Am Donnerstag, 15. Mai 2014 05:27:12 UTC+2 schrieb miku...@math.tu-berlin.de:

I apologize for the dull question but I am bit confused how I could create an interface to have a C++ class and invoke parser / tree walker functions in C.

I'm not exactly sure what your issues are, but what helped me to get started can be found at [1] and [2].

 

Thank you in advance!

HTH.

 

Patryk

Chris 

[1] http://contrapunctus.net/blog/2012/antlr-c

[2] http://stackoverflow.com/questions/8537214/a-simple-antlr-3-4-example-for-c-target-runtime

[Jim Idle]

You can also download the examples tar. 

--