Manually creating semantic trees and/or sub-trees

[Steve Ebersole]

I am back looking at Hibernate and Antlr4.  In that effort we have started a proof-of-concept to start concretely playing with our grammar and Antlr4.  We want to simultaneously look at 2 solutions, one using re-writing and another using decoration.  I am working on the re-writing proof (because I honestly completely do not understand the decoration approach).

I have a grammar, and have generated the listeners I need.  I have successfully parsed a number of queries, and have passed those trees through multiple listeners.  So I have those basics down.  

The listener I am working on right now handles explicit from-clause elements (this is a query language parser).  There are implicit from-clause elements too, but the key to understanding the query overall is understanding its explicit from-clauses.  So what I have this listeber doing at the moment is to build a separate model representing the FromClause and friends.  Again, that part works pretty well.

Ultimately what I am thinking is to have this produce a new parse/semantic tree.  I am wanting to write a second grammar that describes the structure of that tree solely for the purpose of generating listeners/visitors.  Feel free to stop me here if this is not reasonable :)  The difficulty I am having is that the generated listeners/visitors are expecting "ParserRuleContext" subclass arguments which because they are subclasses have specific constructor expectations which I am totally unfamiliar with.  So assuming this is a valid way to attack this problem, how would I go about manually building this tree of "ParserRuleContext" objects?

Thanks!

[Terence Parr]

Hi Steve, I’m not totally sure I understand what you’re trying to build. If you’re trying to build a new tree, I would suggest creating a visitor whose return value type is YourNewTreeNodeType, rather than trying to mimic ANTLR’s automatic mechanism. In other words, roll your own tree node types and then construct the tree as you want with your own constructors.

Ter

[Steve Ebersole]

Hi Ter,

Let me back up :)

We are moving initial semantic analysis of HQL/JPQL into a sub-project.  The reason being that lots of projects we work on need these initial steps; need a resolved semantic understanding of the HQL/JPQL.  Each project however ultimately does different stuff with that.  Individual projects might need to render that to SQL or to various NoSQL queries or to Lucene searches or ...

It is my opinion (and I could be wrong) that it is not appropriate for consumers of this HQL parser project to have to know/understand this initial parse tree.  It contains generic rules at many points where semantically there are multiple "alternatives" where the alternatives cannot be decided between until later.  I strongly believe that the consumer should not have to deal with that, that they should really just have to deal with the semantic tree.  However, just returning the tree still means that each consumer needs to handle walking that semantic tree itself.  Antlr 4 generates really easy to use listeners which work great.  Which got me thinking I'd really like to be able to produce an Antlr-generated listener for this semantic tree for use by the consumers in walking the returned tree.  Granted I could just write my semantic tree sans any reference to Antlr and then write my own listener/walker.  But again, this is stuff that Antlr excels at.  So what I'd like to do is to build that semantic tree such that a second Antlr grammar can produce listeners for it.  The reason for the second grammar is again that IMO it is not appropriate for consumers of this HQL parser project to have to know/understand the initial parse tree.

I have been using a listener rather than a visitor for processing the initial parse tree.  It is building a semantic tree.  That part is not the problem.  Its that "next step" that is the problem; allowing the consumers of this project to more easily consume the semantic tree.  But I have no idea how to mock or provide these ParserRuleContext objects to the nodes in my tree.  Again, I get that I could just build my tree without extending or using ParserRuleContext.  In fact I already have that largely working.   I am just trying to see the feasibility in having Antlr generate listeners for that semantic tree.  I also realize I could make my semantic tree be an Antlr3-compliant tree and write a tree grammar for walking it, but trying to stick to one version of Antlr.