Serializing anltr4 C++

[Alainmarcel]

Hi all,

I'd really like to be able to cache the parse tree on disk using say Google Flatbuffers and be able to run the Listener on the restored tree.

I have performances issues with the parse time.

Can I get some pointers on which DS needs to be persisted for this to succeed?

Thanks

Alain

[Stuart Maclean]

Hi there, I am not sure what 'DS' means in your post.

I am still using Antlr3, and I wanted to serialize the Trees produced by Antlr when the grammar is of the output=AST variety.  I noted that CommonTree was not java.io.Serializable, so I added add this to my .g file:

          /*
              CommonTree not serializable, so we subclass
              and supply the Serializable interface. Our
              SerializableTree object then created by way
              of our CommonTreeAdaptor instance set in the parser
            */
            static class SerializableTree extends CommonTree
                implements Serializable {
                SerializableTree( Token t ) {
                    super( t );
                }
            }  
 
                wincParser parser = new wincParser( tokens,
                    new BlankDebugEventListener() );
                parser.setTreeAdaptor( new CommonTreeAdaptor() {
                        @Override
                        public Object create( Token t ) {
                            return new SerializableTree( t );
                        }
                    } );

In my case, my source was not changing, so I had no need to parse over and over. 

I know this isn't quite what you were after, but I thought it related enough to share.

Stuart

[Alainmarcel]

Thanks Stuart,
I think I mostly solved my problem by instrumenting (automatically, via a script) all listener actions and creating a generic node object at each call bottom up (Node has a typedef for each rule, parent, child, sibling index structure) then saving/restoring that object tree via Flatbuffers.
On restore I can't use the antlr built in listener, but I can create my own listener/iterator. No need to re-parse constantly.
Alain