C++ Antlr Warm-up time for 1000s of unit tests

[Alainmarcel]

Hi Mike,

Please let us know if there is a way to serialize a sample state to minimize the warm-up time:

https://github.com/alainmarcel/Surelog/issues/117

We have 1000s of unit tests that would only require a couple of milliseconds to parse (when warm), but instead take 0.5 seconds each because the warm-up time.

Is there a way to use a default state saved on disk to minimize warm-up?

Thanks

Alain

[Mike Lischke]

There's no such feature to serialize and unserialize the cached states. What could perhaps be improved is the way how the shared_ptr instances are used. There's a highly recursive approach in the prediction machinery which passes several such pointers around. I have no numbers that would indicate that these calls are the main culprit, but from experience I know this is a good candidate.

Mike
-- 
www.soft-gems.net

[Alainmarcel]

We might be able to contribute such a feature using Flatbuffer or one of the Google flavors. What Data structures should we be looking for to serialize?

[Mike Lischke]

We might be able to contribute such a feature using Flatbuffer or one of the Google flavors. What Data structures should we be looking for to serialize?

I believe it's just the DFA. The construction of it is also the only place in the ANTLR4 C++ runtime where we use a lock to serialize access (the DFA is shared between all instance of a particular parser/lexer class). See also the adaptivePredict function in the ATNSimulator classes (there's one for parsers and for lexers). This function does the lookahead and creates the DFA cache on the way.

Note: there are DFASerializer and LexerDFASerializer classes, which however only create string expressions of the DFA, for human consumption. But they could perhaps serve as starting point - at least for the serialization functionality.

Mike
-- 
www.soft-gems.net