parser works under Java but does not under C++, how to debug?

[Michal Orsák]

Hello,

https://github.com/Nic30/HWToolkit/tree/cppHdlParser/hdl_toolkit/parser/hdlConvertor

I have written a parser for VHDL and Verilog in Java, ANTLR4. (I mean  parsers generated by antlr. )
Recently I made a C++ version. I used ANTLR4 runtime (I used antlr4-4.5.4-SNAPSHOT.jar 21.June 2016).
Most of the parser works, but there are some parts which does not.

I know about some rules which are not resolved correctly.

For example, for rule (vhdl.g4 from git ablowe):
package_body
  : PACKAGE BODY identifier IS
    package_body_declarative_part
    END ( PACKAGE BODY )? ( identifier )? SEMI
  ;

with input:
package body misc_pkg is
    function zeros(count : natural; width : natural) return std_logic_vector
    is  -- there is line 14
        constant vec : std_logic_vector(count * width - 1 downto 0) := (others => '0');
    begin
        return vec;
    end function;
....

I got errors:
14:1:Error:mismatched input 'is' expecting {RANGE, TOLERANCE, BASIC_IDENTIFIER, EXTENDED_IDENTIFIER, ';', '(', '.'}
16:1:Error:extraneous input 'begin' expecting {ALIAS, CONSTANT, END, FILE, FUNCTION, GROUP, IMPURE, PROCEDURE, PURE, SHARED, SUBTYPE, TYPE, USE, VARIABLE}
18:5:Error:extraneous input 'function' expecting ';'
20:1:Error:extraneous input 'function' expecting {<EOF>, ARCHITECTURE, CONFIGURATION, ENTITY, LIBRARY, PACKAGE, USE}
24:16:Error:mismatched input ';' expecting {BASIC_IDENTIFIER, EXTENDED_IDENTIFIER}

Input is correct, parser in java does accept it.
In c++ I got errors ablowe and package_body_declarative_part is null.

Can someone give me advice how to debug such a bugs?
It seems that parser is not generated correctly, but I don't know how to find out, what is wrong.

Thanks!

[Mike Lischke]

Hi Michal,

https://github.com/Nic30/HWToolkit/tree/cppHdlParser/hdl_toolkit/parser/hdlConvertor

I have written a parser for VHDL and Verilog in Java, ANTLR4. (I mean  parsers generated by antlr. )
Recently I made a C++ version. I used ANTLR4 runtime (I used antlr4-4.5.4-SNAPSHOT.jar 21.June 2016).
Most of the parser works, but there are some parts which does not.

...

Can someone give me advice how to debug such a bugs?
It seems that parser is not generated correctly, but I don't know how to find out, what is wrong.

That's a tricky one. Maybe there is still a bug hidden, that shows up under a very specific situation, even tho all runtime tests passed (and they test many possible grammar constructs). The best way I found to debug problems like that is to step through the Java parser and at the same time through the C++ parser and see what paths they take.

You can start with the parser workflow, without digging into the prediction process and see when things go out of synch. This is easy because of the recursive-decent nature of the generated parser. Once you identified the deviation point you can then dig deeper to see why different predictions where found. Also try to minimize the grammar, so you get a small parser to debug.

Mike
-- 
www.soft-gems.net

[Devlin Poster]

https://github.com/Nic30/HWToolkit/tree/cppHdlParser/hdl_toolkit/parser/hdlConvertor

I have written a parser for VHDL and Verilog in Java, ANTLR4. (I mean  parsers generated by antlr. )
Recently I made a C++ version. I used ANTLR4 runtime (I used antlr4-4.5.4-SNAPSHOT.jar 21.June 2016).
Most of the parser works, but there are some parts which does not.

Michal,

I've encountered a similar bug, but haven't had time to look into it.

Please consider adding an issue to https://github.com/DanMcLaughlin/antlr4/issues so we can discuss it and compare notes.

Thanks!