Mismatched input error in nested C# like generic. Why does it fail?

[Milo Veg]

Hi,

my grammar fails in a way I did not expect. I have boiled it down to a minimal example where I try to parse 'type':

grammar Grammar;

type : NAME ('<' type (',' type)* '>')?;

operator : '>>';

NAME : ([a-zA-Z_]+[a-zA-Z_0-9]*);

WHITESPACE : (' '|'\r'|'\t'|'\n') -> skip;

The input string to test the error:
GenericName<A, B<C>>

It fails with:
line 1:18 mismatched input '>>' expecting '<'

Obviously, Antlr parses the operator instead of the '>' of the type. If I remove the operator, it works just fine. It also seems to identify the type just fine. Why does it try to find the operator here? I'm using Antlr 4.9.2. I did not find this case when I searched. Sorry if I created a duplicate here.



Thanks
Milo

[John B Brodie]

Greetings!

ANTLR Lexers operate without any context information from the Parser. In fact, in the usual control stream, the entire source text input is processed by the Lexer before the Parse is even invoked.

I would suggest looking for other existing ANTLR grammars for languages that have this same feature, eg C++, C#, Java or whatever, and see how those grammars deal with this issue.

Another wild possibility would be to have the operator rule be:

operator : '>' '>' ;

this is bad because any skip'd or HIDDEN tokens found by the Lexer would be accepted between the '>'s. Eg. '>      >' or '>/*comment here*/>' would be treated as valid.

Off the top of my head, using the above rule with a semantic predicate that compares the source stream locations of the 2 '>'s to ensure that they are adjacent might work. But, ofc, i haven't tried to make that work.

Best to look for the solution in an existing grammar for a language with this feature.

Hope this helps...

   -jbb

--
You received this message because you are subscribed to the Google Groups "antlr-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to antlr-discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/antlr-discussion/1a60b5b4-0a3d-4848-9b36-e2f2c70c61afn%40googlegroups.com.