Possible ANTLRv4 labeling issue with "+="

[Sebastian YEPES]

Hello,

Currently trying to migrate some ANTLR grammar with embedded actions to the new visitor/listener model. Basically I generate a data structure from the input file for a post treatment.
I have made a very simplified example to illustrate the problem:

Example input file:
--------------
LOGNAME "Error_logfile" DESC "Description blabla"
  LOGPATH "/var/log/dummy.log"
  TYPE "file"
  FAST_READ
  ONLY_ONCE

Output:
--------------
ds = {"LOGNAME" : "Error_logfile",
      "DESC" : "Description blabla",
      "LOGPATH" : "/var/log/dummy.log",
      "TYPE" : "file",
      "FAST_READ" : true,
      "ONLY_ONCE" : true };

The issue I am facing is that I am using the labeling operator "+=" and it seems to have a bug or at least, I think so.
If we check the generated parser code from the following grammar and have a closure look at the class "LogoptContext", why is it defining the tokens with there id's and not there names?

The issue I am facing is that I am using the labeling operator "+=" and it seems to have a bug or at least, I think so.
If we check the generated parser code from the following grammar and have a closure look at the class "LogoptContext", why is it defining the tokens with there id's and not there names?FileReadIssue.g
--------------
grammar FileReadIssue;

file:         logfile;

// Header info
logfile:      'LOGNAME' lname=string 'DESC' ldesc=string logopt;

// Specific settings can be either strings, numbers or booleans
logopt:       ('LOGPATH' lpath=string
              |'TYPE' ltype=string
              |'DELAY' ldelay=number
              |lopts+='FAST_READ'
              |lopts+='ONLY_ONCE'
              |lopts+='READ_ONLY'
              )+
              ;
// Standard ANTLR lexer rules
number:       INT | FLOAT;
string:       ID | STRING;

FileReadIssueParser.java
--------------
    public static class LogoptContext extends ParserRuleContext {
        public StringContext lpath;
        public StringContext ltype;
        public NumberContext ldelay;
        public Token 6;  // <-- Why is it not using the TokenName: FAST_READ?
        public List<Token> lopts = new ArrayList<Token>();
        public Token 3; // <-- Why is it not using the TokenName: ONLY_ONCE?
        public Token 1; // <-- Why is it not using the TokenName: READ_ONLY? 

Compilation error:
--------------
antlr -no-listener -visitor FileReadIssue.g
javac -cp ./antlr-4.0b4-complete.jar *.java
FileReadIssueParser.java:132: error: <identifier> expected
                public Token 6;
                            ^
FileReadIssueParser.java:134: error: <identifier> expected
                public Token 3;
                            ^
FileReadIssueParser.java:135: error: <identifier> expected
                public Token 1;
                            ^
FileReadIssueParser.java:193: error: not a statement
                                        setState(24); ((LogoptContext)_localctx).6 = match(6);
                                                      ^
..                                        ((LogoptContext)_localctx).lopts.add(((LogoptContext)_localctx).1);
....
..
FileReadIssueParser.java:206: error: illegal start of expression
                                        ((LogoptContext)_localctx).lopts.add(((LogoptContext)_localctx).1);
                                                                                                         ^
15 errors

Questions:

• Am I correctly using the lable "+=" operator correctly?
  
• Is there a better solution resolve this kind grammar problematic?
  

Best regards and thanks in advance for any suggestions.

[Jim Idle]

In general, I recommend that you don't use 'literals' in your grammar for other reasons (see antlr.markmail.org) so changing to:

xxx: CCC;

CCC: 'CCC';

Will get you around that I suspect. 

However, this does look like a bug. In v3 you could not use the same label for += on different token types - not sure if this is still the case for v4.

Jim

--
 
 

[Sam Harwell]

This issue was fixed in the following commit:

https://github.com/antlr/antlr4/commit/35202df715180fd70add7bcd5c1deaa930330d6c