Lexer predicates in Antlr 3

25 views
Skip to first unread message

Anakreontas Mentis

unread,
May 8, 2015, 6:40:17 AM5/8/15
to antlr-di...@googlegroups.com
Hello.

I am developing a lexer which will be used  in XText. The problem at hand could be nicely solved with lexer modes, but XText uses antlr version 3.

The language supports string literals. However, when a string literal begins with character '@', the lexer should tokenize the content of the string. For example for input
"This is a string" I should get a single token while for
"@name" I must get two, token '@' and token 'name' with type ID.

I use a boolean variable to record if the lexer is inside a string which should be parsed (the second input) and the following rule:

T_STRING   :
      {inQuotes}? '"' { skip(); inQuotes = false; }
    | {!inQuotes}? 
      '"'
        (
          '"'  //empty string
        | '@' { $type = T_SCRIPT; inQuotes = true; inScript = true; }  //special case
        | ~'@' ('\\"' | ~('"' | '\r' | '\n'))* '"' //a non empty string
        )
    ;

My understanding is that an alternative is followed only when the predicate holds. This should be wrong however because I get an error stating:
..... rule T_STRING failed predicate: {!inQuotes}?

Could you please explain what the predicate in a rule alternative means?

Thank you in advance,

Anakreontas
Reply all
Reply to author
Forward
0 new messages