Can I programmatically change lexer modes?

[Ramon Mendes]

As said in the chapter 12 of the book about lexer modes:

One of the most important requirements for mode switching
is that the language have clear lexical sentinels that can trigger switching
back and forth, such as left and right angle brackets. To be clear, modes rely
on the fact that the lexer doesn’t need syntactic context to distinguish between
different regions in the input

I need to have my grammar to support 2 different modes: mode 1 where it parse the language itself, and mode 2 where it should consume everything else (any text). Problem is I don't have those 'lexical sentinels' for switching between these modes in the lexer, well, in fact I have some delimiters, which are curly brackets {}, however the mode 2 text can also contain curly brackets ...

Anyway, my question is, can I programmatically change the lexer mode instead of writing in the lexer grammar those '-> mode(ISLAND);' manual instructions.

Technically, can I call pushMode() in the middle of the lexing phase?

I guess I can if I use a BufferedTokenStream, as is said here: http://www.antlr.org/api/Java/index.html?org/antlr/v4/runtime/BufferedTokenStream.html

This last link even points out this situation where the 'parser/lexer has to set context/mode info before proper lexing of future tokens'

Sorry if my question is too messy

thks

[Sam Harwell]

Hi Ramon,

 

You can call pushMode any time, but be aware that it will only affect the next token, not the current token (unless you override nextToken and place the call to pushMode before you call super.nextToken).

 

Thank you,

Sam Harwell

--
You received this message because you are subscribed to the Google Groups "antlr-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to antlr-discussi...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.