Modal lexers have been around at least since Lex, and are really handy to support a BNF grammar. The documentation in The Definitive ANTLR Reference is a bit sketchy, as are on-line resources. I have attached a very simple Maven project that demonstrates why I am uncertain.
There is one lexer grammar with 2 modes, and a parser grammar. The parser grammar includes the tokens from the lexer grammar. (If I import the lexer grammar from an import directory, then the non-default mode never gets created. From another thread in this discussion group, that sounds like a known issue.)
Here are the contents:
ModalLexer.g4:
lexer grammar ModalLexer;
ID : [A-Z-a-z]+ ;
INT : [0-9]+ ;
WS : [ \t\r\n]+ -> skip;
MODE_ON : '*-' -> pushMode (MyMode) ;
MODE_OFF : '-*' -> popMode ;
mode MyMode ;
MYMODE_WS : [ \t\r\n]+ -> skip;
FOO : 'foo' ;
FOO_INT : [0-9]+ ;
ModalGrammar.g4:
parser grammar ModalParser;
options {
tokenVocab=ModalLexer;
}
stmt :
ID
| INT
| MODE_ON FOO FOO_INT MODE_OFF
;
The target that is built with maven is a simple executable that takes a string to parse on the command line.
Command lines like the following succeed:
~/experiment/antlr4: mvn exec:java -Dexec.mainClass=com.foo.ParseThis -Dexec.args="foo2"
~/experiment/antlr4: mvn exec:java -Dexec.mainClass=com.foo.ParseThis -Dexec.args="22"
~/experiment/antlr4: mvn exec:java -Dexec.mainClass=com.foo.ParseThis -Dexec.args="foo"
The following fails:
~/experiment/antlr4: mvn exec:java -Dexec.mainClass=com.foo.ParseThis -Dexec.args="*- foo 32 -*"
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building test-antlr4-modal 0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- exec-maven-plugin:1.4.0:java (default-cli) @ test-antlr4-modal ---
line 1:2 mismatched input '<EOF>' expecting 'foo'
Particulars:
- JDK 1.8_60
- Mac running El Capitan
- Maven 3.3
If this test program is incorrectly formulated, what would be the recommended way to have a modal lexer work with a parser grammar?
Thanks for your assistance!