is there a way to attach my lexer to the generated parser?

[konstantin tantin]

I have a transition table driven lexer similar to that described in the "Dragon Book". I need to use it with ANTLR4 parser instead of a generated lexer. Are there any examples showing how to implement a custom lexer?

Thanks!

Konstantin

[Mike Lischke]

I have a transition table driven lexer similar to that described in the "Dragon Book". I need to use it with ANTLR4 parser instead of a generated lexer. Are there any examples showing how to implement a custom lexer?

This should not be a big problem. In your grammar (you need a split grammar in this case) you can define a base class to be used for your generated lexer:

options {

superClass = MyLexer;

}

In that lexer you have to implement the same interface as the standard lexer class provides, so that the parser can work with it. Most important is the `nextToken()` method, but a number of others must be supported as well, to make everything working. You can implement them step by step as needed to compile everything, by looking at the original implementation to see how it's done there.

Mike

-- 
www.soft-gems.net

[konstantin tantin]

Thank you. I am working with the Javascript target. Is `superClass` option used there as well? How does this work in Javascript?

[Mike Lischke]

Thank you. I am working with the Javascript target. Is `superClass` option used there as well? How does this work in Javascript?

Forget about the superClass option. That was nonsense. If you would use a grammar to generate a lexer, you wouldn't use your own.

The correct answer is in the second part of my reply. Change your custom lexer so that it provides the same interface as the one from ANTLR. It doesn't matter which target language you use, the general class structure in the runtime is the same. You could also wrap your custom lexer with a class that provides the interface and uses your custom lexer as source. The details depend on how your custom class is structured. Maybe you can also use it as token source instead of a full lexer.

[konstantin tantin]

пятница, 15 сентября 2017 г., 16:02:22 UTC-4 пользователь Mike Lischke написал:

Thank you. I am working with the Javascript target. Is `superClass` option used there as well? How does this work in Javascript?

Forget about the superClass option. That was nonsense. If you would use a grammar to generate a lexer, you wouldn't use your own.

In the generated Javascript code, the name that I give for the superClass option is used in the import statement that imports the lexer. So it must not  be complete nonsense...