Lexer generated for python target including java code, is this a bug?

63 views
Skip to first unread message

BPL

unread,
Jun 6, 2019, 12:24:44 PM6/6/19
to antlr-discussion
Hi everyone, this is my first time posting in the group and also using Antlr, I must to say this is a really awesome project and I'd like to use
it eventually on my python+qt software.

Right now I'm basically toying around while reading the docs, one of my first experiments was to play with https://github.com/antlr/grammars-v4/blob/master/python3/Python3.g4 , 
after I've generated all the python content using `antlr Python3.g4 -Dlanguage=Python3` I've got a lexer which it's unusable with python + https://pypi.org/project/antlr4-python3-runtime/ 
as the lexer will contain all the java inlined code in the python files :O

Please take a look Python3Lexer.py (file truncated for some reason when uploaded to dropbox but you'll see some java code in there).

Probably my question is silly but... is this an antlr bug? Or it's just me not calling correctly antlr? 

Or it's just maybe grammar that's not intended to be used with the python runtime? Although the docs says: 

An ANTLR4 grammar for Python 3 based on version 3.6 of The Python Language Reference.
This grammar has been tested against the Python 3's standard library, the contents of the asyncio folder there is included in the examples section.
Note that there are two grammars here, one for Java and other for Python3 target. Target of grammars are mentioned at the top of the grammar. This grammar contains embedded code that handles the insertion of INDENT and DEDENT tokens. The embedded code, located inside the NEWLINE lexer rule as well as the @lexer::members section is well documented, so people trying to port this grammar to another target should not have much difficulty.

Thanks in advance! 

michal.o

unread,
Jun 6, 2019, 10:10:42 PM6/6/19
to antlr-di...@googlegroups.com
Hi spscener84,

this is a correct function of ANTL4, the grammar is not written as a target independent.
There is simply hardcoded java in the grammar. This sections are not translated and are used as is.

The doc probably refers to some other file.


Michal.
--
You received this message because you are subscribed to the Google Groups "antlr-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to antlr-discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/antlr-discussion/c6be2109-e82d-407f-ac9e-6a3bdb5c1b5c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


BPL

unread,
Jun 7, 2019, 2:57:24 AM6/7/19
to antlr-discussion
Hi Michal,
 
this is a correct function of ANTL4, the grammar is not written as a target independent.
There is simply hardcoded java in the grammar. This sections are not translated and are used as is.

The doc probably refers to some other file.

Reply all
Reply to author
Forward
0 new messages