How to parse a string in Python (not a file)?

125 views
Skip to first unread message

Natanji

unread,
Dec 10, 2017, 10:25:45 AM12/10/17
to antlr-discussion
I'm trying to use ANTLR in python3 for the first time (previously used it from Java) and I've run into a problem when trying to parse input that is not from a file, but from a string inside my program.

I've followed the wiki entry here: https://github.com/antlr/antlr4/blob/master/doc/python-target.md

That works, but it uses a FileStream as input, which requires the input to be in a file on disk. What I would like to do instead is parsing a normal python string. From what I've found so far, the StringIO class emulates the input stream behaviour of files - but apparently, it is not enough and cannot be used as the input for the Lexer. When I try to use a StringIO(myString) as the input for the Lexer, this crashes in Lexer.py, line 111, in nextToken:

    tokenStartMarker = self._input.mark()
AttributeError: '_io.StringIO' object has no attribute 'mark'

The way I understand is that apparently a FileStream object would have this attribute "mark", but a StringIO object doesn't have it?

Am I doing this completely wrong? Surely there must be a way to use ANTLR in Python without needing to access an external file.

Dave Parfitt

unread,
Dec 17, 2017, 7:12:47 PM12/17/17
to antlr-discussion
You can use antlr4.InputStream in place of FileStream to read from a string.

Cheers -
Dave
Reply all
Reply to author
Forward
0 new messages