Convert a "stopIndex" into line/charPositionInLine

[Michael Toy]

I tried to stare at code and figure this out, and so far I am failing. Is it possible to convert a "stopIndex" of a token into a line/char pair? If all tokens began and ended on the same line I could use the ".line" of the token, but there are tokens (e.g. multi-line strings) where the end of the token might be several lines away from the beginning of the token.

[Terence Parr]

Hi Michael!

Do you just have the stopIndex or do you have the token or treenode whence that came? Is that a token stop or a char stop? If a token index, you can ask the token stream to get that token object then ask for line/charPos.  Does this help?

If it’s a char index, I don’t think we have a function to compute line/pos.

Ter

On Feb 13, 2022, at 2:14 PM, 'Michael Toy' via antlr-discussion <antlr-di...@googlegroups.com> wrote:

I tried to stare at code and figure this out, and so far I am failing. Is it possible to convert a "stopIndex" of a token into a line/char pair? If all tokens began and ended on the same line I could use the ".line" of the token, but there are tokens (e.g. multi-line strings) where the end of the token might be several lines away from the beginning of the token.

--
You received this message because you are subscribed to the Google Groups "antlr-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to antlr-discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/antlr-discussion/CAK0V5025zGUfiJF9XRZUj%3DO53FVvMtzrPXmBJxUKmqH1E_us3w%40mail.gmail.com.

[Michael Toy]

I have a "Token" which is where I could get "line", and "charPositionInLine" for the start of the token ... it looks to me like Tokens know everything about where they start only the stopIndex of where they end and no way to map an index to a line/char, but I would love to be wrong about this.

To view this discussion on the web visit https://groups.google.com/d/msgid/antlr-discussion/0574AB94-901E-47BA-A263-713307770C62%40gmail.com.

[Terence Parr]

Poking around I don't see anything that will take a character index and return the line number. If you were trying to find the line and character position on the line for the end of a token, could you look at the next token and ask what it starting line/pos is? We need to be careful about the last character of the current token but something like this would work I think.

Ter

To view this discussion on the web visit https://groups.google.com/d/msgid/antlr-discussion/CAK0V503moETa3sP-C%3D9RZ0OVKER4cfbLMuwvpXcOgt7cSahBNQ%40mail.gmail.com.

[Michael Toy]

Thanks, that's the conclusion I came to, I was hoping i was wrong.

To view this discussion on the web visit https://groups.google.com/d/msgid/antlr-discussion/DE212FDF-02BA-43E7-941F-30F3FEECED18%40gmail.com.

[Love Leifland]

> If you were trying to find the line and character position on the line for the end of a token, could you look at the next token and ask what it starting line/pos is?

For anyone else that might reach here in the future, please note that there is a caveat with this approach when you allow white space in a hidden channel. In that situation the next token (that is not on the hidden channel) might not start on the same line as the previous (non hidden) token ended.

/Love