How to understand (debug) why ANTLR4 makes a certain choice?

89 views
Skip to first unread message

Kostya Y

unread,
Dec 4, 2022, 9:45:35 AM12/4/22
to antlr-discussion
Hey all
I have a syntax for a templating language I've defined that works most of the time.
However, sometimes an input causes unexpected matches which are not quite clear to me.
Is there some kind of debug/verbose mode I can use to understand why ANTLR decides something is a specific token A even though there is an error parsing it while token B is a perfect match?
I am familiar with the various IDE plugins/lab.antlr4.org, however none of those explain why ANTLR thinkins something is a token of this or that type.

Thanks in advance.

Andy Hewitt

unread,
Dec 4, 2022, 10:50:07 AM12/4/22
to antlr-di...@googlegroups.com

Hi, I would use gun in token mode.

 

Sent from Mail for Windows

--
You received this message because you are subscribed to the Google Groups "antlr-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to antlr-discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/antlr-discussion/a767e0ee-a27c-45ee-bf5c-6efbc2d9d717n%40googlegroups.com.

 

Kostya

unread,
Dec 4, 2022, 10:59:52 AM12/4/22
to antlr-di...@googlegroups.com
I'm not familiar with it and wasnt able to find any reference to it in the documentation.
Can you please elaborate?

You received this message because you are subscribed to a topic in the Google Groups "antlr-discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/antlr-discussion/45Z6EL8th9A/unsubscribe.
To unsubscribe from this group and all its topics, send an email to antlr-discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/antlr-discussion/638cc1ad.a70a0220.a35a6.27f2SMTPIN_ADDED_BROKEN%40gmr-mx.google.com.

Andy Hewitt

unread,
Dec 4, 2022, 11:11:38 AM12/4/22
to antlr-di...@googlegroups.com

Apologies its ‘grun’ email system replacement.

Ross Patterson

unread,
Dec 5, 2022, 10:52:22 AM12/5/22
to antlr-discussion
The "TestRig" tool shipped with ANTLR might help.  It's commonly called "grun", but you won't find any code by that name.  You run it from the command line via "java org.antlr.v4.runtime.misc.TestRig", after first compiling the grammar.  It has several options, but the one that will show you the list of tokens is "-tokens".  If the lexer is correctly determining the tokens, but the parser isn't using the syntax rule you expect, you might find the "-gui" option useful - it will open up a window showing the full parse tree.  That gets you the "what", but not the "why".  ANTLR doesn't appear to have a "tell me why you chose that parse alternative" capability.

Ross

Wanadoo

unread,
Dec 5, 2022, 11:02:44 AM12/5/22
to antlr-di...@googlegroups.com
You can now do this online in Antlr lab, no build required

Envoyé de mon iPhone

Le 5 déc. 2022 à 16:52, Ross Patterson <ross.pa...@gmail.com> a écrit :

The "TestRig" tool shipped with ANTLR might help.  It's commonly called "grun", but you won't find any code by that name.  You run it from the command line via "java org.antlr.v4.runtime.misc.TestRig", after first compiling the grammar.  It has several options, but the one that will show you the list of tokens is "-tokens".  If the lexer is correctly determining the tokens, but the parser isn't using the syntax rule you expect, you might find the "-gui" option useful - it will open up a window showing the full parse tree.  That gets you the "what", but not the "why".  ANTLR doesn't appear to have a "tell me why you chose that parse alternative" capability.
--
You received this message because you are subscribed to the Google Groups "antlr-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to antlr-discussi...@googlegroups.com.

Terence Parr

unread,
Dec 6, 2022, 12:13:52 PM12/6/22
to antlr-di...@googlegroups.com
Hi. There is a way in the intellij plugin to learn about the different possibilities for parsing but not lexing.  Sorry. All you know is that the lexer matches the longest token or, if ambiguous, the first defined in grammar.
Ter

--
You received this message because you are subscribed to the Google Groups "antlr-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to antlr-discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/antlr-discussion/a767e0ee-a27c-45ee-bf5c-6efbc2d9d717n%40googlegroups.com.


--
Dictation in use. Please excuse homophones, malapropisms, and nonsense.

Daniel Motz

unread,
Oct 17, 2024, 9:48:09 AM10/17/24
to antlr-discussion
Hey!
Now there is a Visual Studio Code Plugin by Mike Lischke: https://github.com/mike-lischke/vscode-antlr4/
I am employed in a project at the University of Jena, which is currently doing something very similar. The project is called πάρεδρος/Paredros. We are currently struggling with understanding the parsing process from the traces printed by the parser. Maybe you have a hint @mike?

Appended is a trace file, which we are currently working on. Some (important) steps are implicit, which gives us a real headache atm.
trace.txt

Daniel Motz

unread,
Oct 17, 2024, 9:52:49 AM10/17/24
to antlr-discussion
PS. Are there more flags we can turn on to get more information, especially about predictions, in the trace? Another option might be to turn off the ALL(*) paradigm and just use LL for debugging. But the flags do not seem to have any influence, at least in Python.

Mike Lischke

unread,
Oct 17, 2024, 10:17:20 AM10/17/24
to antlr-di...@googlegroups.com
Hey Daniel,

This list is just for announcements (although there aren't many announcements here these days). For questions I recommend using Stackoverflow (or the AI of your choice) and for discussions use https://github.com/antlr/antlr4/discussions (and mention me with my GH id @mike-lischke).

Reply all
Reply to author
Forward
0 new messages