How to draw syntax trees?

540 views
Skip to first unread message

BPL

unread,
Jun 8, 2019, 7:27:48 PM6/8/19
to antlr-discussion
Hi, I'm aware there is `org.antlr.v4.gui.TestRig` tool that allows you to draw trees, problem being it requires to generate java files and compile them. That's not
convenient in case you're just using the python runtime.

So, I'd like to know how can I use pydot to draw the trees? In fact, I'd also like to know how to pretty print trees using the python runtime (yeah, I'm aware of the existing
classmethod toStringTree.

I've found this snippet https://stackoverflow.com/a/23662164/3809375 but it doesn't work with antlr4 but just with antlr3 

Could anyone provide snippets that could help here?

Thanks.

Mike Lischke

unread,
Jun 9, 2019, 7:48:24 AM6/9/19
to antlr-discussion
Hi, I'm aware there is `org.antlr.v4.gui.TestRig` tool that allows you to draw trees, problem being it requires to generate java files and compile them. That's not
convenient in case you're just using the python runtime.

So, I'd like to know how can I use pydot to draw the trees? In fact, I'd also like to know how to pretty print trees using the python runtime (yeah, I'm aware of the existing
classmethod toStringTree.

Try my vscode extension for ANTLR4 (https://marketplace.visualstudio.com/items?itemName=mike-lischke.vscode-antlr4) which works independent of the target language by using the parser interpreter. Here's an example:




You can export that image to SVG.



BPL

unread,
Jun 10, 2019, 4:57:05 AM6/10/19
to antlr-discussion
Mike,
 
Try my vscode extension for ANTLR4 (https://marketplace.visualstudio.com/items?itemName=mike-lischke.vscode-antlr4) which works independent of the target language by using the parser interpreter. Here's an example:

Wow, that tool looks really handy, beautiful graphs you're generating out there, although I use SublimeText as a text editor.

That said, after trying for few days antlr4 I think I'm giving up with it already, while I can see a lot of advantages of using it:

- Large number of existing grammars with really easy/convenient syntax
- Books/Tutorials
- Clean runtime code 

I also see the generated parsers (at least in python) are extremely slow... I've tested one glsl parser with a 28kb fragment shader and the parsing would take ~1.9s to parse, that's crazy and impossible to use
for real-time editing.

BPL.

michal.o

unread,
Jun 10, 2019, 10:42:11 AM6/10/19
to antlr-di...@googlegroups.com
Hello BPL,
Yes, Python runtime is ridiculously slow. The difference between C++ and Python runtime is more than 3 orders (for vhdl grammar). Python runtime has to be implemented as C++-Python  extension in order to be usefull.

BPL.

Michal.


--
You received this message because you are subscribed to the Google Groups "antlr-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to antlr-discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/antlr-discussion/41b261aa-a0ae-4799-9d8a-eeb298ac529f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


BPL

unread,
Jun 10, 2019, 10:50:23 AM6/10/19
to antlr-discussion
Yes, Python runtime is ridiculously slow. The difference between C++ and Python runtime is more than 3 orders (for vhdl grammar). Python runtime has to be implemented as C++-Python  extension in order to be usefull.

Michal, thing is, I've seen there is tree-sitter which uses GLR parsing that's being used in atom and claims to be a magical solution to be integrated with IDEs, main problem being is a fairly new tech so there isn't
a lot of existing grammars, I was considering already that as a superior alternative to antlr4 but now I've got curious

when you say the c++ runtime is 3 orders of magnitude than the python rutime... do you mean by using the c++ runtime you'd be able to make parsing on realtime (ie: [50-100ms]/keystroke) with fairly large vhdl files? Please
let me know... if this problem was solvable by just using the c++ runtime as a python extension, well, that'd make me reconsider the transition from antlr4->tree-sitter

Thanks!

michal.o

unread,
Jun 10, 2019, 11:07:58 AM6/10/19
to antlr-di...@googlegroups.com

https://github.com/Nic30/hdlConvertor

It very depends on complexity of the file, but 100ms is realistic. But It may require a little bit of async and pipelining to work nicely.

(

Also:

* in hdlConvertor I just switched to vhdl-2008 grammar which is far from optimized yet, but still probably meets your perf. req.

* My knowledge of Python runtime is nearly 1y old, so something may changed.

* tree-sitter looks good, If you decide to use it I would be glad to read a review.

)



Thanks!

Michal.


--
You received this message because you are subscribed to the Google Groups "antlr-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to antlr-discussi...@googlegroups.com.

Win Pooh

unread,
Sep 6, 2019, 4:08:08 AM9/6/19
to antlr-discussion
Hi Mike, 
the extension looks great.

Does something like this for Visual Studio exist?



воскресенье, 9 июня 2019 г., 13:48:24 UTC+2 пользователь Mike Lischke написал:

Mike Lischke

unread,
Sep 6, 2019, 8:00:54 AM9/6/19
to antlr-discussion
the extension looks great.

Does something like this for Visual Studio exist?

There's an extension for Visual Studio from Sam Harwell, but I don't know if it draws syntax diagrams. ANTLR4 itself can export dot graphs, maybe that helps?

-- 
You received this message because you are subscribed to the Google Groups "antlr-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to antlr-discussi...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages