ANTLR JavaScript runtime: Is it or isn't it a CommonJS module?

[JBK]

I have a working setup that generates Lexer+Parser for the JavaScript grammar with ANTLR version 4.8 for the JavaScript target. Unfortunately it suffers from a max stack call size exceeded issue - a known issue that was recently fixed (https://github.com/antlr/grammars-v4/issues/2007).
The system is running directly in the browser and is using requirejs to handle loading of the CommonJS runtime module files (require, export... etc).

I'm trying to update to the newest version 4.9.1 in the hope that it fixes the bug, but the 4.9+ versions of ANTLR seem to have changed to generating ES module-like code instead (import, class, default...etc). The base classes in the grammar have also been updated to use import rather than require. The documentation for it (https://github.com/antlr/antlr4/blob/master/doc/javascript-target.md) even mentions
import antlr4 from 'antlr4';
So it seems like the runtime should be an ESM too.

The JavaScript runtime distributed from the download page for version 4.9.1 (https://www.antlr.org/download/index.html) seems to be a CommonJS module. At least it has an index.js with code that looks like it.

I have tried webpacking the runtime directly from git but that too generates a CommonJS module which doesn't work with the generated code since the generated code uses import.

Am I missing a super obvious way of acquiring an ESM version of the ANTLR4 JavaScript runtime?

Or how are you supposed to use ANTLR 4.9+ directly in the browser?

-- JBK

[eric vergnaud]

You are not supposed to use ANTLR directly in the browser, rather you are expected to package the runtime with your code.

[Janus B]

Unfortunately the project I'm working on lives entirely in the browser, there is no server as such. Very limited prepackaging is available.

1) Users specify a grammar and lexer in the browser

2) ANTLR is run in a JavaScript-based JavaVM to compile the corresponding lexers and parsers in the browser

3) Requirejs is used to import the generated lexer+parser back into the page to build parse-trees, transpilers etc.

In step (2) it is optimal to use the JavaScript target since the JavaScript ANTLR runtime is orders of magnitude faster to run than using the Java target and Java-runtime with the JavaVM option in the browser to also parse the code in step (3). When using the JavaScript target the parser runs natively in JavaScript in the browser with no overhead whatsoever. In comparison the JavaVM adds a LOT of overhead.

This worked with Antlr 4.8 when using requirejs to import the CommonJS modules generated in combination with the CommonJS-based JavaScript runtime.

Unfortunately now with 4.9+ the generated code being ESM and the JavaScript runtime still being CommonJS the browsers can no longer figure out how to load the generated code.

Personally I would prefer to have both the generated code and the runtime be ESM rather than CommonJS - that way there would be no requirement for requirejs anymore.

But it seems we are stuck in a kinda half-way-there scenario right now.

-- JBK

--
You received this message because you are subscribed to the Google Groups "antlr-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to antlr-discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/antlr-discussion/51c43629-c101-4b84-8958-1109a32a2968n%40googlegroups.com.

[eric vergnaud]

There is definitely room for improvement in the JS runtime packaging.

Feel free to come up with a proposal that works for you (and doesn't break for other users).