ANN: New ANTLR4 TypeScript runtime

[Mike Lischke]

What is this about?

It's my pleasure to announce the immediate availability of a new (full) TypeScript runtime for ANTLR4, named antlr4ng, where the ng appendix stands for "Next Generation", which is a hint to a new project I call ANTLRng. However, this announcement is for the new runtime, which fits seamlessly into the existing ANTLR4 landscape. It is available as a Node.js package (https://www.npmjs.com/package/antlr4ng) with a size of just 140KB of minified JavaScript code, ready for direct import into websites or for use in your Node.js projects. The homepage of antlr4ng is https://github.com/mike-lischke/antlr4ng.

The new runtime passes the ANTLR4 test suite completely (with some changes to work with the new runtime) and is already used in several projects (antlr4-c3, the ANTLR4 VS Code extension and the MySQL Shell GUI extension for VS Code). To generate parsers and lexers compatible with the new runtime, use the (also new) antlr4ng-cli node package (https://www.npmjs.com/package/antlr4ng-cli).

Some more details

The new runtime is based on the great JavaScript runtime written mostly by Eric Vernaud and to a small extent on the long-time top dog antlr4ts. It contains numerous improvements and bug fixes, as well as omissions found in the JS runtime, is closer to the Java runtime than the other two, and is entirely written in TypeScript. It's compatible with the latest ANTLR4 version and released under the same license (BSD-3-clause).

However, the new runtime will not become a part of the ANTLR4 ecosystem, but run on its own (just like antlr4ts does). If you want to see what I changed in ANTLR4 for the new runtime check my fork (https://github.com/mike-lischke/antlr4/commits/dev, it's not much actually).

Give it a try, if you need a real TS runtime and let me know what you think about it.

Thanks,

Mike
-- 
www.soft-gems.net

[Michael Toy]

curious why yet another fork, the antlr4ts vs antlr4 fork is already a problem f0r me because i went with antlr4ts when it was the only choice, and then antlr4 typescript target showed up incompatoble and non functionhal when i tried to switch, leaving me stranded on antlr4ts until i can spend the time to debug antlr js. i have limited time and i would like to land my language somewhere that is going to have good long term support and this muddies the waters for me.

let 1000 flowers bloom, its' all open source, but it would be helpful for to know, why there are three forks and how a person would decide between the three.

[Mike Lischke]

curious why yet another fork, the antlr4ts vs antlr4 fork is already a problem f0r me because i went with antlr4ts when it was the only choice, and then antlr4 typescript target showed up incompatoble and non functionhal when i tried to switch, leaving me stranded on antlr4ts until i can spend the time to debug antlr js. i have limited time and i would like to land my language somewhere that is going to have good long term support and this muddies the waters for me.

let 1000 flowers bloom, its' all open source, but it would be helpful for to know, why there are three forks and how a person would decide between the three.

You already quoted most of the explanation (btw. where's that cited from?). The TS version in ANTLR4 (which is not a fork btw.) is type definitions on top of the JS runtime. Additionally, the generator template and the test infrastructure have been updated for the generated TS code (which is in fact pure TS). 

The antlr4ts fork is a special version, not compatible with ANTLR4, as it contains some special optimisations, which make it sometimes faster, sometimes slower than the original code. I used it for years but its development has been stopped (it's still an alpha version and effectively abandoned) and hence it is missing new features like case insensitive rules.

For these reasons I started a new TS runtime (my second runtime, after C++) and tried WebAssembly to get the speed of C++ (https://github.com/mike-lischke/antlr4wasm). This approach, however, is problematic as it is slower than the JS runtime and has lots of memory and exception handling problems. Measuring speed for that new runtime showed me how fast the JS runtime actually is. I was pretty surprised. That's why I decided to create the new TS runtime using the existing JS runtime as start (and stop wasting my time with the wasm attempt).

So, considering all this it should be easy to decide what to use. If you want to stay on the safe side and are not affected by all the limitations of the current "TS" runtime, then go with the main ANTLR4 version. In all other cases I recommend considering my new TS runtime.

Mike
-- 
www.soft-gems.net

[Adi Barda]

Hi Mike,
I'm using  antlr4ts for a long time now and considering moving to your new antlr4ng lib (Thank you for creating & sharing it BTW).
I would like to estimate the effort for porting my code to your new lib? is there any compatibility at all with antlr4ts or I will have to change pretty much all the  antlr4ts  related code?

Thanks,
Adi 

[Mike Lischke]

Hi Adi,

I'm using  antlr4ts for a long time now and considering moving to your new antlr4ng lib (Thank you for creating & sharing it BTW).

I would like to estimate the effort for porting my code to your new lib? is there any compatibility at all with antlr4ts or I will have to change pretty much all the  antlr4ts  related code?

I'm pretty sure you can do the migration within an hour or two. Most time you will need is to do the first steps (integrating into your build pipeline, searching for members with different names) and for testing after the migration. Also you need to get familiar with antlr4ng-cli to generate your parsers, but also that is almost the same like with antlr4ts.

Hop over to the Github repo for the runtime to ask questions, if needed.

Mike
-- 
www.soft-gems.net