webpack doesn't like 'static' properties in antler-generated classes

[Eben Haber]

Hi all!

 

I’ve been building a parser that I want to use in a web application. I created the parser using Antlr4, I’ve tested it with node.js, everything seems to work fine.

 

To use in my web application, I’m trying to use webpack as described here: https://github.com/antlr/antlr4/blob/master/doc/javascript-target.md

 

I’m using the latest versions of antrl4 and webpack 5.

 

When I run webpack, it complains about all the ‘static’ properties that are found in the antrl4-generated classes, with messages like the following:

 

Module parse failed: Unexpected token (1840:27)

You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders

| export default class n1qlParser extends antlr4.Parser {

| 

>     static grammarFileName = "n1ql.g4";

|     static literalNames = [ null, null, null, null, null, null, null, "'.'", 

 

Does anybody out there know how to get webpack to work with antlr4-generated parsers?

 

Thanks,

 

-Eben

[Eben Haber]

Alternately, is there some way to include the antlr runtime without using webpack? I tried installing the antlr4 npm package, and I get the error:

TypeError: Failed to resolve module specifier "fs". Relative references must start with either "/", "./", or "../".

[Mike Lischke]

Alternately, is there some way to include the antlr runtime without using webpack? I tried installing the antlr4 npm package, and I get the error:

TypeError: Failed to resolve module specifier "fs". Relative references must start with either "/", "./", or "../".

The antlr4 package won't help you here. Instead go a different route. I just answered a similar question on Stackoverflow (https://stackoverflow.com/questions/67104441/running-webpack-as-child-process-to-generate-source-files), so I copy-paste my answer here:

I recommend to go a different route here. That webpack loader has not been updated in the last 4 years, uses a pretty old ANTLR4 jar (current version is 9.2) and uses the Javascript runtime, which is known for certain problems.

Instead I recommend that you switch to the antlr4ts runtime and use antlr4ts-cli to generate your files from the grammar. Both are still marked as being alpha, but I have used these packages for years already (e.g. in my vscode extension vscode-antlr4).

With that in place you can remove the webpack loader and generate the parser/lexer files as part of your build process.

---- end of quote

The runtime you will use is antlr4ts (which also works with JS). In my package.json I added a call to a script to generate the source code using antlr4ts-cli:

"scripts": {

"launch-dev-server": "./scripts/launch-dev-server.sh 3001",

"launch-dev-server-win": "call scripts/launch-dev-server.cmd 3001",

"build": "./scripts/production-build.sh",

"build-win": "call scripts/production-build.cmd",

...

},

And that script only contains:

antlr4ts -no-visitor -Xexact-output-dir -o ./src/parsing/mysql/generated src/parsing/mysql/*.g4

It's not necessary to rebuild the parser source files every time you build your app. So this should usually be enough to get you going. You can then include ANTLR4 like this:

import { ParseTree } from "antlr4ts/tree";

import { CharStreams, BailErrorStrategy, DefaultErrorStrategy, CommonTokenStream } from "antlr4ts";

import { PredictionMode } from "antlr4ts/atn/PredictionMode";

import { ParseCancellationException } from "antlr4ts/misc";

For some more comfort you can integrate the generation of the parser source files also when running your dev server (I use React here, but guess this is the same also for pur web pack development). I use a different (and more complicated) script in this case to see if the grammar file is newer than the generated files and if so start the generation. Something along these lines:

target_path=./src/parsing/mysql/generated

run_generation=false

if [ ! -f $target_path/MySQLLexer.ts ] || [ ! -f $target_path/MySQLParser.ts ]

then

run_generation=true

else

# Files exist, now check their file time.

if [ $target_path/../MySQLLexer.g4 -nt $target_path/MySQLLexer.ts ] || [ $target_path/../MySQLParser.g4 -nt $target_path/MySQLParser.ts ]

then

run_generation=true

fi

fi

Mike

-- 
www.soft-gems.net

[eric vergnaud]

Mike,

sorry but I think this advice is misleading.

The TS runtime leverages the JS runtime so any issue affecting the JS runtime also affects the TS one, you were just lucky not to bump into them (I didn't either).

The issue here has nothing to do with the runtime itself.

Rather it relates to webpack 5, which is not currently supported.

The best advice is to use webpack 4.

Eric

[Mike Lischke]

Hey Eric,

sorry but I think this advice is misleading.

The TS runtime leverages the JS runtime so any issue affecting the JS runtime also affects the TS one, you were just lucky not to bump into them (I didn't either).

No, the TS runtime is a completely own implementation. It does not use the JS runtime in any way. Check the code https://github.com/tunnelvisionlabs/antlr4ts and you will see there's no reference to the JS runtime. In fact I heard considerations a while ago to dump the current JS runtime and use the TS runtime instead. The antlr4ts node module actually is Javascript, transpiled from the TS sources.

The issue here has nothing to do with the runtime itself.

Rather it relates to webpack 5, which is not currently supported.

The best advice is to use webpack 4.

OK, that's probably the right advice, if the OP wants to stay with the pure JS runtime.

Mike
-- 
www.soft-gems.net

[eric vergnaud]

I thought the TS runtime followed the design of this PR: https://github.com/antlr/antlr4/pull/1256

I would be reluctant recommending unofficial runtime targets...

[Mike Lischke]

I thought the TS runtime followed the design of this PR: https://github.com/antlr/antlr4/pull/1256

Sam and Burt have started that separate project and kept it separate since then. I recommended in the past to merge it to the main repo, but the guys preferred not to do so.

I would be reluctant recommending unofficial runtime targets...

In general I agree, but I'm working with that target several years now (don't know why the guys still consider it alpha, probably to be free to do radical changes if they like to). So I have first hand and long time experience with it and built several tools/products around it.

Mike
-- 
www.soft-gems.net

[Eben Haber]

Thank you Mike,

 

I was able to install antlr4ts and generate my parser as typescript. However, I have one additional question: what target do you use when compiling the resulting parser with tsc?

 

When I tried “ES5” it compiled, but throws an error when I test the resulting code with node.js:

 

Object.defineProperty(exports, "__esModule", { value: true });

                      ^

 

ReferenceError: exports is not defined

 

Yet when I try any higher target (ES6, ES2018, etc.), I get tsc compilation errors such as:

 

n1qlParser/n1qlParser.ts:23399:15 - error TS2339: Property 'getRuleContext' does not exist on type 'SavepointContext'.

 

23399  return this.getRuleContext(0, Savepoint_nameContext);

 

 

Thank you,

 

-Eben Haber

 

 

From: 'Mike Lischke' via antlr-discussion <antlr-di...@googlegroups.com>
Reply-To: "antlr-di...@googlegroups.com" <antlr-di...@googlegroups.com>
Date: Friday, April 16, 2021 at 12:16 AM
To: "'rtm...@googlemail.com' via antlr-discussion" <antlr-di...@googlegroups.com>
Subject: Re: [antlr-discussion] webpack doesn't like 'static' properties in antler-generated classes

 

EXTERNAL EMAIL:  Use caution when opening attachments or clicking on links

 

 

--
You received this message because you are subscribed to a topic in the Google Groups "antlr-discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/antlr-discussion/imkUQ33T0ac/unsubscribe.
To unsubscribe from this group and all its topics, send an email to antlr-discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/antlr-discussion/A96331D5-4DDD-4711-A42C-2F2E90DC5D72%40googlemail.com.

[Mike Lischke]

 

I was able to install antlr4ts and generate my parser as typescript. However, I have one additional question: what target do you use when compiling the resulting parser with tsc?

 

When I tried “ES5” it compiled, but throws an error when I test the resulting code with node.js:

 

Object.defineProperty(exports, "__esModule", { value: true });

                      ^

 

ReferenceError: exports is not defined

 

I use ES6 in my vscode extension https://github.com/mike-lischke/vscode-antlr4/blob/master/tsconfig.json and ES2020 in a new project of mine:

{

"compilerOptions": {

"target": "es2020",

"module": "esnext",

"allowJs": true,

"outDir": "./output/",

"removeComments": true,

"sourceMap": true,

"esModuleInterop": true,

"strict": true,

"forceConsistentCasingInFileNames": true,

"moduleResolution": "node",

"resolveJsonModule": true,

"isolatedModules": true,

"noEmit": true,

"noImplicitAny": true,

"suppressImplicitAnyIndexErrors": true,

"jsx": "react",

"sourceRoot": "src",

"lib": [

"es2020",

"dom"

],

"skipLibCheck": false,

"experimentalDecorators": false,

"allowSyntheticDefaultImports": true,

"strictPropertyInitialization": false,

"noFallthroughCasesInSwitch": true,

"emitDecoratorMetadata": false

},

"include": [

"src"

]

}

Yet when I try any higher target (ES6, ES2018, etc.), I get tsc compilation errors such as:

 

n1qlParser/n1qlParser.ts:23399:15 - error TS2339: Property 'getRuleContext' does not exist on type 'SavepointContext'.

 

23399  return this.getRuleContext(0, Savepoint_nameContext);

 

 

That's odd. The method is defined in ParserRuleContext.d.ts. Btw. You can usually get sub contexts also via properties, including those that appear multiple times (an array is then returned for you).

Mike
-- 
www.soft-gems.net