unable to require('telnet-client')

[Ben Liblit]

I am trying to process a tiny program that uses CommonJS require to load telnet-client as a dependency. The source file, stored as main.js:

    const { Telnet } = require('telnet-client')
    new Telnet().connect({})

The corresponding package.json to declare the dependency:

    {
      "name": "third-party",
      "main": "main.js",
      "version": "1.0.0",
      "dependencies": {
        "telnet-client": "^2.0.3"
      }
    }

I used npm install to install telnet-client and its transitive dependencies, which I now see under node_modules. However, all of my attempts to run Closure fail to load telnet-client, along with other errors. For example:

========

% java -jar ~/Downloads/closure-compiler-v20220601.jar --process_common_js_modules --module_resolution NODE .

--------

./main.js:1:19: ERROR - [JSC_JS_MODULE_LOAD_WARNING] Failed to load module "telnet-client"
  1| const { Telnet } = require('telnet-client')
                        ^

./node_modules/emitter-component/test/emitter.js:2:14: ERROR - [JSC_JS_MODULE_LOAD_WARNING] Failed to load module ".."
  2| var Emitter = require('..');
                   ^

./node_modules/stream/index.js:22:14: ERROR - [JSC_JS_MODULE_LOAD_WARNING] Failed to load module "emitter"
  22| var Emitter = require('emitter');
                    ^

./node_modules/telnet-client/lib/index.js:13:17:
Originally at:
node_modules/telnet-client/src/index.ts:1:0: ERROR - [JSC_JS_MODULE_LOAD_WARNING] Failed to load module "events"

4 error(s), 0 warning(s)

========

% java -jar ~/workplace/closure-compiler-v20220601.jar --process_common_js_modules --module_resolution NODE . '!**/test/**'

--------

./main.js:1:19: ERROR - [JSC_JS_MODULE_LOAD_WARNING] Failed to load module "telnet-client"
  1| const { Telnet } = require('telnet-client')
                        ^

./node_modules/stream/index.js:22:14: ERROR - [JSC_JS_MODULE_LOAD_WARNING] Failed to load module "emitter"
  22| var Emitter = require('emitter');
                    ^

./node_modules/telnet-client/lib/index.js:13:17:
Originally at:
node_modules/telnet-client/src/index.ts:1:0: ERROR - [JSC_JS_MODULE_LOAD_WARNING] Failed to load module "events"

3 error(s), 0 warning(s)

========

% java -jar ~/workplace/closure-compiler-v20220601.jar --process_common_js_modules --module_resolution NODE '**.js' '!**/test/**'

--------

./main.js:1:19: ERROR - [JSC_JS_MODULE_LOAD_WARNING] Failed to load module "telnet-client"
  1| const { Telnet } = require('telnet-client')
                        ^

./node_modules/stream/index.js:22:14: ERROR - [JSC_JS_MODULE_LOAD_WARNING] Failed to load module "emitter"
  22| var Emitter = require('emitter');
                    ^

./node_modules/telnet-client/lib/index.js:13:17:
Originally at:
node_modules/telnet-client/src/index.ts:1:0: ERROR - [JSC_JS_MODULE_LOAD_WARNING] Failed to load module "events"

3 error(s), 0 warning(s)

========

What am I doing wrong here?  Any help would be much appreciated!

-- Ben

[Chinthoorie Yogalingam]

Sorry, I don't have an answer to your question. We are not planning to invest time on improving CommonJS support.

I found a similar issue which mentions a workaround - https://github.com/google/closure-compiler/issues/3719

Hope it helps.

[Ben Liblit]

Thank you for pointing me toward that similar issue. If I use a hand-crafted telnet-client stub instead of the real telnet-client implementation, then Closure's command-line interface can ingest and process both the main program script and the stub. However, Closure's Java API fails, and I cannot figure out how to use the API to replicate whatever the command-line interface is doing.

My main program, stored as main.js:

    const { Telnet } = require('telnet-client')

    new Telnet().connect()

My stub, stored as node_modules/telnet-client.js:

    class Telnet {
      connect() {}
    }
   
    exports.Telnet = Telnet

When I run Closure from the command line, the output is about what I would expect, with the require call rewritten to use synthesized structures that represent module exports:

    % java -jar ~/Downloads/closure-compiler-v20220301.jar --module_resolution=NODE --formatting=PRETTY_PRINT --process_common_js_modules node_modules/telnet-client.js main.js

   
    var module$node_modules$telnet_client = {default:{}};
    module$node_modules$telnet_client.default.Telnet = class {
      connect() {
      }
    };
    const {Telnet} = module$node_modules$telnet_client.default;
    (new module$node_modules$telnet_client.default.Telnet()).connect();

Encouraging! So I tried to do the equivalent thing using Closure's Java API:

    CompilerOptions compilerOptions = new CompilerOptions();
    compilerOptions.setModuleResolutionMode(ResolutionMode.NODE);
    compilerOptions.setPrettyPrint(true);
    compilerOptions.setProcessCommonJSModules(true);

    new Compiler()
        .compile(
            AbstractCommandLineRunner.getBuiltinExterns(BROWSER),
            Arrays.asList(
                SourceFile.fromFile("node_modules/telnet-client.js"),
                SourceFile.fromFile("main.js")),
            compilerOptions);

Unfortunately, this Java code fails:

    Jul 08, 2022 3:11:37 PM com.google.javascript.jscomp.LoggerErrorManager println
    SEVERE: main.js:1:19: ERROR - [JSC_JS_MODULE_LOAD_WARNING] Failed to load module "telnet-client"

      1| const { Telnet } = require('telnet-client')
                            ^
   

    Jul 08, 2022 3:11:37 PM com.google.javascript.jscomp.LoggerErrorManager printSummary
    WARNING: 1 error(s), 0 warning(s)

My ultimate goal is to have programmatic access to an AST similar to the one that the command-line invocation produces, augmented with resolved names and inferred types to the best of Closure's ability. The error above suggests to me that I will not be getting deep enough into Closure's processing layers to have that information. What should I do differently so that my API-driven processing more closely resembles what the command-line interface does?

Thanks,

Ben

[Chad Killingsworth]

With node module resolution, you also have to provide the package.json files as part of the compilation source. The package.json files are required for the algorithm to know how to resolve node modules.

--

---
You received this message because you are subscribed to the Google Groups "Closure Compiler Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to closure-compiler-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/closure-compiler-discuss/37f263d1-5ddb-4d13-a850-1cefedc85507n%40googlegroups.com.

[Ben Liblit]

OK, I discarded my hand-crafted telnet-client.js stub, and created the following package.json stored in the same directory as main.js:

    {
      "name": "experiment",

      "version": "1.0.0",
      "dependencies": {
        "telnet-client": "^2.0.3"
      }

    }

I installed the real telnet-client dependency using npm install. But now which files should I include in the inputs argument to Compiler::compile? Presumably my top-level main.js and package.json. Perhaps also all node_modules/**/*.js and node_modules/**/package.json? If I do that, then even a command-line invocation of Closure fails with eight warnings (omitted here) and the following three errors:

    % java -jar ~/Downloads/closure-compiler-v20220301.jar --process_common_js_modules --module_resolution=NODE --formatting=PRETTY_PRINT --language_in=ECMASCRIPT_NEXT --warning_level=VERBOSE main.js package.json node_modules/**/*.js node_modules/**/package.json

   

    node_modules/emitter-component/test/emitter.js:2:14: ERROR - [JSC_JS_MODULE_LOAD_WARNING] Failed to load module ".."
      2| var Emitter = require('..');
                       ^
   

    node_modules/stream/index.js:22:14: ERROR - [JSC_JS_MODULE_LOAD_WARNING] Failed to load module "emitter"
      22| var Emitter = require('emitter');
                        ^
   

    node_modules/telnet-client/lib/index.js:13:17:
    Originally at:

    node_modules/telnet-client/src/index.ts:1:0: ERROR - [JSC_JS_MODULE_LOAD_WARNING] Failed to load module "events"
   
    3 error(s), 8 warning(s)

That's a big step backward from when I only had main.js and a hand-crafted node_modules/telnet-client.js stub, which seemed to work fine via the command-line, although not via the Java API.

[Chad Killingsworth]

You would only want to include the package.json files of modules which are referenced during compilation. node_modules/**/package.json may seem like a handy shortcut, but it will undoubtedly pull in way more than you intend.

--

---
You received this message because you are subscribed to the Google Groups "Closure Compiler Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to closure-compiler-d...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/closure-compiler-discuss/e0afb96f-2d3a-4a61-b789-fb9d93990335n%40googlegroups.com.

[Ben Liblit]

Per this advice:

You would only want to include the package.json files of modules which are referenced during compilation.

Thank you for this suggestion. I pruned my command line down to just the JavaScript sources and package.json from my top-level application and the telnet-client dependency. This fails from the command line, so I haven't even tried the Java API yet:

    % java -jar ~/Downloads/closure-compiler-v20220301.jar --process_common_js_modules --module_resolution=NODE --formatting=PRETTY_PRINT --language_in=ECMASCRIPT_NEXT --warning_level=VERBOSE main.js package.json node_modules/telnet-client/**/*.js node_modules/telnet-client/**/package.json

   
    node_modules/telnet-client/lib/index.js:13:17:
    Originally at:
    node_modules/telnet-client/src/index.ts:1:0: ERROR - [JSC_JS_MODULE_LOAD_WARNING] Failed to load module "events"
   

    node_modules/telnet-client/lib/index.js:14:14:
    Originally at:
    node_modules/telnet-client/src/index.ts:2:0: ERROR - [JSC_JS_MODULE_LOAD_WARNING] Failed to load module "net"
   
    node_modules/telnet-client/lib/utils.js:4:17:
    Originally at:
    node_modules/telnet-client/src/utils.ts:1:0: ERROR - [JSC_JS_MODULE_LOAD_WARNING] Failed to load module "stream"
   
    3 error(s), 0 warning(s)

What should I try next?

[Chad Killingsworth]

Honestly you have to trace through and see how the command line runner sets options. Its been a while since I have done that. 

--

---
You received this message because you are subscribed to the Google Groups "Closure Compiler Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to closure-compiler-d...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/closure-compiler-discuss/e188b32e-185c-4ba1-b422-df5211cca170n%40googlegroups.com.