Import node modules with closure compiler

[Paul]

Hi all,

I'm at the end of my wits as to how to get a node module to import properly and be compiled into my JS.

I'm running closure compiler with gulp with process_common_js_modules: true, and module_resolution: 'NODE',

I've tried simply using an import ' ' or var svg4everybody = require('svg4everybody')

I've tried adding this to the externs.js file too var svg4everybody = require('svg4everybody');

But every method I've tried has always resulted in this error:

gulp-google-closure-compiler: source/js/externals.js:8: ERROR - [JSC_JS_MODULE_LOAD_WARNING] Failed to load module "svg4everybody"

And I can't find any up to date resources online on how to import node modules using the closure compiler. Any help would be appreciated.

[Nick Reid]

Are the files comprising that node module being passed to the compiler? I'm not familiar with external toolchains, but I do know that the compiler doesn't look up files it isn't explicitly told about.

The implementation of the resolver is here: https://github.com/google/closure-compiler/blob/e82319ba321587474261ebc4cd05a6087b2fa1d7/src/com/google/javascript/jscomp/deps/NodeModuleResolver.java#L192 It can be a bit tough to reason through, but it may give you some insight.

[Paul]

Thanks for replying and for that link, I have a better idea of how closure expects things even though it's not ideal for a front end workflow.

I've tried a few examples I've found around the web and specifying the file I want exactly but I couldn't get it to work. I've tried adding the specific js file as a source and as an external and I still cannot get a node module in.

Using https://www.npmjs.com/package/hello-world-npm as an example of a node module because it is very basic and standard so it shouldn't be anything Closure doesn't like. I've tried having it in source, as an external, importing it from my external.js too and in my source code. I've also tried different methods of importing it but I always get the same Failed to load module error. I've also tried loading its package.json as I read somewhere you can do that but in every package.json I get [JSC_PARSE_ERROR] Parse error. Semi-colon expected  "name": "hello-world-npm", 

Let me know what other information could be useful, I've searched the issue queue and exhausted google search results when it comes to this issue. It also makes it tough that some examples are old and things have changed since then.

[Nick Reid]

Do you have the exact commandline invocation that's being used to run the compiler?

[Paul]

Currently I'm not using a command line setup, instead I'm using Gulp 4 in Node 12, the docs for it are here 

In my gulpfile, in addition to other plugins I am using closureCompiler = require('google-closure-compiler').gulp()

Then in my gulp task I have something that looks like this:

//////////////////

gulp

.src(['source/components/**/*.js', '!source/components/**/&*.js', '!source/components/**/_*.js'])

.pipe(concat('script.js'))

.pipe(closureCompiler({

  compilation_level: 'WHITESPACE_ONLY',

  module_resolution: 'NODE',

  language_out: 'ECMASCRIPT5',

  output_wrapper: '(function(){\n%output%\n}).call(this)',

  js_output_file: 'script.min.js',

  externs: [

  'source/js/externals.js',

  ],

  js_module_root: '*./',

  process_common_js_modules: true,

}))

.pipe(gulp.dest())

//////////////////

The task above gets all the js files found in source/components and concats them into one, then run CC on it. It all works fine except for trying to add in node modules.

I'll try running it via command-line later today but I don't expect any different results.

[Paul]

Ok by using CLI, hopefully this should make it simpler without a full stack of compiling to worry about.

So as per the example, using npx https://github.com/google/closure-compiler-npm/tree/master/packages/google-closure-compiler and reading about some flags here https://github.com/google/closure-compiler/wiki/Flags-and-Options

My source.js file looks like

//////////////

console.log('source.js')

var helloWorld = require('hello-world-npm');

console.log(helloWorld());

/////////////////////////

And currently the output looks exactly the same.

Running this command: lando npx google-closure-compiler --js=source.js --js_output_file=out.js --warning_level=VERBOSE --module_resolution=NODE --process_common_js_modules

Or variations of it with different flags

I get this error:

/////////////////////////

source.js:3: ERROR - [JSC_JS_MODULE_LOAD_WARNING] Failed to load module "hello-world-npm"

var helloWorld = require('hello-world-npm');

////////////////////////

It's a bit tough to write code here without formatting for it, so I hope this makes sense.

[Paul]

Oops just realised, I left in "lando" in my npx command, that's just to run it inside the node image. Can't edit my previous message.

[Wills Bithrey]

I think you need to pass the path to the package.json for `hello-world-npm` and it's source .js files as extra `--js` arguments to the call to closure compiler. AFAIK Closure compiler doesn't magically traverse the filesystem to try and resolve/find files which are `required()` - it only knows about the files passed in to it via the `--js` flag.

--

Wills Bithrey p.ota.to

--

---
You received this message because you are subscribed to the Google Groups "Closure Compiler Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to closure-compiler-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/closure-compiler-discuss/5f694893-4eab-49d6-a3b6-caa1ca4ef01dn%40googlegroups.com.

[Paul]

Ok, I see where I went wrong.

Running:

npx google-closure-compiler --js=source.js node_modules/hello-world-npm/lib/index.js node_modules/hello-world-npm/package.json --js_output_file=out.js --warning_level=VERBOSE --module_resolution=NODE --externs=externs.js --process_common_js_modules

Has a correct working output .

Running it in gulp however will not work the same, as you'll need to deal with files differently if they're json files and I think that's where my mistake was.

I will need to use something else for including node_modules as this approach isn't terribly user friendly.

Thanks all, I will write back here once I have an example solution for anyone landing here from google and I'll try to open a PR to improve the docs around this and make the information more clear with some examples.

[Chad Killingsworth]

I use gulp and include the package.json files as part of my build. I don't have any issues whatsoever. What's your concern here?

[Paul]

Well my first concern is that the lack of node_module traversal is quite a big downside, however I know it's probably not an easy solution and there are other tools that can assist with this, like Rollup or Browserify.

My second concern is that it can be a bit confusing from the documentation if the closure compiler is actually able to automatically resolve imports. As it's a very advanced compiler I initially assumed that it could and it took me a while to figure out what it can and cannot do. So in some places of the documentation relating to the NPM module at least, I was going to open an issue with a PR to mention this and explicitly make it clear that the compiler doesn't do this and suggest other tools that can fulfil that functionality which I think is expected.

It could also be my documentation reading skills being bad, which has happened before :)

[Nick Reid]

It probably is our fault. The truth is that any kind of gulp/grunt/webpack workflow isn't a first-class usecase inside Google. Any support for them is a community contribution and doesn't really have resources directed towards it unless it's a glaring failure.

That's not to say we're happy about this situation, but it's where we are.

[Chad Killingsworth]

You have a couple of options.

1. Use my new utility https://github.com/chadkillingsworth/closure-calculate-chunks. It will provide the source files in order that can be handed directly to the gulp src function. You would do this as a pre-build step and this is exactly how I use it.

2. Over specify the files in the gulp src glob and specifically include the package.json files. If you utilize the dependency management flags of the compiler, src files not referenced will be completed dropped from the compilation. This works better than you would expect.

Hope that helps.

[Paul]

Thanks all for your contribution, this thread can be locked now. I've opened an issue here to update the documentation

To anybody landing here from google, the short of it is that the compiler will not automatically resolve node_modules but it can do that if you pass the module JSON files as its source.

Option 1 example:

npx google-closure-compiler --js=source.js node_modules/hello-world-npm/lib/index.js node_modules/hello-world-npm/package.json --js_output_file=out.js --warning_level=VERBOSE --module_resolution=NODE --externs=externs.js --process_common_js_modules

Option 2:

As Chad mentioned above, you can try using his utility https://github.com/chadkillingsworth/closure-calculate-chunks

Option 3:

You use another tool that can automatically resolve node modules such as Rollup or Browserify.

I've had good success with rollup in gulp, I'll try to add example code in the issue linked above.