goog.require an ES6 module

[Marc Fallows]

Is there support for goog.require-ing an ES6 module? 

My understanding is that ES6 exports are using goog.module/goog.provide under the hood, so it would suggest that we can goog.require an ES6 module, but I am not having any luck figuring that out (or finding any documentation to that effect). Can anyone provide some insight on the status of this?

Essentially I have a set of existing ES6 modules with imports/exports, and I want to use goog.require to pull them in. It would not be ideal to have to change all of the imports/exports to goog.require and goog.module exports.

[Chad Killingsworth]

With an ES6 module, you can just mix the module systems. Just make sure you specify the --language_in=ECMASCRIPT6 flag:

goog.provide('foo.bar.baz');

goog.require('foo.foobar');
import foobaz from'/path/to/foobaz';

I'm working on an example project that shows how to use dependency pruning and use all 3 major module systems (CommonJS, ES6, goog.module) together in a single compilation. It's possible - I now have code doing just that.

Chad

[Martin Probst]

There is some undocumented, "secret" mangling of the file name going on to convert ES6 modules to goog.modules. As the naming scheme is subject to change, you cannot really depend on that.

+Tyler Breisacher what's the plan here?

Martin

--

---
You received this message because you are subscribed to the Google Groups "Closure Compiler Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to closure-compiler-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/closure-compiler-discuss/a2b4ddf7-efa3-4349-b666-eec8bc72c0e2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tyler Breisacher]

No plan currently. I'm (still) not prioritizing ES6 modules until they start to be implemented in browsers.

The "secret" is pretty straightforward (IIRC, use the filename of the module, replacing "/" with "$" and adding the word "module" at the beginning) so it would be easy to depend on it. But it is indeed subject to change. In fact, for some paths in the google repo it produces really long names which is kind of a pain for debugging, so we'll likely try to make it shorter. So any solution that depends on those names *will* probably break in the future.

[Chad Killingsworth]

Why wouldn't just use the ES6 import? Trying to use goog.require to import an ES6 module seems odd and unnecessary.

Chad

To unsubscribe from this group and stop receiving emails from it, send an email to closure-compiler-discuss+unsub...@googlegroups.com.

[Tyler Breisacher]

Sure, that would be ideal. Adding an `import` to a file automatically makes it a module, so any global variables declared there would become module-scoped variables. I guess that would be the reason not to want to use `import` but in most cases it should be fairly easy (if a bit tedious) to do.

To unsubscribe from this group and stop receiving emails from it, send an email to closure-compiler-d...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/closure-compiler-discuss/a2b4ddf7-efa3-4349-b666-eec8bc72c0e2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "Closure Compiler Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to closure-compiler-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/closure-compiler-discuss/cd7eb518-d91d-4734-8644-90be4f70f90e%40googlegroups.com.

[Chad Killingsworth]

Adding an `import` to a file automatically makes it a module

We really need to fix that. I realize we need to support the per-file transpilation case, but module rewriting should occur because a file is imported - not because it contains an import or export statement.

Chad 

To unsubscribe from this group and stop receiving emails from it, send an email to closure-compiler-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/closure-compiler-discuss/a2b4ddf7-efa3-4349-b666-eec8bc72c0e2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "Closure Compiler Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to closure-compiler-discuss+unsub...@googlegroups.com.

[Tyler Breisacher]

By my reading of the grammar in the spec, any file that contains an import statement *is* a module. I'm not sure what's supposed to happen if you attempt to load it as a script instead but it looks like it would be a syntax error.

To unsubscribe from this group and stop receiving emails from it, send an email to closure-compiler-d...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/closure-compiler-discuss/a2b4ddf7-efa3-4349-b666-eec8bc72c0e2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "Closure Compiler Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to closure-compiler-d...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/closure-compiler-discuss/cd7eb518-d91d-4734-8644-90be4f70f90e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "Closure Compiler Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to closure-compiler-d...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/closure-compiler-discuss/98df78cc-1290-4ff8-b9af-da4a60ad2e1a%40googlegroups.com.

[Marc Fallows]

This is exactly why I'm avoiding `import`. I have a module-less ES5 codebase (relying on global namespaces) that would hopefully use some ES6 modules.

A `goog.requireFrom` (or similar) that follows a similar signature to `import` would certainly be handy (or some way to import without ending up as a module - but I read the spec the same way and it wouldn't make sense to have `import` in a file that isn't a module.

To unsubscribe from this group and stop receiving emails from it, send an email to closure-compiler-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/closure-compiler-discuss/a2b4ddf7-efa3-4349-b666-eec8bc72c0e2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "Closure Compiler Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to closure-compiler-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/closure-compiler-discuss/cd7eb518-d91d-4734-8644-90be4f70f90e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "Closure Compiler Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to closure-compiler-discuss+unsub...@googlegroups.com.

[Ben Lickly]

Keep an eye on the upcoming JavaScript Loader standard:
https://github.com/whatwg/loader/
That's the one that's supposed to contain System.import or equivalent.

>>>>>>>> send an email to closure-compiler-d...@googlegroups.com.

>>>>>>>> To view this discussion on the web visit
>>>>>>>> https://groups.google.com/d/msgid/closure-compiler-discuss/a2b4ddf7-efa3-4349-b666-eec8bc72c0e2%40googlegroups.com.
>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>
>>>>>>
>>>>> --
>>>>>
>>>>> ---
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Closure Compiler Discuss" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send

>>>>> an email to closure-compiler-d...@googlegroups.com.

>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/closure-compiler-discuss/cd7eb518-d91d-4734-8644-90be4f70f90e%40googlegroups.com.
>>>>>
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>>
>>> --
>>>
>>> ---
>>> You received this message because you are subscribed to the Google Groups
>>> "Closure Compiler Discuss" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an

>>> email to closure-compiler-d...@googlegroups.com.

>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/closure-compiler-discuss/98df78cc-1290-4ff8-b9af-da4a60ad2e1a%40googlegroups.com.
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>
>>

> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "Closure Compiler Discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to closure-compiler-d...@googlegroups.com.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/closure-compiler-discuss/e5cdedd4-db29-4f0e-b780-2d509c7dcc3b%40googlegroups.com.

[Marc Fallows]

Chad, how are you doing this? I was under the impression you couldn't have `goog.provide` in a module (as per https://github.com/google/closure-compiler/issues/880). Any time I've tried to have `goog.provide` and `import` in the same module, I get an error that the variable I am importing is `undeclared`.

[Chad Killingsworth]

Most of my code is ES6 based which also has 'goog.require' statements. However, it's a problem if you can't use import statements and goog.provide together.

There are multiple discussions happening right now regarding how to detect what constitutes a "module". I'm going to bring this up at today's meeting.

Chad

[Chad Killingsworth]

I looked at your example here: https://github.com/igrep/keep-me-contributing/commit/1e3100c697f8b14590c656c6e4e3ac2de5656ce9

I don't think you should be able to mix goog.provide and export statements. goog.provide and import statements should work though.

Chad

[Marc Fallows]

The ES6 module spec gets a bit in the way in terms of module detection. But having `goog.provide` and `import` work together would be a step forward in allowing you to mix module types (and work with them in a non-module system).

Being able to have a `goog.import` or `goog.requireFrom` that works like `import` (but doesn't treat that script into a module) would also be very beneficial in my case. The ES6 modules are already in `goog.module` form, there's just no easy, documented way to get a hold of the modules except through `import` (which has the side effect described above of then becoming a module).

[John]

You can't use "import" from scripts (only modules), it is a syntax error.  goog.provide is distinctly for scripts.   To load a ES6 module from a script you must use the (unimplemented) System object.   With goog.provide'd files you must be careful not to pollute the global namespace, you should use goog.module instead to avoid that.  It would look something like this:

goog.module('foo.bar.baz');

// One of the loader commands return a promise, you don't want that,

// instead you a "get this or fail" so that the file reference the dependency

// as in static references.

var  foobaz = System.loader.get('/path/to/foobaz');  

But that doesn't help you with dependency management, unless you are looking at System object calls in addition to goog.require's.   If you are going to depend on an ES6 module, I suggest you use an ES6 module so the dependency is natural.

[Steve S]

Hi Chad,

I saw you mentioning an example project, and also this wiki page:

https://github.com/google/closure-compiler/wiki/Working-With-Multiple-Module-Systems

I'd be very curious to see this example project. Is there a link?

I'm using a mix of legacy goog.provide and fresh goog.module, and Closure Library, but want to move my own goog.module files to ES6 or CommonJS for better IDE support and easier migration of code between build systems. Last time I tried (~7 months ago?) by using options.setProcessCommonJSModules(true) in combination with options.setModuleRoots(), it didn't work, possibly because ClosureLibrary has a bunch of files with multiple provides.

Steve

[Chad Killingsworth]

There are 2 pull requests which will fully enable this functionality:

• https://github.com/google/closure-compiler/pull/1348
  
• https://github.com/google/closure-compiler/pull/1329
  

Once these are merged (hoping for this week), all the dependency systems and modules can inter-operate. I have projects using a custom build right now that utilize all three.

One note: if you use the ES6 "import" or "export" statement, or have a CommonJS "require" or "module.exports" statement, module rewriting WILL occur in the file. That's not been a problem for me yet, but has been a roadblock for others. I'd like to remove the CommonJS "require" statement from that list, but that takes more work in the compiler still.

Chad Killingsworth

[Steve S]

Thanks Chad. Looking forward to playing with these changes!

Missing from your wiki page are details on mixing goog.provide/module/require and es6/commonjs. My compilation comprises:

• Closure Library (goog.provide/require, including some files with multiple provide's)
  

• My legacy code:
  
  goog.provide("my.ns.SomeLegacyScript");
  
  goog.require("my.ns.SomeOtherLegacyScript"); // same as per Closure Library
  
  goog.require("my.ns.SomeNewBridgeModuleWithLegacyNamespace"); // I definitely use this in a few places
  
  const SomeNewModule = goog.require("my.ns.SomeNewModule"); // I think I might also use this in a couple of cases
  
  my.ns.SomeOtherLegacyScript.someFunc();
  my.ns.SomeNewBridgeModuleWithLegacyNamespace.someFunc();
  SomeNewModule.someFunc();
  

• My new "bridge" code accessed from legacy code
  
  goog.module("my.ns.SomeNewBridgeModuleWithLegacyNamespace");
  goog.module.declareLegacyNamespace();
  
  const array = goog.require("goog.Array");
  

  const SomeOtherLegacyScript = goog.require("my.ns.SomeOtherLegacyScript");
  const SomeNewModule = goog.require("my.ns.SomeNewModule");
  
  array.find(...);
  SomeOtherLegacyScript.someFunc();
  SomeNewModule.someFunc();

• Entirely new modules which are only used by other modules
  
  goog.module("my.ns.SomeNewModule");
  
  const array = goog.require("goog.Array");
  

  const SomeOtherLegacyScript = goog.require("my.ns.SomeOtherLegacyScript");
  const SomeOtherNewModule = goog.require("my.ns.SomeOtherNewModule ");
  const YetAnotherNewModule = goog.module.get("my.ns.YetAnotherNewModule"); // e.g. avoiding circular references
  
  const React = require("React"); // in-browser runtime loading of React as a CommonJS module, from a separate JS script (I never got this to work with React, but do use it with EventEmitter2)
  
  array.find(...);
  SomeOtherLegacyScript.someFunc();
  SomeOtherNewModule.someFunc();
  YetAnotherNewModule.someFunc();
  
  
  
  

and I want to incrementally move towards either ES6 or CommonJS, perhaps ES6 in preference. Are any of these scenarios incompatible with ES6/CommonJS modules?

(If you like, perhaps I can update your Wiki page to have all from/to permutations for module type combinations with code examples a little like the above, and explanations where known for whether it is valid or not?)

As soon as I use "import" or "export" it does module rewriting - what is this, exactly? Does the compiler modify the AST on the fly prior to regular compilation passes, to turn ES6 and CommonJS modules into goog.module using $ converted absolute paths? 

What does that mean if you are also using goog.module, goog.require and goog.module.get() in the same file? And can a legacy file goog.require that ES6/CommonJS module using my$ns$... notation?

Presumably module rewriting doesn't happen in whitespace-only mode because it isn't "lower from ES6". I have a PR open related to this (wrapping modules in whitespace mode) and presumably could do the same and enable module rewriting when using ES6 language out via a CompilerOptions property.

I know ES6 modules aren't a priority until browsers support them. But when they do, presumably you're not going to want to serve up every module as a separate HTTP request, so only-concatenated (whitespace only) development would still need module rewriting?

(We use whitespace mode extensively for rapid development. With our own Java wrapper classes which add "watch" and caching of filesystem data, a 30+ second advanced compilation takes <500ms in whitespace only + module wrapping mode, warm run).

[Chad Killingsworth]

My wiki article is incomplete, because that's all that currently works. The PRs I mentioned fix pretty much everything you mentioned - definitely multiple provides in a single file. I'll update the wiki page with full examples once they actually work. I'm using a custom build of the compiler on my projects right now, so I have real world application already.

I have switched to authoring code as ES6 modules, but I will be using external libraries that are Closure and CommonJS based for a long time. Everything you mentioned will work when these PRs land. One caveat - you are using "require('lib')" to dynamically load an external library at runtime. If you enable the "--process_commonjs_modules" flag, that file would be included at compilation time (or broken if the source wasn't included). I'd like to have a way to designate that case, but for now you can alias "require" to work around the issue. "var externalRequire = require; externalRequire('lib');"

On Module Rewriting

Closure compiler normalizes the disparate module/dependency systems. Internally they all become goog.require/provide. However since CommonJS and ES6 modules do not execute in the global scope, part of the normalization process rewrites/namespaces all global references inside any of the module files.

I believe this normalization DOES occur in WHITESPACE_ONLY mode - but that may be triggered by a language out that is lower than ES6. I've got an issue out to rename that to NO_OPTIMIZATIONS, but I'd like to get new documentation for the compiler project before that change. A primary purpose for WHITESPACE_ONLY mode has become transpilation.

I'm using the grunt/gulp plugins I just released for rapid development with differing levels of compilation tied to a watch task. Your use case is far from unique.

Another fix which you didn't mention is that all of this needs to be supported when the "--module" flags of the compiler are used. I'm utilizing those flags to compile unit tests along with main source, but separate the output.

Chad Killingsworth

[Steve S]

Chad

Thanks for the tip about externalRequire, I'd already found the page where you first suggested that; works well.

Regarding renaming to NO_OPTIMIZATIONS, we use it for rapid serving of minimally-changed scripts to modern Chrome which supports all the ES6 features we actually use. We don't want transpilation in this case. So just like with my whitespace wrapping goog.modules PR, we need that mode still to be able to do normalisation even when no checks and no transpilation is happening..

Currently name normalisation (conversion from URI to goog-provide equivalent name) uses ES6ModuleLoader.toModuleName() which replaces slashes with $ and other punctuation with _. I read somewhere - perhaps this discussion - that it would not be a good idea to depend on this behaviour. Is this still the case after your PR? Or is this irrelevant after your PR?  So can a legacy goog.provided file use goog.require("some$normalised$Name") to include a non-legacy file, or should it use require / import while continuing to goog.require old files?

Steve

[Steve S]

... and will you share sources to that example project I think you mentioned that demonstrates mixing module systems?

[Chad Killingsworth]

On Friday, January 8, 2016 at 8:04:13 AM UTC-6, Steve S wrote:

... and will you share sources to that example project I think you mentioned that demonstrates mixing module systems?

It doesn't exist yet. I have plans to create a reference project for this and many other reasons, but I haven't done it yet. The code I have that uses it isn't public (nor can I make it such).

Chad Killingsworth