Handling a define() from a script tag include

[Jakob Heuser]

This has come up a few times on the Inject side, and I wanted to ask the mailing list for guidance. The spec says this about anonymous modules:

The first argument, id, is a string literal. It specifies the id of the module being defined. This argument is optional, and if it is not present, the module id should default to the id of the module that the loader was requesting for the given response script. When present, the module id MUST be a "top-level" or absolute id (relative ids are not allowed).

Which works for environments where things have been included using an AMD loader system. What happens with a script tag?

<script src="foo.js"></script>

// foo.js

define([], {

  name: 'foo'

});

In the above, what should the "ID" be for the foo.js at define() resolution? Since it was included via a manual script tag, we've been batting around a few ideas in the Inject group:

1. Give the module an unreachable id (for example "^anon-1" which won't match any require syntax).
2. Assign the module id as "null" and resolve all relative modules against an empty string.
3. Scan for the newest script tag and infer the module based on that.
4. Encourage libraries only to set define.amd when they are in the middle of a require() call so that define() calls from within other script tags are ignored

Thoughts on the matter? Every time someone mixes AMD and Non-AMD, they seem to hit this wall.

[John Hann]

> Every time someone mixes AMD and Non-AMD, they seem to hit this wall.

+1

1. Give the module an unreachable id (for example "^anon-1" which won't match any require syntax).
2. Assign the module id as "null" and resolve all relative modules against an empty string.
3. Scan for the newest script tag and infer the module based on that.
4. Encourage libraries only to set define.amd when they are in the middle of a require() call so that define() calls from within other script tags are ignored

I like that you're trying to "do the right thing" for the newbs.  However, I'm not sure there is anything that can be done except to throw.  This is what curl.js does (although until last weekend, the error message was cryptic).

> 1. Give the module an unreachable id (for example "^anon-1" which won't match any require syntax).

This, effectively, just silently fails, no?  That seems like it's not helpful. :(

> 2. Assign the module id as "null" and resolve all relative modules against an empty string.

Unless, by accident, the relative modules are at the baseUrl, attempts to fetch them will fail.  This might be better than silently failing.  However, the error messages won't be very useful.  This seems to be an user error to me: why would the module have dependencies?  I don't know of any global+AMD third-party libs that have relative dependencies.

> 4. Encourage libraries only to set define.amd when they are in the middle of a require() call so that define() calls from within other script tags are ignored

Turning off `define.amd` sounds like an interesting way to foil a global+AMD lib, but there's no way to know when to turn it on or off.  Browsers don't tell us when they are *about to* evaluate a script.  They only tell us *after* the script has been evaluated (or *during* evaluation in IE).  

> 3. Scan for the newest script tag and infer the module based on that.

How can we know which script is the "newest"?  DOM order is not [necessarily] indicative.  Hmmm..... load events?  Yuk, load events don't bubble, so we'd have to add an event handler to every script element in the document.  Still, I guess that's possible.  But then how do we know the module id?  We'd have to infer that everything after baseUrl is an id (yuk).  Either that, or we could try to inverse-map against config.paths.

Imho, if the dev loads a file via a script element, they mean to use it outside of AMD (e.g. as a browser global).  The right solution, it follows, would be to turn off define() until all other scripts are loaded.

I tried to think of a few solutions that involve adding event handlers to the other script elements in the document.  It's a tough problem since there's no way to know (cross-browser and cross-origin) if a script element has already been loaded.

Sorry, I don't have a solution. :(  Anyone else got an idea?

-- John

[James Burke]

This is just a weakness with what a loader can detect using script
tags. I have tried a few passes at just ignoring those calls, but it
never works out -- it is possible that one of those scripts gets
called, interleaved with require calls for loader-loaded script tags.
Particularly in older IE where it can execute a few scripts before the
first onload is triggered. So requirejs throws in this case.

BTW, this is the main reason jquery calls define with a named define
-- the possibilities of jquery getting loaded out of band of an AMD
loader is so great it was the best solution to avoid errors.

The fix will be waiting for ES modules. Other fixes could be getting
way of knowing what script tag is currently executing into browsers,
but I expect that would have a similar lead time to ES modules.

James

> --
> You received this message because you are subscribed to the Google Groups
> "amd-implement" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to amd-implemen...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Jakob Heuser]

John, James, the mixed environment is a challenge here at LinkedIn, as there's just no way to flip a switch and be running 100% module system.

Trying to solve it on the loader side:

===

For Inject, we can actually enable the define.amd flag within the local execution scope due to the XHR design. This would be the most like #2 in the original proposal.

For script tag loaders, unless we pull a huge amount of lab.js' logic into everyone's code, there's no good way to manage toggling the amd flag. Maybe it's just a documentation solution then about how the define.amd flag works, and a simple "use at your own risk" pattern for people who are locked in a mixed environment:

define.disableAmd = define.amd;

delete define['amd'];

// include your script tags, then

define.amd = define.disableAmd;

delete define['disableAmd'];

Not that I want to encourage bad code, as the goal is to ultimately get to a module loader.

Trying to solve it on the AMD module developer's side:

===

Maybe we just need to update the pattern for detecting AMD and calling define, with a note specifically about try/catching if you're using an anonymous define. Since everyone throws in one form or another, the following code is a lot more bulletproof. Note the try/catch added to the anonymous define call.

if (define && define.amd) {

  try {

    define([], myExports);

  } catch (e) {

    (module && module.exports) ? module.exports = myExports : window.myName = myExports;

  }

}

else if (module && module.exports) {

  module.exports = myExports;

}

else {

  window.myName = myExports;

}

Thanks for keeping the discussion going.

--Jakob

You received this message because you are subscribed to a topic in the Google Groups "amd-implement" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/amd-implement/c9EcRZyKmvA/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to amd-implemen...@googlegroups.com.

[John Hann]

Hi Jakob,

My responses inline:

(Love this interesting discussion, btw!)  

On Wed, Mar 20, 2013 at 4:00 AM, Jakob Heuser <ja...@felocity.com> wrote:

John, James, the mixed environment is a challenge here at LinkedIn, as there's just no way to flip a switch and be running 100% module system.

Trying to solve it on the loader side:

===

For Inject, we can actually enable the define.amd flag within the local execution scope due to the XHR design. This would be the most like #2 in the original proposal.

Oohhhhh, you're using XHR+eval instead of script injection?  You've got complete control, then!  There's no limit to what you can do.

 

For script tag loaders, unless we pull a huge amount of lab.js' logic into everyone's code,

LABjs has the *exact* same restrictions that AMD loaders do (actually, it has more restrictions).  I'm not sure what you mean.  Hm... unless by "you", you mean end-users, not AMD tool authors?  But maybe this isn't important...

 

there's no good way to manage toggling the amd flag. Maybe it's just a documentation solution then about how the define.amd flag works, and a simple "use at your own risk" pattern for people who are locked in a mixed environment:

Don't forget that most AMD modules don't care about `define.amd` and not all UMD patterns do, either.  (There are plenty of module writers who don't care about legacy, non-module environments.)

So, just to clarify: it seems you want a solution for loading *well-defined, UMD-wrapped modules* to be loaded via traditional script elements in an AMD environment.  This doesn't seem like something that belongs in a spec.  It sounds like it belongs in your loader and in your "best practices" documentation.  No?

 

define.disableAmd = define.amd;

delete define['amd'];

// include your script tags, then

define.amd = define.disableAmd;

delete define['disableAmd'];

Not that I want to encourage bad code, as the goal is to ultimately get to a module loader.

Trying to solve it on the AMD module developer's side:

===

Maybe we just need to update the pattern for detecting AMD and calling define, with a note specifically about try/catching if you're using an anonymous define. Since everyone throws in one form or another, the following code is a lot more bulletproof. Note the try/catch added to the anonymous define call.

if (define && define.amd) {

  try {

    define([], myExports);

  } catch (e) {

    (module && module.exports) ? module.exports = myExports : window.myName = myExports;

  }

}

else if (module && module.exports) {

  module.exports = myExports;

}

else {

  window.myName = myExports;

}

Sorry, when I said "curl.js throws", it actually just propagates an error up the errback chain.  It doesn't throw while executing the `define()`.  There's no way to know (afaik, when using script injection) that the script wasn't fetched with a static <script> element while executing the `define()`.

[John Hann]

On Wed, Mar 20, 2013 at 8:10 AM, Brian Cray <bcra...@gmail.com> wrote:

If this was part of the original document can't you do document.getElementsByTagName('script')[document.getElementsByTagName('script').length - 1].src?

Two gotchas:

1. The script may have been loaded from another origin.  The "src" is not available for security reasons.

2. The last script in the document isn't likely the one that is executing.  The files are fetched async and could arrive in any order.

-- J

[Jakob Heuser]

Thanks John.

It sounds like the consensus is:

1. users: "Don't mix <script> and require(), it's unsupported."
   
2. library authors: "Either create a named define (like jQuery), or do NOT use an anonymous define. Declaring an anonymous define() will break users who use script loaders and would like to concatenate your file to another AMD module."
   

I would like to get some agreement on:

1. if a user does: <script>define([], function() { console.log('a'); })</script> what should happen?
2. if a user does: <script>define(['foo'], function(foo) { console.log(foo); })</script> what should happen?

Personally, I think at the time define() is invoked, if the ID cannot be determined, we should probably throw an error. The spec would read (emphasis mine): "The first argument, id, is a string literal. It specifies the id of the module being defined. This argument is optional, and if it is not present, the module id should default to the id of the module that the loader was requesting for the given response script. If no module ID can be determined (for example due to inclusion via a script tag), a loader should throw an error to the end-user. When present, the module id MUST be a "top-level" or absolute id (relative ids are not allowed)."

This would make #1 and #2 both throw errors.

Last, if nobody sees any ecosystem impact, I'd like to modify Inject to only enable define.amd during the dependency resolution chain. This would be coming in 0.4.3 since 0.4.2 is already in RC.

[John Hann]

On Wed, Mar 20, 2013 at 2:08 PM, Jakob Heuser <ja...@felocity.com> wrote:

Thanks John.

It sounds like the consensus is:

1. users: "Don't mix <script> and require(), it's unsupported."
   
2. library authors: "Either create a named define (like jQuery), or do NOT use an anonymous define. 
   

Not trying to be contentious, but I didn't get the impression that we all agreed to discourage library authors from using anonymous modules.  o_O  

The alternative to anonymous modules is explicitly configured paths (e.g. `paths: { jquery: "libs/jquery-1.8.2/jquery.min" }`).  There are plenty of users and authors who hate having to configure their loaders.  This would make it worse.  (To be fair, you can also place modules at the baseUrl and rename them to match their id.)

To be forthright, I have been reconsidering the benefits and consequences of anonymous modules lately and may be in favor of recommending named modules for third-party libs as you suggest.  (So far, it seems ES6 will require ids on *all* modules, as well.)  Would you like to open a new thread to open the discussion about it?

 

Declaring an anonymous define() will break users who use script loaders and would like to concatenate your file to another AMD module."

Sorry, I am confused.  Isn't this what tools like what r.js and cram.js do?  They concatenate the modules and inject the ids.  Maybe I am missing some context.  Can you explain your thinking?

 

I would like to get some agreement on:

1. if a user does: <script>define([], function() { console.log('a'); })</script> what should happen?
2. if a user does: <script>define(['foo'], function(foo) { console.log(foo); })</script> what should happen?

Personally, I think at the time define() is invoked, if the ID cannot be determined, we should probably throw an error. The spec would read (emphasis mine): "The first argument, id, is a string literal. It specifies the id of the module being defined. This argument is optional, and if it is not present, the module id should default to the id of the module that the loader was requesting for the given response script. If no module ID can be determined (for example due to inclusion via a script tag), a loader should throw an error to the end-user. When present, the module id MUST be a "top-level" or absolute id (relative ids are not allowed)."

I want to +1 this wholeheartedly, but I can't.  When using script injection, afaict there's no way the ID can be determined at the time the define() is called (except in legacy IE and the user is not using multiple loading contexts).  

Coincidentally, I have also been re-evaluating my thinking on script injection versus XHR+eval.  In this case, XHR+eval seems to have a clear advantage.  

-- John

[Jakob Heuser]

Hah, contentious is good (and I think I jumped the gun). We only seemed to have consensus around informing the end user and saying "don't mix AMD and <script> tags."

Is it possible to have an AMD "global" namespace, perhaps prefixed with a colon? For example: define(':jquery', [], jQuery); It wouldn't change for anyone who's already using named modules, but going forward, we could endorse a named module system for library developers. Alternatively, we could caution library developers about defining anonymous modules in their code and how concat can cause them grief. Tools like r.js can actually wrap your code safely if you're using module/module.exports so perhaps people offer an AMD and non-AMD version.

I'm out of brainstorming there. On to topic 2!

I don't know how we address this. Inject throws an unrelated error, require uses onError to throw a mismatch, and curl bubbles their error up as well. I guess all I'd like here is consistency. As a library developer, if I had a reliable way to know define() failed, they could route their public interface to the proper destination. Sadly, I tried to counter John's statement in the bowels of LAB.js, $script, etc, but there isn't a consistent way.

[James Burke]

Responding to a few posts at once:

* The recommendation should be "if you are using script tags, specify
them before the script tag for the AMD loader". This avoids the issues
around anon defines, since define() will not exist when those script
tags run.

* For XHR+eval loaders, I can also see where you can just hide the
global define() instead of just define.amd, and only include it in the
evaled script:

var define = loader.define; /*script here*/ define = null;

* XHR+eval will not work in CSP-enabled environments that disallow
eval, like Chrome packaged apps, and certain types of privileged apps
for Firefox OS. This continues a trend of JS environments becoming
more hostile to eval over time.

* anon defines are the way to go. I do not want to encourage named
define calls as it leads to config messes. John, not sure where you
got the "ES is going with named defines", but that will not be true
for single modules in a file. They do have a named syntax, `module
"some/id" {}` but that is for the cases where AMD has named defines
now -- when inlining modules. FWIW, ES modules do not have these
problems, since the ES loader does the loading, and has full control
of the fetching/execution (more similar to XHR+eval capabilities),
with nothing like a "global define() that may be called anonymously".

* Trying to get a try/catch around a define() will not work for the
reasons John mentioned: the association of an ID with an anonymous
define happens at a later point for script tag-based loaders.

James

[John Hann]

Hey James,

All of the recent notes I've seen fly by in the TC39 meetings seem to stipulate that ES6 modules *must* be named.  Have you heard differently?  Recently?

`module "some/id" {}` <-- known good

`module {}` <-- I have not seen this evar :)

It's true that eval() is in hostile territory.  I agree.  However, it'd be possible to allow script injection as a fallback (with known caveats and limitations).  Also, there's no way eval() will be removed from the broader language until quasi-literals are broadly available. (IMHO, YMMV :) )

-- John

[James Burke]

On Wed, Mar 20, 2013 at 3:09 PM, John Hann <jo...@unscriptable.com> wrote:
> All of the recent notes I've seen fly by in the TC39 meetings seem to
> stipulate that ES6 modules *must* be named. Have you heard differently?
> Recently?
>
> `module "some/id" {}` <-- known good
> `module {}` <-- I have not seen this evar :)

It would just be the module body, as I understand it. So, named:

module "some/id" {
import { something } from "dep";
}

"anonymous" version can only be in a file and does not have a wrapper
(like a node module):

import { something } from "dep";

> It's true that eval() is in hostile territory. I agree. However, it'd be
> possible to allow script injection as a fallback (with known caveats and
> limitations). Also, there's no way eval() will be removed from the broader
> language until quasi-literals are broadly available. (IMHO, YMMV :) )

With the right CSP settings, eval is effectively removed, throws an
error if it is used. There probably are good use cases for XHR+eval
loaders, but it requires the end user to know the sharp edges and to
take mitigations for those cases, like perhaps always do builds, use a
modified loader.

I commented just to point out more environments that are eval hostile,
but did not mean to dissuade. FWIW I also maintain an XHR+eval capable
loader here:
https://github.com/requirejs/cajon

James

[John Hann]

Thanks, James.  I didn't realize that ES6 modules could be "naked" like CommonJS modules.

I'm going to think more about XHR+eval.  I understand there are sharp edges, but there are sharp edges with script injection, too.

-- J

[Jakob Heuser]

Thanks James and John,

At this point, I think the safest thing to do is add a page to Inject's HowTo guides (we've started a collection for common problems) on "Mixing Loaders With <script> Tags". The only 100% safe way to manage this is to place scripts above or below your loader, depending on your desired effect.

If placed before your loader, many libraries will simply write to the global window object. A solution needed in very complex sites that can't covert to AMD systems straight away.

If placed after your loader, some libraries such as jQuery may function, but it is safer to include these dependencies in your define() and require() calls instead of trusting the library to auto-register to a namespace.

~

I still have some concerns about libraries like fastClick https://github.com/ftlabs/fastclick/blob/master/lib/fastclick.js This library cannot be concatenated since it forces an anonymous define. Do tools such as r.js fix these anonymous defines and map them to a proper module path?

--Jakob

[John Hann]

I still have some concerns about libraries like fastClick https://github.com/ftlabs/fastclick/blob/master/lib/fastclick.js This library cannot be concatenated since it forces an anonymous define. Do tools such as r.js fix these anonymous defines and map them to a proper module path?

--Jakob

That is correct.  They inject the module's normalized id. :)

-- J