Execution behavior of define calls

37 views
Skip to first unread message

James Burke

unread,
Nov 5, 2011, 3:46:43 PM11/5/11
to amd-im...@googlegroups.com
John Hann discovered a difference in how curl and requirejs execute
define() calls, and the implications it has for libraries opting in to
calling define. This came up after using the AMD registration in
jQuery.

Specifically, jQuery calls define() in the middle of the file, before
the execution of the body of jQuery is done. In curl, it seems to call
code that has already loaded that depends on jQuery immediately when
receiving the define() call for jQuery. This can cause a problem if
that dependent code immediately uses something in jQuery that is
defined after the define call.

In requirejs, it waits for the current script to finish executing
before seeing if dependencies can be satisfied.

Should we try to standardize this execution behavior one way or another?

I like the behavior requirejs uses because it is the behavior that
must occur for anonymous modules in the browser (the name is not known
until the script finishes executing, at least in non-IE browsers), and
it also maps better to the built environment, where jquery would be
first in the built file, execute completely before the next module is
run.

I also believe it gives script authors more flexibility in adopting
optional calls to define() -- the patches to existing code can be
smaller for code with no dependencies, particularly if they also do a
commonjs/node call setup, where they may want to use exports to hang
their properties. In that case, the adapter code can happen at the top
of the file.

Example:

(function () {
var myLib = {};
if (typeof exports === 'object') {
myLib = exports;
} else if (typeof define === 'function' && define.amd) {
define(myLib);
} else {
//window globals.
this.myLib = myLib;
}

//Set up myLib here.
}.call(this));

While it is possible to move that define() call to the bottom of the
file, I believe most people would prefer to have all the module
adapter logic together. At least I would.

But it would be good to get other implementers to comment though, and
for John to expand on the execution model in curl. Rawld, what does
dojo do?

James

rbackhouse

unread,
Nov 5, 2011, 6:18:22 PM11/5/11
to amd-implement
James, can you clarify. Are talking about the point in time where the
define's factory is invoked if it has one ?

Richard

James Burke

unread,
Nov 5, 2011, 7:33:52 PM11/5/11
to amd-im...@googlegroups.com
On Sat, Nov 5, 2011 at 3:18 PM, rbackhouse <richarda...@gmail.com> wrote:
> James, can you clarify. Are talking about the point in time where the
> define's factory is invoked if it has one ?

It is a bit more subtle than that: it is when modules that depend on
the output of the factory function are called.

That means the factory function has been called, but in the case of
jQuery, if the factory function was called right away, it would not
necessarily be bad, but giving the result of the factory function to
other factories that depended on jQuery could result in an error since
the file containing jQuery is still executing -- not all of the
properties have been attached yet to the jQuery "export".

So, imagine a module called "a" that depends on jquery and calls
jQuery.extend(), and it has loaded before jQuery loads. It looks
something like:

define('a', ['jquery'], function ($) {
//part of setting up a is to use jQuery
$.extend();
});

And jQuery loads, and its content looks roughly like this:

(function () {
function jQuery() {};

define('jquery', [], function () {
return jQuery;
});

jQuery.extend = function () {};
}());


So if the loader gives jQuery to 'a' as part of jQuery's call to
define(), instead of waiting for the file containing jQuery finishes
executing, there will be an error.

James

John Hann

unread,
Nov 5, 2011, 8:07:33 PM11/5/11
to amd-im...@googlegroups.com
There's a jsFiddle of it here: http://jsfiddle.net/unscriptable/GqfFM/

-- John

unscriptable

unread,
Nov 5, 2011, 10:44:50 PM11/5/11
to amd-implement, Rawld Gill
My 2¢...

> I like the behavior requirejs uses because it is the behavior that
> must occur for anonymous modules in the browser (the name is not known
> until the script finishes executing, at least in non-IE browsers), and

Of course nobody's asking you to change the behavior of requirejs. :)
That's not the issue. The issue is how to instruct devs to code their
modules. Imho, the define() call is the signal that the module has
been defined. Therefore, there are two ways to write modules: wrapped
in a define() or trailed by a define().

It's fine if there are conditions that could delay the transfer of
dependent modules to requiring modules (e.g. unloaded, anonymous
modules in non-IE browsers). However, I don't believe we should have
to enforce delays in order to make this behavior consistent. I really
don't want to have to slow down curl.js even by a few dozen
milliseconds.

In addition, we've encountered plenty of cases in which we want to
asynchronously load additional resources (via require() or xhr).
Having these kick-off as soon as possible is also important for app
performance.

So, does requirejs also wait for the script to finish loading in IE
before signaling dependencies? In all cases (anonymous or named),
curl.js signals dependencies asap (i.e. doesn't wait for script load
event) in IE.


> it also maps better to the built environment, where jquery would be
> first in the built file, execute completely before the next module is
> run.
>

This may be how requirejs creates built files, but it's not
necessarily how all AMD build tools will (or should) work. I've got
plans for cram.js, some of which may exclude modules from being listed
in the exact dependency order.

Speaking of environments, why should we enforce a rule that is only
important to current browsers under certain conditions? In a server-
side environment, I would think it would be even more important to
eliminate any unnecessary delays.



> (function () {
>     var myLib = {};
>     if (typeof exports === 'object') {
>         myLib = exports;
>     } else if (typeof define === 'function' && define.amd) {
>         define(myLib);
>     } else {
>         //window globals.
>         this.myLib = myLib;
>     }
>
>     //Set up myLib here.
>
> }.call(this));

Ok, how about this then?:

(function (myLib) {

//Set up myLib here.

if (typeof define === 'function' && define.amd) define(myLib);
}.call(this, typeof exports == 'object' ? exports : (this.myLib =
{})));

From the dozen or so people I've chatted with about universal module
formats (I know, a dozen is not necessarily a good sampling), it's
preferable to define your module at the top of the file and put the
boilerplate at the bottom.


> But it would be good to get other implementers to comment though, and
> for John to expand on the execution model in curl. Rawld, what does
> dojo do?

There's not much to say except that curl.js does not wait for the
script to finish loading if it doesn't have to (i.e. it has the id or
can obtain it reliably (in IE)).

I'm interested to see what other people think!

-- John

On Nov 5, 3:46 pm, James Burke <jrbu...@gmail.com> wrote:

John Hann

unread,
Nov 5, 2011, 11:27:54 PM11/5/11
to amd-implement, Rawld Gill
If I am interpreting this correctly, section 3.1.10 of the CommonJS Modules/2.0 spec would imply that the module be completely defined (memoized) before being returned by the factory function:


This means that the define must either be wrapped

in a define() or trailed by a define().

-- John

rbackhouse

unread,
Nov 6, 2011, 7:21:55 AM11/6/11
to amd-implement
Ok, got it.

I tried out John's test from jsfiddle in lsjs and it loads jquery
without error.

My approach for lsjs was to ensure all dependencies have been fully
loaded before calling factories that use them. This to me seems like
the better approach to take if unexpected behavior is to be avoided
however I can see why John wants to wring out the best performance he
can with his loader.

Richard

On Nov 5, 6:33 pm, James Burke <jrbu...@gmail.com> wrote:

James Burke

unread,
Nov 7, 2011, 1:30:26 AM11/7/11
to amd-im...@googlegroups.com
On Sat, Nov 5, 2011 at 7:44 PM, unscriptable <jo...@e-numera.com> wrote:
> My 2¢...
>
>> I like the behavior requirejs uses because it is the behavior that
>> must occur for anonymous modules in the browser (the name is not known
>> until the script finishes executing, at least in non-IE browsers), and
>
> Of course nobody's asking you to change the behavior of requirejs. :)
> That's not the issue.  The issue is how to instruct devs to code their
> modules.  Imho, the define() call is the signal that the module has
> been defined.  Therefore, there are two ways to write modules: wrapped
> in a define() or trailed by a define().

Right, I was not framing this as a need to change requirejs, but more
for module users and authors, what behavior should they expect. I
mentioned how requirejs does it because it is one way, I'm not sure
what others do, and it is a consistent behavior that gives a bit more
flexibility for module authors.

The performance arguments do not resonate with me (in particular the
sync server case -- node does a few IO operations w/try/catch to load
modules, and the perf is fine for them). Builds will be more effective
overall with achieving performance, and for me consistency is a more
important design goal.

But now we're just talking about perceptions and performance, we would
need more real data.

So let's keep the arguments focused on the tension between these two
items, which are really the core of my concern:

1) What would an AMD implementer expect the behavior of a define call to be?
2) What is easy for existing library authors to implement?

1) In a "modules only" world, I believe it is reasonable to assume
that once define() has returned, the module is defined.

2) However, for existing libraries, that approach limits the options
for boilerplate. I was concerned about libraries like underscore or
backbone, whose authors are not necessarily warm on the idea of
supporting AMD.

However, I think there are workable solutions for those cases. It will
require another change to underscore to fit in with this behavior, and
I'm not too excited about pushing for that, but the change is fairly
straightforward, just move the module detection/registration block to
the end of the file since underscore wants to export a function, so it
defines it up front anyway -- it does not want to rely on a exports
value to start.

So, I'm OK with just stating that when define() returns, the module
should be considered completely constructed at that point. I can add
some wording to this effect in the AMD API page too. I want to
illustrate that API page with a few more examples too. I'll post back
to the list when I get to it, which may not be for a little while.

On a side note about umd boilerplate:

> From the dozen or so people I've chatted with about universal module
> formats (I know, a dozen is not necessarily a good sampling), it's
> preferable to define your module at the top of the file and put the
> boilerplate at the bottom.

Agreed, I like it at the bottom, but for items with dependencies, I
think it may work out better with the boilerplate at the top, much to
my disappointment. Compare this patch I was trying to do with backbone
that does the work mostly at the bottom:
https://github.com/jrburke/backbone/compare/documentcloud:master...jrburke:optamd

vs doing it at the top:
https://github.com/jrburke/backbone/compare/documentcloud:master...jrburke:optamd3

Which is a variation inspired from Kris Kowal's q library:
https://github.com/kriskowal/q/blob/master/q.js#L13

With the boilerplate at the bottom, it really requires another level
of indentation, and it seems a bit more wordy. I'm not sure if either
approach will work for Backbone yet, but doing it at the top seems
like a less invasive change for an existing library than doing the
work at the bottom.

James

John Hann

unread,
Nov 7, 2011, 9:08:12 AM11/7/11
to amd-im...@googlegroups.com
I'd really like to hear some other opinions! (hint hint)

To me, this feels like a decision between backward-compatibility and forward-compatibility.  

Excerpt from the node.js docs (while they were contemplating AMD compatibility http://nodejs.org/docs/v0.5.0/api/modules.html):
Important: Despite being called "AMD", the node module loader is in fact synchronous, and using define() does not change this fact. Node executes the callback immediately, so please plan your programs accordingly.

Hopefully, we can convince them to support the define() function again.  If we do, then they'll likely still want to execute the callback immediately.  I fear if we don't enforce third-party modules and transitional libs from completely defining their modules before calling define(), then we'll have a helluva time later when server-side environments natively support AMD.  

I'm searching for another reference, but I can't seem to find it now.  That reference talked about CommonJS or node.js modules and stated that the API of the exported module should be completely defined before being returned, but the actual implementation could be lazily loaded.  In short: always export the complete API.  (My google-fu is not strong today. :( )

Again, please chime in.  We want to hear your opinion!

-- John

Ben Hockey

unread,
Nov 7, 2011, 9:32:16 AM11/7/11
to amd-im...@googlegroups.com, jo...@e-numera.com
since you're asking for opinions...   as i see it, this discussion is a question of how to author modules.  loader implementers should be free to provide an implementation that works with what is considered a properly authored module.   the only "module is now defined" flagging mechanism that a module author can control is when the call to define happens.  i don't think that it's necessarily wrong to have a loader that is more conservative (ie uses some other flag that is after the call to define) or needs to wait a little longer to figure out which module has been defined, but module authors should write their modules in such a way that calling define is a flag to the loader that at the end of the execution of define their module is ready for consumption.

ben...

Timmy Willison

unread,
Nov 7, 2011, 9:23:11 AM11/7/11
to amd-im...@googlegroups.com
Though I am inclined to agree with James concerning waiting until scripts have been executed on the client-side (which does provide more flexibility in the browser), I also see the necessity for enforcing a fully defined export on the server-side.  I think I could be persuaded to land either way.

On a side note, jQuery will probably need to put its define at the bottom, as that seems to be the safest approach for all of the loaders.

James Burke

unread,
Nov 7, 2011, 12:56:08 PM11/7/11
to amd-im...@googlegroups.com
On Mon, Nov 7, 2011 at 6:08 AM, John Hann <jo...@e-numera.com> wrote:
> I'd really like to hear some other opinions! (hint hint)
> To me, this feels like a decision between backward-compatibility and
> forward-compatibility.
> Excerpt from the node.js docs (while they were contemplating AMD
> compatibility http://nodejs.org/docs/v0.5.0/api/modules.html):
>>
>> Important: Despite being called "AMD", the node module loader is in fact
>> synchronous, and using define() does not change this fact. Node executes the
>> callback immediately, so please plan your programs accordingly.
>
> Hopefully, we can convince them to support the define() function again.  If
> we do, then they'll likely still want to execute the callback immediately.
>  I fear if we don't enforce third-party modules and transitional libs from
> completely defining their modules before calling define(), then we'll have a
> helluva time later when server-side environments natively support AMD.

In a sync environment, the module would still run to completion before
continuing with the execution of the module that required that
dependency, the define() call was just used to register the exports.
Otherwise the call sequencing gets impossible to track.

That said, I still think you make a good point that if you just look
at define() code, anyone looking at that and trying to implement a
loader would expect "once the factory function is completed the module
is defined".

Note that this is only an issue with code that does not have any
dependencies. Once there is a dependency, this sort of gray area goes
away, the module will likely need to be completely wrapped in a define
call execution to work.

I am concerned that we make it easy for these base libraries to opt in
to calling AMD. I think that is a valid concern, but giving those
cases a *slight bit* more wiggle room in a way that muddies
understanding if/when AMD becomes standardized does not seem to be
worth the tradeoff now for me.

James

Domenic Denicola

unread,
Nov 18, 2011, 1:23:59 AM11/18/11
to amd-im...@googlegroups.com, Rawld Gill, jo...@e-numera.com
3.1.10 is actually a manifestation of the old "do not execute the factory function until require is called" debate; it is saying that an empty exports object should be memoized before even *running* the factory function, which will augment that exports object. Note that the purple-background text below it should be referring to 3.1.10 instead of 2.4.9.

James Burke

unread,
Nov 18, 2011, 1:04:52 PM11/18/11
to amd-im...@googlegroups.com

I did not quite follow this -- I do not know what those numbers
reference. Maybe you could clarify. It sounds a little bit like
CommonJS stuff.

If so, the issue in this discussion is a bit more nuanced. While the
"wait to execute factory until first require call" behavior you cite
would avoid this particular issue as a side effect, that behavior
makes the APIs more verbose, and AMD works fine without doing so.

The more direct question to ask is "should a module be considered
defined once the factory function has finished?". I think John makes a
good case that the answer should be "yes". It makes sense. It is just
a bit tricky since we are dealing with bootstrapping existing web
libraries into modular behavior, and the opt-in AMD calls brought up
the issue.

James

Domenic Denicola

unread,
Nov 18, 2011, 3:14:58 PM11/18/11
to amd-im...@googlegroups.com

> I did not quite follow this -- I do not know what those numbers
> reference. Maybe you could clarify. It sounds a little bit like
> CommonJS stuff.
>
> If so, the issue in this discussion is a bit more nuanced. While the
> "wait to execute factory until first require call" behavior you cite
> would avoid this particular issue as a side effect, that behavior
> makes the APIs more verbose, and AMD works fine without doing so.
>
> The more direct question to ask is "should a module be considered
> defined once the factory function has finished?". I think John makes a
> good case that the answer should be "yes". It makes sense. It is just
> a bit tricky since we are dealing with bootstrapping existing web
> libraries into modular behavior, and the opt-in AMD calls brought up
> the issue.
>
> James

Sorry, it looks like I should have quoted the post I was responding to. (Damn web interface!) It was unscriptable's post about section 3.1.10 of the CommonJS Modules/2.0 spec. I was trying to clarify that it wasn't a very good guide for the current situation, since that portion of that spec is talking more about the delayed factory execution behavior and not about when a module is considered defined. Which is pretty much what you're saying above :)
Reply all
Reply to author
Forward
0 new messages