factory-on-define vs factory-on-require

[Patrick Mueller]

I've been having a conversation with Dojo developers over on

    http://mail.dojotoolkit.org/pipermail/dojo-contributors/2012-April/027333.html

    

regarding using Dojo (and other AMD impls, really) with Apache Cordova.

I'd like to move at least part of the discussion here, because I think it's AMD-general as opposed to Dojo-specific.

Little background: we are currently using an AMD-lite kind of story in Cordova, where we define primitive define() and require() functions used in our runtime, really just for our own modules.  We author our modules is CJS style - there's no define() wrapper in our modules, instead, at build-time, we gen our final script that defines all our modules:

    { require/define logic }

    define('a', function(require,exports,module)) {...}

    define('b', function(require,exports,module)) {...}

    ...

    require(theMainModule)

    

All of our require() invocations are of the 1-arg module id form.  All of our define() invocations are of the form specified above.  We have no module pre-req information explicitly available, in require() invocations, in define() invocations, nor in the order in which the define()s are run in our script.

One thing we were hoping to do is to allow folks to replace that {require/define logic} bit at the top of our genned script, with an AMD implementation.  

We thought we were playing by the rules, but I suspect we're not.  At this point I've observed two AMD implementations which execute the factory function during the define() invocation, at least for the way that we use the define() function.  I'll call this factory-on-define invocation.

That only works when all the modules are define()d in pre-req order.

We'd like to not have to specify the order in which we do defines.  Not clear that we might have circular define()s anyway.

Another way of looking at this is to say that I want factory-on-require invocation.  I do NOT want the factory run on define(), but on the first require() of that module.

Am I right in thinking that AMD is factory-on-define, at least in the case of no pre-reqs, or perhaps even the case of 'has pre-reqs, but they've already been loaded'?  If so, I think that should be spelled out in the AMD spec - here, I guess: https://github.com/amdjs/amdjs-api/wiki/AMD .  Maybe it's there and I missed it?

Moving along, I assume there are already folks dependent on factory-on-define invocation.  So, that behavior can't be changed.  So, I'd like a way to be able to tell AMD that I want a module's factory run in factory-on-require mode instead.  Eg

    define.lazyFactory(moduleId, factoryFunction)

    

or

  

    define.lazyFactory(moduleId)

    define(moduleId, factoryFunction)

or

    factoryFunction.lazy = true

    define(moduleId, factoryFunction)

or whatever.

--
Patrick Mueller
http://muellerware.org

[Taka Kojima]

RequireJS is factory-on-require and I believe the majority of other AMD loaders are as well.

That only works when all the modules are define()d in pre-req order.

NeedsJS (https://github.com/gigafied/NeedsJS) is actually factory-on-define, but this does not mean that define() calls have to be made in the right order. In fact, it really does not matter, if a define() needs other modules (even circular) it requests those when the define happens and defers the define until all recursive dependencies are resolved.

I've been able to swap out NeedsJS and RequireJS at will (their API is the same, though NeedsJS doesn't supoort everything RequireJS does). The internal are very different, but they essentially "seem" to behave the same way, even though they don't.

The main difference is that if I manually load `moduleA.js`, all recursive dependencies are loaded along with it. The factory function is then invoked once all dependencies have been loaded. At this point if you require('moduleA'), the callback function is invoked immediately, as the dependencies/factory already exists.

If you do the same thing using RequireJS (manually loading `moduleA.js`) it loads moduleA.js, but none of it's dependencies. If I were to then require('moduleA'), it would load the dependencies, then call the callback function after those are loaded and the factory has been executed.

With all that being said, I plan on moving NeedsJS over to factory-on-require, as there are some pros I overlooked by doing it that way.

I am pretty sure factory-on-require is the most common method or "standard", I don't know that it will make it into the spec though.

- Taka

[James Burke]

On Mon, Apr 30, 2012 at 12:25 PM, Patrick Mueller <pmu...@gmail.com> wrote:

> Am I right in thinking that AMD is factory-on-define, at least in the case
> of no pre-reqs, or perhaps even the case of 'has pre-reqs, but they've
> already been loaded'?  If so, I think that should be spelled out in the AMD
> spec - here, I guess: https://github.com/amdjs/amdjs-api/wiki/AMD .  Maybe
> it's there and I missed it?

It has not been specified, but seems like it is time to do so. Thanks
for taking the time to describe the issue.

As Taka mentions, it depends on the loader. requirejs does
"define-on-require" or more precisely, "on end of dependency load"
which can be at a script.onload event. But it is closer to
define-on-require then otherwise, and will be even more so with
requirejs 2.0.

I originally thought for builds that it is best to order dependencies
and then have the define calls all execute in order when a require()
call was hit for any modules that were defined, as that maps closest
to how regular "just use globals" scripts work in the browser if they
are concatenated together (that case *does* need precise ordering).

However, based on your feedback with almond and introducing an
'unordered' option for almond, and based on this ticket for requirejs:

https://github.com/jrburke/requirejs/issues/183

I now believe that "define-on-require" is a better route to go since it will:

* keep the execution order similar to the "non-built" case, where
defines come in as they are required (the issue in that requirejs
ticket)
* For requirejs 2.0, I want to allow someone to bundle in some
define() calls without having them execute to save on execution
startup time. I believe your feedback related to how costly some
modules in phonegap might be to auto-execute convinced me of this
benefit.

So for requirejs 2.0 I'm going to go strictly define-on-require for
AMD modules, and I plan to update almond to only do so, which means
that the "unordered" flag will no longer be necessary.

End result: I suggest we just agree on "define-on-require" as the
suggested execution behavior and update the spec page with that
language.

Any concerns?

James

[Patrick Mueller]

On Mon, Apr 30, 2012 at 16:39, James Burke <jrb...@gmail.com> wrote:

On Mon, Apr 30, 2012 at 12:25 PM, Patrick Mueller <pmu...@gmail.com> wrote:

> Am I right in thinking that AMD is factory-on-define, at least in the case
> of no pre-reqs, or perhaps even the case of 'has pre-reqs, but they've
> already been loaded'?  If so, I think that should be spelled out in the AMD
> spec - here, I guess: https://github.com/amdjs/amdjs-api/wiki/AMD .  Maybe
> it's there and I missed it?

It has not been specified, but seems like it is time to do so. Thanks
for taking the time to describe the issue.

As Taka mentions, it depends on the loader. requirejs does

"define-on-require" ...

Just to make sure we're on the same page, I used 'factory-on-require' which I assume you are calling 'define-on-require'.  By 'factory-on-require' I mean "a module's factory function is only ever executed in response to a require() invocation for that module, not by a define() call of that module".

Are we talking about the same thing?  I hope so.  :-)

The only problem I see with making this 'default' is that it breaks implementations which were dependent on this behavior. I assume there exists code which is dependent on this behavior.

Maybe this issue will also force a definition of what a 'global require' is :-) (if not already defined - I don't track AMD development that closely).

 

[Taka Kojima]

The only problem I see with making this 'default' is that it breaks implementations which were dependent on this behavior. I assume there exists code which is dependent on this behavior.

I don't think any "good" code would depend on this behavior, in fact it kinda scares me to think about what kind of code would break should an AMD loader switch from factory-on-define to factory-on-require.

AMD modules should only care about these things:

1) Dependencies

2) Export/Factory

3) [Optionally] Own Module ID

Modules should only care that their dependencies are defined/exist when the factory function executes. Modules should not care about how/when they are loaded, as long as they are there when the factory function is invoked.

Nothing in the factory function will execute until dependencies are resolved in either scenario (factory-on-define or factory-on-require). Thus I can't see how/why it would break any pre-existing code to make the switch from factory-on-define to factory-on-require.

-Taka

[James Burke]

On Mon, Apr 30, 2012 at 2:07 PM, Patrick Mueller <pmu...@gmail.com> wrote:
> On Mon, Apr 30, 2012 at 16:39, James Burke <jrb...@gmail.com> wrote:
> Just to make sure we're on the same page, I used 'factory-on-require' which
> I assume you are calling 'define-on-require'.  By 'factory-on-require' I
> mean "a module's factory function is only ever executed in response to a
> require() invocation for that module, not by a define() call of that
> module".
>
> Are we talking about the same thing?  I hope so.  :-)

Ah yes, sorry, the voices in my head did a swap of words, I meant
"factory-on-require" as you outline above.

> The only problem I see with making this 'default' is that it breaks
> implementations which were dependent on this behavior. I assume there exists
> code which is dependent on this behavior.

My first thought is that it should not cause major problems, at least
for the dynamic loaders, since they need to do "on-demand" calls
anyway.

I would expect to be more of an issue for specialty shims for builds,
like almond. However, those are normally very specific and can be tied
to the build tool that generates the built file.

One subtlety I still want to work out for almond builds:
"factory-on-define" is nice because it does not require a "require()"
call at the end of the file to trigger loads. However,
"factory-on-require" means that I probably need to change the build
output. Details:

In a build with almond, it is common to modify the script that that
was for require.js to just be for the built file that includes almond.
So this tag:

<script src="require.js" data-main="main"></script>

gets replaced with:

<script src="main-built.js"></script>

main.js in source form can just be a module:

define(function(require){

});

The data-main attribute is the implicit require() call that kicks off
loading. However, after the almond build, there is effectively no
top-level require call. How will module resolution occur?

My plan is for the optimizer to insert a require(['main'], function()
{}) call at the end of the built almond script, but still need to work
out the mechanics for that. It may mean indicating this require is
desired via a build property.

I'm OK doing this for almond since it is still at a 0.0.x release, but
others should speak up if going with "factory-on-require" causes a
problem for them.

James

[Richard Backhouse]

Both AMD loaders I have written use 'factory-on-require'.

I have always assumed that developers will have to trigger the initial
load by calling the entry-point into the loader, whether that is via a
global async "require" call or via an implementation specific name.

Richard

[Ben Hockey]

On Monday, April 30, 2012 5:07:43 PM UTC-4, Patrick Mueller wrote:

Just to make sure we're on the same page, I used 'factory-on-require' which I assume you are calling 'define-on-require'.  By 'factory-on-require' I mean "a module's factory function is only ever executed in response to a require() invocation for that module, not by a define() call of that module".

Are we talking about the same thing?  I hope so.  :-)

to the best of my knowledge, dojo is 'factory-on-require' - see http://jsfiddle.net/neonstalwart/RAXkb/ (snippet follows)  

    define("some/dependency", function (require, exports, module) {
        alert("some/dependency factory is being run");        
    }); 
    
    define("some/module", function (require, exports, module) {
        alert("some/module factory is being run");
        var dep = require("some/dependency");
    });   
    
    alert("about to require...");
    require(["some/module"], function () {
        alert("some/module should be loaded");
    });

you should see alerts as follows:

• "about to require..."
  
• "some/dependency factory is being run"
  
• "some/module factory is being run"
  
• "some/module should be loaded"
  

(for everyone, not just patrick) what should be different?

ben...

[Domenic Denicola]

I’d like to get some clarification. Is anyone considering and/or implementing the following order?

 

·         "about to require..."

·         "some/module factory is being run"

·         "some/dependency factory is being run"

·         "some/module should be loaded"

 

This seems more in line with “factory-on-require,” as opposed to, say, “factory-upon-being-depended-on.”

 

Cf. James’s earlier reply in a related thread. I just want to make sure all involved are aware of the subtle difference he describes there. It seems there are at least three alternatives, even though only two names are being used.

[Patrick Mueller]

On Mon, Apr 30, 2012 at 22:34, Ben Hockey <neonst...@gmail.com> wrote:

to the best of my knowledge, dojo is 'factory-on-require' - see http://jsfiddle.net/neonstalwart/RAXkb/ (snippet follows)  

I believe I'm seeing a problem just with Dojo at the moment, and so following up there.

I will note, in general, that the example is more interesting if define("some/dependency") comes AFTER define("some/module").  Because defining the modules in "loadable order" - like Ben's original example - will actually work for some factory-on-define AMDs like almond.  If you define them "out of order", almond will fail (without the define.unordered = true switch (or whatever)).

[Rawld]

I'm not sure which two you are referring, but I suspect you think dojo
is doing this which is not the case--so long as you're in a pure AMD mode.

One of the painful/powerful things about the dojo loader is that it
supports so many modes in order to take care of our customer's
backcompat needs. All of this can be built out and the loader is very
small when so built. In unbuilt, source form you can get the loader into
pure AMD mode like this:

<script src="path/to/dojo.js"
data-dojo-config="has:{'dojo-sync-loader':0}"></script>

The current dojo loader in trunk does immediately demand (download) all
dependencies when a module is defined (i.e., when the define application
is applied...which is different from when the factory is applied). I
made an experimental dojo loader that delays demanding dependencies
until the depending module is demanded by require (directly or
indirectly). This is experiment is available at
http://rcgill.dojotoolkit.org/dojo.js

>
> That only works when all the modules are define()d in pre-req order.
>
> We'd like to not have to specify the order in which we do defines.
> Not clear that we might have circular define()s anyway.
>
> Another way of looking at this is to say that I want
> factory-on-require invocation. I do NOT want the factory run on
> define(), but on the first require() of that module.

> Am I right in thinking that AMD is factory-on-define, at least in the
> case of no pre-reqs, or perhaps even the case of 'has pre-reqs, but
> they've already been loaded'? If so, I think that should be spelled
> out in the AMD spec - here, I guess:
> https://github.com/amdjs/amdjs-api/wiki/AMD . Maybe it's there and I
> missed it?
>

To be clear: proper AMD does run factory on define(). I'm pretty sure
most, if not all, of the other loaders agree with this.

Lastly, I would *strongly* encourage you to reconsider the way you've
decided to express your modules. The "pure AMD way"....using no absolute
module id's and one module def per file will make your code much more
portable and optimizable, and should control complexity.

--Rawld

[Rawld]

On 04/30/2012 01:39 PM, James Burke wrote:
> On Mon, Apr 30, 2012 at 12:25 PM, Patrick Mueller<pmu...@gmail.com> wrote:
>> Am I right in thinking that AMD is factory-on-define, at least in the case
>> of no pre-reqs, or perhaps even the case of 'has pre-reqs, but they've
>> already been loaded'? If so, I think that should be spelled out in the AMD
>> spec - here, I guess: https://github.com/amdjs/amdjs-api/wiki/AMD . Maybe
>> it's there and I missed it?
> It has not been specified, but seems like it is time to do so. Thanks
> for taking the time to describe the issue.
>
> As Taka mentions, it depends on the loader. requirejs does
> "define-on-require" or more precisely, "on end of dependency load"
> which can be at a script.onload event. But it is closer to
> define-on-require then otherwise, and will be even more so with
> requirejs 2.0.
>
> I originally thought for builds that it is best to order dependencies
> and then have the define calls all execute in order when a require()
> call was hit for any modules that were defined, as that maps closest
> to how regular "just use globals" scripts work in the browser if they
> are concatenated together (that case *does* need precise ordering).

At some point we probably need to talk about how things should be built.
I have a strong opinion that built code should work just like unbuilt
code (and a design that supports that idea). But that's another topic.

> However, based on your feedback with almond and introducing an
> 'unordered' option for almond, and based on this ticket for requirejs:
>
> https://github.com/jrburke/requirejs/issues/183
>
> I now believe that "define-on-require" is a better route to go since it will:
>
> * keep the execution order similar to the "non-built" case, where
> defines come in as they are required (the issue in that requirejs
> ticket)
> * For requirejs 2.0, I want to allow someone to bundle in some
> define() calls without having them execute to save on execution
> startup time. I believe your feedback related to how costly some
> modules in phonegap might be to auto-execute convinced me of this
> benefit.
>
> So for requirejs 2.0 I'm going to go strictly define-on-require for
> AMD modules, and I plan to update almond to only do so, which means
> that the "unordered" flag will no longer be necessary.
>
> End result: I suggest we just agree on "define-on-require" as the
> suggested execution behavior and update the spec page with that
> language.

I agree...particularly since dojo/backdraft has always been this way in
pure AMD mode :)

--Rawld

[Rawld]

On 04/30/2012 02:28 PM, Taka Kojima wrote:

The only problem I see with making this 'default' is that it breaks implementations which were dependent on this behavior. I assume there exists code which is dependent on this behavior.

I don't think any "good" code would depend on this behavior, in fact it kinda scares me to think about what kind of code would break should an AMD loader switch from factory-on-define to factory-on-require.

AMD modules should only care about these things:

1) Dependencies

2) Export/Factory

3) [Optionally] Own Module ID

Modules should only care that their dependencies are defined/exist when the factory function executes. Modules should not care about how/when they are loaded, as long as they are there when the factory function is invoked.

Nothing in the factory function will execute until dependencies are resolved in either scenario (factory-on-define or factory-on-require). Thus I can't see how/why it would break any pre-existing code to make the switch from factory-on-define to factory-on-require.

-

I agree 100%. And it's quite important we agree on this "theory of operation". Also, it shouldn't matter whether the code is built/unbuilt...this theory should still apply.

--Rawld

[Rawld]

On 04/30/2012 08:09 PM, Patrick Mueller wrote:

On Mon, Apr 30, 2012 at 22:34, Ben Hockey <neonst...@gmail.com> wrote:

to the best of my knowledge, dojo is 'factory-on-require' - see http://jsfiddle.net/neonstalwart/RAXkb/ (snippet follows)  

I believe I'm seeing a problem just with Dojo at the moment, and so following up there.

Just to keep the thread history clear...

The issue you are seeing with dojo is that, assuming you are in pure AMD mode, dojo is *demanding the dependencies* given by a define() application. It is *not* running the factory. But since the code example causes the script injection of the dependencies of the first define to fail, the load fails (as of 1200 PST 1 MAY 2012).

There is an experimental version of  the dojo loader that waits demand dependencies until the depending module is demanded by require at http://rcgill.dojotoolkit.org/dojo.js

Let me know how that works.

--Rawld

[Patrick Mueller]

On Tue, May 1, 2012 at 14:52, Rawld <rg...@altoviso.com> wrote:

To be clear: proper AMD does run factory on define(). I'm pretty sure most, if not all, of the other loaders agree with this.

But in a later post to this thread you respond to a post from James ...

So, I'm confused.  Are you in the factory-on-define camp or in the factory-on-require camp?

I'm also confused by some of your terminology - what do the following mean:

- proper AMD

- pure AMD

You've also use "legacy" before, but that just may be in reference to Dojo.  If it's applicable to AMD itself, please define.

Are these in the spec?

Lastly, I would *strongly* encourage you to reconsider the way you've decided to express your modules. The "pure AMD way"....using no absolute module id's and one module def per file will make your code much more portable and optimizable, and should control complexity.

I'm following the AMD spec here, as near as I can tell - point out to me where I'm not.  If I am following the spec, I don't see how my code could not be portable.

-- 

Patrick Mueller
http://muellerware.org

[Rawld]

On 05/01/2012 01:29 PM, Patrick Mueller wrote:

On Tue, May 1, 2012 at 14:52, Rawld <rg...@altoviso.com> wrote:

To be clear: proper AMD does run factory on define(). I'm pretty sure most, if not all, of the other loaders agree with this.

But in a later post to this thread you respond to a post from James ...

Ugh...did i really write that. SORRY to everybody!!! What I meant to say was factory on require.

On Tue, May 1, 2012 at 14:57, Rawld <rg...@altoviso.com> wrote:

On 04/30/2012 01:39 PM, James Burke wrote:

End result: I suggest we just agree on "define-on-require" as the

suggested execution behavior and update the spec page with that
language.

I agree...particularly since dojo/backdraft has always been this way in pure AMD mode :)

So, I'm confused.  Are you in the factory-on-define camp or in the factory-on-require camp?

I'm also confused by some of your terminology - what do the following mean:

- proper AMD

- pure AMD

I use "pure" to indicate the dojo loader mode that follows the AMD spec.

I use proper when implementers and users implement loaders and modules without touching the not-fully-defined (e.g., a factory scan is not "REQUIRED") part of the spec. I agree that's a bit of a subjective term.

You've also use "legacy" before, but that just may be in reference to Dojo.  If it's applicable to AMD itself, please define.

Legacy only has to do with dojo.

Are these in the spec?

Lastly, I would *strongly* encourage you to reconsider the way you've decided to express your modules. The "pure AMD way"....using no absolute module id's and one module def per file will make your code much more portable and optimizable, and should control complexity.

I'm following the AMD spec here, as near as I can tell - point out to me where I'm not.  If I am following the spec, I don't see how my code could not be portable.

The spec allows specifying an absolute module id. But this is strongly discouraged. And once you eliminate the absolute module id, you must have a 1-to-1 arrangement of modules to resources. Lastly, depending on the factory scan to determine dependencies is not a required feature of AMD...it's a "MAY" clause. So I would recommend specifying each modules dependency explicitly.

My sincere apologies again for the confusion above.

--Rawld

[Patrick Mueller]

On Tue, May 1, 2012 at 6:53 PM, Rawld <rg...@altoviso.com> wrote:

On 05/01/2012 01:29 PM, Patrick Mueller wrote:

I'm following the AMD spec here, as near as I can tell - point out to me where I'm not.  If I am following the spec, I don't see how my code could not be portable.

The spec allows specifying an absolute module id. But this is strongly discouraged.

This is odd wording, and should be removed from the spec. Or, usage of absolute module ids in define() should  removed from the spec.  One or the other.

 

 And once you eliminate 

the absolute module id, you must have a 1-to-1 arrangement of modules to resources.

Our "big wad of define() invocations" is built from modules where each module is in a separate file.  But our "big wad of define() invocations" file - the built script - clearly has multiple modules in one file.  I assume other people do "builds" this way as well, right?  And those define() invocations presumably have absolute module ids in them.

So, still not clear on what I'm doing wrong.

 

Lastly, depending on 

the factory scan to determine dependencies is not a required feature of AMD...it's a "MAY" clause. So I would recommend specifying each modules dependency explicitly.

Right.  I don't need or want module scanning.  All scanning is going to do for us is eat CPU cycles.  Maybe come up with a false positive.  Nothing but problems.  Don't want it.

Since this is mentioned in the spec, there should probably be some way of providing a hint to the runtime that "I don't want no stinkin' scanning".  Although this will quickly devolve into wanting to be able to set it for individual modules.  

 

[Daniel Dotsenko]

Splitting hairs here a bit, as I see a significant meaningful difference between:

'factory-on-require' and 'define-on-require'

For defines like:

     define(

          'id'

          , ['a','b','c'] // <- this is the important part

          , factory

     )

My understanding of James's original 'define-on-require' is:

- 'a','b','c' are *resolved*(which may include eval) only if/when 'id' is require()'d somewhere. 

- then factory is called.

'factory-on-require' can be interpreted as:

- 'a','b','c' are *resolved*(which may include eval) either at the point of define call or at the point of require, depending on loader's author's desire

- factory is called at the point of require.

(As Rawld indicated below, Dojo had it as 'dependents-resolve-on-define+factory-on-require' )

The difference becomes important while simulating a "shim" functionality with plain AMD API:

    define(

        'underscore'

        , ['js!path/to/underscore.js']

        , function(){return _}

    )

    define(

        'backbone'

        , ['js!path/to/backbone.js']

        , function(){return Backbone}

    )

    require(['underscore'],function(_){

        // intentionally nesting to insure bb's eval

        // happens after underscore's eval is done

        require(['backbone'],function(bb){

            // do something with bb, _

        })

    })

In case of 'dependents-resolve-on-define+factory-on-require' interpretation, the above can break as path/to/underscore.js and path/to/backbone.js are no longer guaranteed to be evaled in order they are are "consumed" by require tree.

The above is a contrived example, but a plausible one. 

It is possible to clarify if the consensus reached on this subject was:

'dependents-resolve-on-require+factory-on-require' or

'dependents-resolve-whenever+factory-on-require' or

'dependents-resolve-on-require+factory-on-require'

Thanks.

Daniel

On Monday, April 30, 2012 2:28:20 PM UTC-7, James Burke wrote:

On Mon, Apr 30, 2012 at 2:07 PM, Patrick Mueller <pmu...@gmail.com> wrote:
> On Mon, Apr 30, 2012 at 16:39, James Burke <jrb...@gmail.com> wrote:
> Just to make sure we're on the same page, I used 'factory-on-require' which
> I assume you are calling 'define-on-require'.  By 'factory-on-require' I
> mean "a module's factory function is only ever executed in response to a
> require() invocation for that module, not by a define() call of that
> module".
>
> Are we talking about the same thing?  I hope so.  :-)

Ah yes, sorry, the voices in my head did a swap of words, I meant
"factory-on-require" as you outline above.

James

[Rawld Gill]

Inline

> -----Original Message-----
> From: amd-im...@googlegroups.com [mailto:amd-
> impl...@googlegroups.com] On Behalf Of Daniel Dotsenko
> Sent: Thursday, January 10, 2013 7:12 PM
> To: amd-im...@googlegroups.com
> Subject: Re: [amd-implement] factory-on-define vs factory-on-require
>

> Splitting hairs here a bit, as I see a significant meaningful difference between:
>
> 'factory-on-require' and 'define-on-require'
>
> For defines like:
>
> define(
> 'id'
> , ['a','b','c'] // <- this is the important part
> , factory
> )
>
> My understanding of James's original 'define-on-require' is:
> - 'a','b','c' are *resolved*(which may include eval) only if/when 'id' is
> require()'d somewhere.

As long as we're being precise...'a','b','c' could also be resolved if they were dependencies in some other require graph.

> - then factory is called.
>
> 'factory-on-require' can be interpreted as:
> - 'a','b','c' are *resolved*(which may include eval) either at the point of
> define call or at the point of require, depending on loader's author's desire
> - factory is called at the point of require.

Disagree. In the above example, the loader author may not decide to resolve 'a','b','c' solely consequent to seeing them in a define dependency vector.

>
> (As Rawld indicated below, Dojo had it as 'dependents-resolve-on-
> define+factory-on-require' )

So far as a pure AMD loader is concerned, if I said that, I did not intend to. But, I also confess I don't use this lingo, so I may have said something incorrect.

Now, if we're talking about running dojo's loader in legacy mode, that's a whole 'nother ball of worms...which we should take to another place to discuss because it will only confuse things here.

>
> The difference becomes important while simulating a "shim" functionality
> with plain AMD API:
>
> define(
> 'underscore'
> , ['js!path/to/underscore.js']
> , function(){return _}
> )
>
> define(
> 'backbone'
> , ['js!path/to/backbone.js']
> , function(){return Backbone}
> )
>
> require(['underscore'],function(_){
> // intentionally nesting to insure bb's eval
> // happens after underscore's eval is done
> require(['backbone'],function(bb){
> // do something with bb, _
> })
> })
>
> In case of 'dependents-resolve-on-define+factory-on-require' interpretation,
> the above can break as path/to/underscore.js and path/to/backbone.js are
> no longer guaranteed to be evaled in order they are are "consumed" by
> require tree.

If an AMD loader breaks as you indicate possible, then I think that loader has a bug.

>
> The above is a contrived example, but a plausible one.
>
> It is possible to clarify if the consensus reached on this subject was:
>
> 'dependents-resolve-on-require+factory-on-require' or
>
> 'dependents-resolve-whenever+factory-on-require' or
>
> 'dependents-resolve-on-require+factory-on-require'

This language is not illuminating to me. Here are the rules as I understand them:

define() never *directly* causes anything to be resolved.
require() causes the implied dependency graph to be resolved and then the factory executed (if any).

>
>
> Thanks.
>
> Daniel
>
> On Monday, April 30, 2012 2:28:20 PM UTC-7, James Burke wrote:
>
> On Mon, Apr 30, 2012 at 2:07 PM, Patrick Mueller
> <pmu...@gmail.com <javascript:> > wrote:
> > On Mon, Apr 30, 2012 at 16:39, James Burke <jrb...@gmail.com
> <javascript:> > wrote:
> > Just to make sure we're on the same page, I used 'factory-on-
> require' which
> > I assume you are calling 'define-on-require'. By 'factory-on-require'
> I
> > mean "a module's factory function is only ever executed in
> response to a
> > require() invocation for that module, not by a define() call of that
> > module".
> >
> > Are we talking about the same thing? I hope so. :-)
>
> Ah yes, sorry, the voices in my head did a swap of words, I meant
> "factory-on-require" as you outline above.
>
> James
>

Best,
Rawld

[Ben Hockey]

hi daniel,

i'm sure you're a smart guy so i hope you aren't offended by some honest criticism.  i'm thinking your example is flawed. i'll try to explain below but before i start i want to define 2 terms i'll use that seem less clumsy to me than terms like 'dependents-resolve-on-require+factory-on-require'

pre-emptive loading: dependencies are resolved as soon as define is called.  factory is run when another module has a dependency on the define()d module and the dependencies have been fully resolved.

lazy-loading: dependencies are only resolved when another module has a dependency on the define()d module and then the factory is run.

The difference becomes important while simulating a "shim" functionality with plain AMD API:

    define(

        'underscore'

        , ['js!path/to/underscore.js']

        , function(){return _}

    )

    define(

        'backbone'

        , ['js!path/to/backbone.js']

        , function(){return Backbone}

    )

    require(['underscore'],function(_){

        // intentionally nesting to insure bb's eval

        // happens after underscore's eval is done

        require(['backbone'],function(bb){

            // do something with bb, _

        })

    })

i think you may have made some assumptions here and without all the information there are 2 perspectives to consider:

1. someone who is not at all familiar with backbone and underscore and just sees those as 2 names without any implied relationship
2. someone who is familiar with backbone and underscore and realizes that backbone needs underscore in order to function properly

both types of perspectives are probably represented in this group.

from the first perspective, there is no error in the code regardless of when you resolve dependencies.  whether you use pre-emptive loading or lazy-loading, you won't see an error with 2 independent modules loading at different times.

form the 2nd perspective, backbone depends on underscore and so you have an error in your code - "underscore" should be added as a dependency of "backbone".  your example shows that this error only becomes evident if a loader pre-emptively loads dependencies because the 'js!path/to/backbone.js' dependency of backbone may resolve before the 'js!path/to/underscore.js' dependency of underscore and so the backbone library will not work because it relies on underscore.  the solution is to add "underscore" as a dependency for "backbone".

once the example is fixed, then it doesn't really help shed any light on which way to split the hairs.

thanks,

ben...

[Daniel Dotsenko]

Thank you Ben, Rawld.

You both have provided the right lingo I can now use. Rawld also had provided the actual answer to my question.

I don't have any issues with the above contrived example on CurlJS. My inquiry was only to hear from other AMD-style loader authors about the details of the timing of events, so I can in-line things in most fool-proof way.

My interest was in the established timing of resolution of dependencies stated in a define call:

Rawld calls "implied dependency graph to be resolved"

and Ben calls "lazy loading"

I saw that James mentioned that earlier versions of RequireJS and Almond had what Ben calls "pre-emptive loading" but committed to rewrite that to "lazy loading" (resolving define's dependency graph at the point of require)

I saw Rawld say this:

The current dojo loader in trunk does immediately demand (download) all 
dependencies when a module is defined (i.e., when the define application 
is applied...which is different from when the factory is applied). I 

made an experimental dojo loader that delays demanding dependencies 
until the depending module is demanded by require (directly or 
indirectly)

https://groups.google.com/d/msg/amd-implement/ot46tE-N6L0/z7FPAXQ_FJgJ


I strongly favor "lazy loading" (Ben's term). Although Prune.js forms the module graph and inlines the modules per the topological sort order, and should be able to handle define call dependency pre-ordering, I am still generally concerned about niche cases. Hence this email.

If there are no major AMD loaders left out there that do "pre-emptive loading" (of dependencies declared in a define call) I am very happy. If there are some, could you please, chime in?

Thank you.

Daniel.

[James Burke]

On Fri, Jan 11, 2013 at 10:36 AM, Daniel Dotsenko <dvdot...@gmail.com> wrote:
> I saw that James mentioned that earlier versions of RequireJS and Almond had
> what Ben calls "pre-emptive loading" but committed to rewrite that to "lazy
> loading" (resolving define's dependency graph at the point of require)

Just responding to confirm that requirejs 2.0+ does "lazy loading":
https://github.com/jrburke/requirejs/wiki/Upgrading-to-RequireJS-2.0#wiki-delayed

James

[Rawld Gill]

inline

> -----Original Message-----
> From: amd-im...@googlegroups.com [mailto:amd-
> impl...@googlegroups.com] On Behalf Of Daniel Dotsenko
> Sent: Friday, January 11, 2013 10:36 AM
> To: amd-im...@googlegroups.com
> Subject: Re: [amd-implement] factory-on-define vs factory-on-require
>

> Thank you Ben, Rawld.
>
> You both have provided the right lingo I can now use. Rawld also had
> provided the actual answer to my question.
>
> I don't have any issues with the above contrived example on CurlJS. My
> inquiry was only to hear from other AMD-style loader authors about the
> details of the timing of events, so I can in-line things in most fool-proof way.
>
> My interest was in the established timing of resolution of dependencies
> stated in a define call:
> Rawld calls "implied dependency graph to be resolved"
>
> and Ben calls "lazy loading"
>
> I saw that James mentioned that earlier versions of RequireJS and Almond
> had what Ben calls "pre-emptive loading" but committed to rewrite that to
> "lazy loading" (resolving define's dependency graph at the point of require)
>
> I saw Rawld say this:
> The current dojo loader in trunk does immediately demand (download) all
> dependencies when a module is defined (i.e., when the define application is
> applied...which is different from when the factory is applied). I made an
> experimental dojo loader that delays demanding dependencies until the
> depending module is demanded by require (directly or
> indirectly)

And I still stand by that: download a module !== evaluate a module. In Dojo/bdLoad, when a define application is processed, all dependencies are script-injected (a script element is attached that points to the module resource). That *does not* cause the module to be evaluated.

The problem is we've never decided on which words to use to describe the stages of an AMD module's lifetime. This vocabulary is important to this discussion because the words loaded, defined, and evaluated are ambiguous in that they can be with reference to the module resource or the module value--two completely different things. I've always had documentation in the Dojo/backdraft loader to describe this sequence (https://github.com/dojo/dojo/blob/master/dojo.js#L394). Here is the slightly edited text of that commentary:

1. Requested: some other module's definition or a require application contained the requested module in its dependency vector or executing code explicitly demands a module via require().
2. Injected: A module resource was demanded by attaching a script element to the document that points to the module's resource URL.
3. Loaded: the module resource injected in [2] has been evaluated (that is, the JavaScript code contained in the resource that resides at the injected URL was evaluated).
4. Defined: the resource contained a define statement that advised the loader about the module.
5. Evaluated: A module's value is memoized by the loader by recursively evaluating all the module's dependencies (if any) and applying the modules "factory" (if the "factory" is not a function, than the value of the factory is memorized). This can happen only after Step 4 and only consequent to fulfilling a require() application.

Dojo/bdLoad preemptively load modules the moment they are seen in a dependency vector.
Dojo/bdLoad lazy evaluates modules, the trigger event to evaluation being a require().

I've made experimental lazy loaders. I think they are only needed to make improperly-expressed AMD modules work, have a potentially high perf cost, and a non-zero implementation cost.

>
> https://groups.google.com/d/msg/amd-implement/ot46tE-
> N6L0/z7FPAXQ_FJgJ

> 1. someone who is not at all familiar with backbone and

> underscore and just sees those as 2 names without any implied relationship

> 2. someone who is familiar with backbone and underscore and

> realizes that backbone needs underscore in order to function properly
>
> both types of perspectives are probably represented in this group.
>
>
> from the first perspective, there is no error in the code regardless of
> when you resolve dependencies. whether you use pre-emptive loading or
> lazy-loading, you won't see an error with 2 independent modules loading at
> different times.
>
>
> form the 2nd perspective, backbone depends on underscore and so
> you have an error in your code - "underscore" should be added as a
> dependency of "backbone". your example shows that this error only
> becomes evident if a loader pre-emptively loads dependencies because the
> 'js!path/to/backbone.js' dependency of backbone may resolve before the
> 'js!path/to/underscore.js' dependency of underscore and so the backbone
> library will not work because it relies on underscore. the solution is to add
> "underscore" as a dependency for "backbone".
>
> once the example is fixed, then it doesn't really help shed any light
> on which way to split the hairs.
>
> thanks,
>
> ben...

--Rawld

[Rawld Gill]

James, after reading the link, it looks like you are "lazy evaluating" and don't mention when a script is injected (lazy or preemptive). Take a look at my proposed vocabulary in my previous message.

--Rawld

> -----Original Message-----
> From: amd-im...@googlegroups.com [mailto:amd-
> impl...@googlegroups.com] On Behalf Of James Burke
> Sent: Friday, January 11, 2013 10:57 AM
> To: amd-im...@googlegroups.com
> Subject: Re: [amd-implement] factory-on-define vs factory-on-require
>

[Rawld Gill]

Once clarification:

"I've made experimental lazy loaders."

More precisely:

"I've made experimental lazy loading loaders."

All my loaders have always been lazy evaluating loaders (assuming async amd).

--Rawld

[Daniel Dotsenko]

Thank you Rawld. Great point about the difference between "pre-emptive loading" and "lazy eval"

Indeed, it's the "eval" definition and timing of a given define call contents eval that I am concerned about.

You example and terminology is exactly perfect for exemplifying my worry.

First, if you don't mind, could you say if my understanding of your (below-re-quoted) point 5 is correct?

You said:

5. Evaluated: A module's value is memoized by the loader by recursively evaluating all the module's dependencies (if any) and applying the modules "factory" (if the "factory" is not a function, than the value of the factory is memorized). This can happen only after Step 4 and only consequent to fulfilling a require() application.

Dojo/bdLoad preemptively load modules the moment they are seen in a dependency vector. 
Dojo/bdLoad lazy evaluates modules, the trigger event to evaluation being a require(). 

Am I correct in understanding that in Dojo in the following example, the body of the plain JavaScript file could be evaluated before any of the require() is called?

    define(

        'jqueryplugin'

        , ['js!path/to/plainjsfile.js']

        , function(){return window.jQuery}

    )

    require(['jquery'], function(){

        require(['jqueryplugin'], function($){

            // do something with augmented jQuery

        }

    })

Daniel.

[Rawld Gill]

> -----Original Message-----
> From: Daniel Dotsenko

[snip]

>
> Am I correct in understanding that in Dojo in the following example, the body
> of the plain JavaScript file could be evaluated before any of the require() is
> called?
>
> define(
>
> 'jqueryplugin'
> , ['js!path/to/plainjsfile.js']
> , function(){return window.jQuery}
> )
>
> require(['jquery'], function(){
> require(['jqueryplugin'], function($){
> // do something with augmented jQuery
> }
> })
>

I'm not completely sure of your definition of "evaluated before any of the require() is called". So let me just talk through the code path for this resource:

1. The resource is evaluated by the browser, at the browser's chosen time (but not when another script is executing a code path {except for some odd IE exceptions}) after the browser has successfully downloaded the resource via http.

1.1 define() is applied which causes the loader to...

1.1.1 Demand that the js! plugin provide the value of the plugin resource id "path/to/plainjsfile.js"
if the js pluging is already evaluated, then
1.1.1.a pass path/to/planjsfile.js to that plugin and let the plugin do whatever it needs to do in order to provide a value for this plugin resource back to the loader. This report back could be synchronous or asynchronous...the loader does not know or care.
Otherwise
1.1.1.b inject (Step 2) the js plugin, evaluate (Step 5) it as soon as it is defined (Step 4). Upon completion of this likely-asynchronous sequence, go to 1.1.1a.

1.2 require() is applied, which causes the loader to...

1.2.1 If the jquery module is *not* already evaluated, then
inject (Step 2) the jquery module, evaluate (Step 5) it as soon as it and all of its dependencies (if any) are defined (at least) (Step 4). Upon completion of this likely-asynchronous sequence, go to 1.2.2
otherwise
1.2.2 Execute the factory function
1.2.2.1 require() is applied
If the js! plugin has reported the value of path/to/plainjsfile.js back to the loader then
1.2.2.1.1 run the factory given by the define consumed in 1.1 (if that wasn't already done consequent to anther require somewhere).
1.2.2.1.2 run the factory given by the require consumed in 1.2.2.1
Otherwise, when, at some time in the future, the js! pluging reports the value of path/to/plainjsfile.js back to the loader, execute 1.2.2.1.1 and 1.2.2.1.2.

--Rawld

[Daniel Dotsenko]

Thank you again, Rawld

Your point 1.1.1.a had solidified my understanding that Dojo (if designed like you described, "Dependency tree Pre-fetch" + "Lazy factory eval") may actually cause some portion of code linked as dependencies to defines to "run" before the module declared by that define is used in require.

This saddens me a bit. I always thought i can control all code eval timing through require callback tree. 

I guess what I see as safest way to handle *define* calls is:

"Lazy dependency tree fetch" (only when module represented by this define is used in a require)

"Lazy factory eval"

Thanks to all.

Daniel.

[James Burke]

On Fri, Jan 11, 2013 at 11:19 AM, Rawld Gill <rg...@altoviso.com> wrote:
> James, after reading the link, it looks like you are "lazy evaluating" and don't mention when a script is injected (lazy or preemptive). Take a look at my proposed vocabulary in my previous message.

requirejs does lazy injection. I chose lazy injection when I converted
to lazy evaluation. The more work to put off until later the better,
particularly for mobile applications.

I think for most cases it does not matter -- if a top-level require
call triggers the load of a define, then that define's dependencies
are loaded right away regardless of a lazy or preemptive injection --
they dependencies are needed now.

The difference only comes up in built file cases, and only if not all
of the dependency graph has been built into the built file, and that
part of the dependency graph is not needed on first require().

I believe the cases for not building in part of the dependency graph
are mostly "load a base lib, like jquery" from a cdn, so it will
likely be part of the first require() dependency graph anyway. So, it
is a *very small* number of use cases that would actually observe the
difference in injection styles.

James

[Rawld Gill]

> -----Original Message-----
> From: amd-im...@googlegroups.com [mailto:amd-
> impl...@googlegroups.com] On Behalf Of James Burke
> Sent: Friday, January 11, 2013 5:33 PM
> To: amd-im...@googlegroups.com
> Subject: Re: [amd-implement] factory-on-define vs factory-on-require
>

> On Fri, Jan 11, 2013 at 11:19 AM, Rawld Gill <rg...@altoviso.com> wrote:
> > James, after reading the link, it looks like you are "lazy evaluating" and
> don't mention when a script is injected (lazy or preemptive). Take a look at
> my proposed vocabulary in my previous message.
>
> requirejs does lazy injection. I chose lazy injection when I converted to lazy
> evaluation. The more work to put off until later the better, particularly for
> mobile applications.

I'm not sure I agree with that last sentence. But I agree with your next point, which means our disagreement is mostly pointless. Therefore this is probably not worth spending too much time on unless someone can show us a use case where the difference is important.

>
> I think for most cases it does not matter -- if a top-level require call triggers
> the load of a define, then that define's dependencies are loaded right away
> regardless of a lazy or preemptive injection -- they dependencies are needed
> now.

Agree 100%.

>
> The difference only comes up in built file cases, and only if not all of the
> dependency graph has been built into the built file, and that part of the
> dependency graph is not needed on first require().

Not necessarily. It depends on how the files are "built". The Dojo builder and bdBuild express the module cache in such a way that the load behavior is identical to the unbuilt version...with the intended exception that a built module does not require script injection in order to have its resource evaluated.

>
> I believe the cases for not building in part of the dependency graph are
> mostly "load a base lib, like jquery" from a cdn, so it will likely be part of the
> first require() dependency graph anyway. So, it is a *very small* number of
> use cases that would actually observe the difference in injection styles.
>
> James

--Rawld

[Rawld Gill]

> From: Daniel Dotsenko

>
> Thank you again, Rawld
>
> Your point 1.1.1.a had solidified my understanding that Dojo (if designed like
> you described, "Dependency tree Pre-fetch" + "Lazy factory eval") may
> actually cause some portion of code linked as dependencies to defines to
> "run" before the module declared by that define is used in require.
>
> This saddens me a bit. I always thought i can control all code eval timing
> through require callback tree.

You can.

So, in your example, I'm guessing you're trying to get the following eval order

1. jquery (which is jquery wrapped by an AMD module).
2. path/to/plainjsfile.js (which contains a jquery plugin and is not expressed as an AMD module but depends on jquery being loaded before it can be eval'd).
3. function($){ /* do something with augmented jQuery */ }

You simply need to have a AMD plugin that loads jQuery plugins; something like:

//jQueryPluginLoader
define(["jquery", "dojo/text"], function(jquery, text){
return {
load:function(id, require, load){
text.load(id, require, function(src){
// this eval is sloppy. There are better ways to do this,
// but that's not the point of this example
eval(src);
load(window.jQuery);
});
}
};
});

[You could even implement the jQueryPluginLoader load function with script injection if you hate eval.]

Now, express your program like this:

require(["jquery", "jQueryPluginLoader!path/to/plainjsfile.js"], function($){

// do something with augmented jQuery

});

Of course you can use jQueryPluginLoader over and over in this and other programs.

Sidebar of optimism for anybody still reading:
The moral to this story is that the AMD spec is simple yet deceptively powerful. I have yet to be presented with a load problem that it cannot solve. This is what keeps me interested--even excited--about the work we are all doing together.

--Rawld