multiple ids mapped to the same url

[Ben Hockey]

what should happen here?

// script tag in page

require({

baseUrl: './',

paths: {

multiple: 'multiple',

duplicate: 'multiple'

}

}, ['multiple', 'duplicate'], function (multiple, duplicate) {

multiple.multi = true;

console.log('multiple', multiple.multi);

console.log('duplicate', duplicate.multi);

});

// multiple.js

console.log('loaded');

define(function (require, exports, module) {

console.log('defined', module.id);

return {};

});

i'd expect that a "good" loader would at least not try to fetch the same url twice - not sure that would be something needing to be mandated in a spec but might be worth discussing what each of us would expect in case someone thinks there's a good reason to try to fetch the code from the same url twice.

should the "multiple" and "duplicate" modules be the same module - ie the module's factory only gets run once and the exports are cached and shared - or should the module's factory be run for each module id that maps to it and have independent exports for each module?

my opinion is that these should be 2 different modules since module ids are a layer of abstraction that maps to urls.  if 2 module ids happen to map to the same url, they are still 2 logically distinct modules.  i'd expect to see the following in the console:

loaded

defined multiple

defined duplicate

multiple true

duplicate undefined

what does everyone else think?

is there even agreement that we could have a valid case where 2 module ids could map to the same url?

thanks,

ben...

[James Burke]

On Wed, Feb 29, 2012 at 7:17 AM, Ben Hockey <neonst...@gmail.com> wrote:
> my opinion is that these should be 2 different modules since module ids are
> a layer of abstraction that maps to urls.  if 2 module ids happen to map to
> the same url, they are still 2 logically distinct modules.  i'd expect to
> see the following in the console:

There are some nuances, or tradeoffs with that approach:

1) multiple.js might actually be a built layer. So there may not be
any anonymous module in that file, making it difficult to decide what
to give for 'duplicate'. If there is a named 'duplicate' module in the
built file, the answer is easy (return that module), but this use case
seems to be about anonymous modules.

In any case, it seems realistic to want to map multiple built modules
to the same file, and that config would look similar to the above.
Taka Kojima suggested in a comment for this gist:
https://gist.github.com/1630091

that perhaps that sort of config would be a different type of config.
But I think that is just an optimization of the format for modules
that fit under a module ID, for specifying that 'a' and 'b' are in the
same JS file, I expect it would look like the config in this example.

2) To enable this behavior, it would mean a loader would have to hold
on to the factory functions for every anonymous module, just in case
this config case comes up. At first thought, I do not like that -- it
means more bookkeeping and keeping around more in memory for things
that usually are not needed.

3) How would this optimize? It seems like two versions of the
multiple.js would show up in the built file. That seems wasteful.

So while the behavior your advocate makes logical sense, this behavior
is a bit too tricky (it looks like a way to do object instantiation,
where modules should be more like singletons/statics), and the use
case for this being useful is so small that it is overshadowed by the
extra bookkeeping/memory cost and possible confusion and extra file
size cost when mapping to built files.

At least that is my first pass thinking through it. I'm open to
counterarguments. In particular, building up how this use case is
useful and would occur frequently enough to warrant the costs for it.

James

[Taka Kojima]

I am definitely in favor of one anonymous module definition per file.

It gets super messy doing anything else, especially in older browsers that don't handle onLoad events as they should.

I don't think any AMD loader should have to support more than one anonymous module per file, and don't think it should be standardized upon at all. IMO, it could turn into a maintenance nightmare if you are developing your application this way.

I am in favor of being able to configure your paths so that you can map moduleA, moduleB and moduleC to the same file IF they are all named modules, which would usually mean it's a built file.

Having a builder spit out a list of module > file mappings, seems like a good simple way to handle module lookup, without having to modify the built files.

I think RequireJS does something like require.pause() at the top of each built file, then require.resume() at the end and this is how RequireJS handles multiple named modules per file. Please correct me if I'm wrong James.

I built a library to handle a lot of what AMD does before I really dug into AMD. (This is why I wrote an AMD loader, it started off as me porting what I had over to an AMD format). This is how I handled this in my other project:

Much like RequireJS you would specify a main module(s) to build, the build tool would then recursively look up dependencies and concat + minify them all into one file.

The build tool would then spit out configuration code. This could be written to a file instead, but in essence it looked like:

        minion.configure({paths: [

            {

                "file": "path/to/js/classes1.min.js",

                "classes": [

                        "example.Example",

                        "example.Example2",

                        "example.Example3"

                ]

            },

            {

                "file": "path/to/js/classes2.min.js",

                "classes": [

                        "example.Example4",

                        "example.Example5",

                        "example.Example6"

                ]

            }

        ]});

If you swap out "classes" with "modules" and the periods to "/", I think something like this should exist as part of the spec.

This allows you to specify upfront what files your modules are mapped to, and it removes the need for having any extra library specific code compiled into your built files.

It also lets you easily set up a js file during dev with multiple module definitions in it, without having to add any extra code at the top/bottom of your JS file.

Taka

[Ben Hockey]

On Wednesday, February 29, 2012 2:43:36 PM UTC-5, Taka Kojima wrote:

I am definitely in favor of one anonymous module definition per file.

It gets super messy doing anything else, especially in older browsers that don't handle onLoad events as they should.

 

i think you misunderstood my question - there's only one anonymous define in the module.

ben...

[Taka Kojima]

Yeah, I just realized I didn't answer your question exactly, which is funny because I kind of assumed what you were trying to do without looking at the contents of `multiple.js`.

So, the use case for mapping multiple modules to the same file has different meaning in both of our scenarios, and I personally see the need for mine vs yours as a more common need, but I could be wrong.

In your case, if you really needed to do that what about something like:

// multiple.js

(function(){

    var fac = {};

    define("multiple", function (require, exports, module){

        return fac;

    });

    define("duplicate", function (require, exports, module){

        return fac;

    });

});

[Taka Kojima]

Or better yet:

    define("multiple", function (require, exports, module){

        return {};

    });

    define("duplicate", require("multiple"));

[Ben Hockey]

my example is contrived and i don't have a real use case for it - that's why i was also asking if there was agreement that this case should even be considered valid. 

to give a little bit of background to my question, i was playing with looking at heap snapshots with chrome and saw that dojo's loader was caching calls to require([]) in the same way that results from define are cached.  i opened a ticket in dojo ( http://bugs.dojotoolkit.org/ticket/14909) with a patch to stop caching those calls.  rawld made a comment on that ticket about other things which were cached that could be removed too.  then i got off on a tangent wondering if those other things which were cached might ever be useful and i realized that everything that was needed to run the factory again was cached.  so i tried to think of a scenario where running the factory again might be valid.  that's what led to this question.

since i don't have a real problem i'm not really looking for alternatives to how this could be done.  i just wanted to get an answer to an interesting question.  it would probably only be a matter of time before somebody unknowingly tries this - particularly once "map" is in wider use.  it wouldn't take much for someone to lose track and once they've mapped a dependency into a module that then resolves to another module id which resolves via a path to a url they may start to wonder why changing foo in moduleA affects bar in moduleB.  i'm not sure that's even the same as my example but anyhow, i'm just exploring some of the dark corners of what naive users might attempt to do and wanted to get some opinions about what should happen.

thanks,

ben...

[Taka Kojima]

Gotcha.

The way NeedsJS  behaves is as follows (the code in index.html is a bit different):

// script tag in page

require.config({

baseUrl: 'js',

paths: {

multiple: 'multiple',

duplicate: 'multiple'

}

});

require(['multiple', 'duplicate'], function (multiple, duplicate) {

multiple.multi = true;

console.log('multiple', multiple.multi);

console.log('duplicate', duplicate.multi);

});

// js/multiple.js

console.log('loaded');

define(function (require, exports, module) {

console.log('defined', module.id);

return {};

});

This is what gets logged:

loaded

multiple.js:4defined multiple

Needs and RequireJS both don't try to load module.js twice and both never call the callback function from the require() call.

This is because if you specify multiple paths for the same file, it expects two define calls from that file. This means that you could have an async define call in module.js, i.e. something like setTimeout(define...), but until the second define() call is made Needs just sits and waits.

The timeout doesn't kick in here, so no error ever gets fired, maybe it should, this is an interesting scenario and maybe I should be firing a timeout error here.

You can swap Needs and RequireJS out with the above code, RequireJS does fire the error timeout.

2

loaded

multiple.js:4defined multiple

1. require.js:27Uncaught Error: Load timeout for modules: duplicate http://requirejs.org/docs/errors.html#timeout

The main difference is that I fire an error timeout if a request for a js file fails to return after x seconds. RequireJS seems to tie the timeout to the actual define() calls.

I like how RequireJS handles it better.

Taka

[Rawld]

On 02/29/2012 07:17 AM, Ben Hockey wrote:

what should happen here?

// script tag in page

require({

baseUrl: './',

paths: {

multiple: 'multiple',

duplicate: 'multiple'

}

}, ['multiple', 'duplicate'], function (multiple, duplicate) {

multiple.multi = true;

console.log('multiple', multiple.multi);

console.log('duplicate', duplicate.multi);

});

// multiple.js

console.log('loaded');

define(function (require, exports, module) {

console.log('defined', module.id);

return {};

});

i'd expect that a "good" loader would at least not try to fetch the same url twice - not sure that would be something needing to be mandated in a spec but might be worth discussing what each of us would expect in case someone thinks there's a good reason to try to fetch the code from the same url twice.

I had such an impl back in the early days. I became convinced that a deployed app being used by normal users with normal cache settings would not be harmed by two fetches to the same URL because "good" browsers always return the cached module...so I dropped that feature.

should the "multiple" and "duplicate" modules be the same module - ie the module's factory only gets run once and the exports are cached and shared - or should the module's factory be run for each module id that maps to it and have independent exports for each module?

my opinion is that these should be 2 different modules since module ids are a layer of abstraction that maps to urls.  if 2 module ids happen to map to the same url, they are still 2 logically distinct modules.  i'd expect to see the following in the console:

loaded

defined multiple

defined duplicate

multiple true

duplicate undefined

what does everyone else think?

I agree; please tell me that's what dojo does :).

Note also, that it's been mentioned elsewhere that the the resource the contains the define application may have other code (just like your example has) that the programmer intends to execute outside the scope of the define application. That is the programmer's choice, and if they want to do that, I think they should be allowed.

is there even agreement that we could have a valid case where 2 module ids could map to the same url?

Sure. The DOH using dojo modules to test dojo modules.

--Rawld

[Rawld]

On 02/29/2012 09:34 AM, James Burke wrote:
> On Wed, Feb 29, 2012 at 7:17 AM, Ben Hockey<neonst...@gmail.com> wrote:
>> my opinion is that these should be 2 different modules since module ids are
>> a layer of abstraction that maps to urls. if 2 module ids happen to map to
>> the same url, they are still 2 logically distinct modules. i'd expect to
>> see the following in the console:
> There are some nuances, or tradeoffs with that approach:
>
> 1) multiple.js might actually be a built layer.

A built version should not cause behavior different than an unbuilt
version other than eliminating server transactions.

> So there may not be
> any anonymous module in that file, making it difficult to decide what
> to give for 'duplicate'. If there is a named 'duplicate' module in the
> built file, the answer is easy (return that module), but this use case
> seems to be about anonymous modules.

Built layers can easily provide anonymous modules (the dojo
loader/builder can do this).

> In any case, it seems realistic to want to map multiple built modules
> to the same file, and that config would look similar to the above.
> Taka Kojima suggested in a comment for this gist:
> https://gist.github.com/1630091
>
> that perhaps that sort of config would be a different type of config.
> But I think that is just an optimization of the format for modules
> that fit under a module ID, for specifying that 'a' and 'b' are in the
> same JS file, I expect it would look like the config in this example.
>
> 2) To enable this behavior, it would mean a loader would have to hold
> on to the factory functions for every anonymous module, just in case
> this config case comes up. At first thought, I do not like that -- it
> means more bookkeeping and keeping around more in memory for things
> that usually are not needed.

iiuc "this behavior" means avoiding the second server transaction...I agree.

> 3) How would this optimize? It seems like two versions of the
> multiple.js would show up in the built file. That seems wasteful.

That should be left up to the dev who specified this kind of weird
config to specify a build profile to get the desired effect. It is
certainly possible to optimize.

--Rawld

[Rawld]

On 02/29/2012 11:50 AM, Taka Kojima wrote:

In your case, if you really needed to do that what about something like:

// multiple.js

(function(){

    var fac = {};

    define("multiple", function (require, exports, module){

        return fac;

    });

    define("duplicate", function (require, exports, module){

        return fac;

    });

});

I am strongly against *ever* specifying an absolute module id in the define application...including built modules.

It is only necessary when doing something foolish like manually script injecting a module as we must allow in dojo for backcompat reasons.

--Rawld

[Ben Hockey]

On Tuesday, March 6, 2012 5:08:23 AM UTC-5, Rawld wrote:

I agree; please tell me that's what dojo does :).

i expected it would work in dojo but it seems to fail somewhere.  the callback of the top level require is never executed.  note that i don't have a use case for this so i don't *need* a fix - i'm just providing you with feedback.

 

ben...

[Rawld]

bummer. I filed a ticket: http://trac.dojotoolkit.org/ticket/14970

[James Burke]

On Tue, Mar 6, 2012 at 2:08 AM, Rawld <rg...@altoviso.com> wrote:
> I had such an impl back in the early days. I became convinced that a
> deployed app being used by normal users with normal cache settings would not
> be harmed by two fetches to the same URL because "good" browsers always
> return the cached module...so I dropped that feature.

I had a bug filed a while back on requirejs to fix this because I was
doing reloads for each ID even though they mapped to the same URL. It
is far more common to want to only fetch the script once, and map
multiple IDs that were built into one file to one script, and only
have it loaded once, than to try to get this other behavior.

Maybe using paths settings is not the right way to solve that build
issue, maybe there needs to be something else.

However, this kind of "create two different IDs that load the same
file that have an anonymous define() to get two different module
exports from the same factory function" is likely going to lead to
unexpected, undesirable consequences. Such as multiple inclusions of
the same factory function in a built file once that config is used in
a build.

So I'm still missing the use case that makes creating these other
hazards worthwhile. It would also imply creating some other config to
express "all these IDs are in this built file".

James