Relative module ID resolution

376 views
Skip to first unread message

James Burke

unread,
May 10, 2011, 4:25:49 PM5/10/11
to amd-im...@googlegroups.com
This came up on the dojo-contributors list, full thread here:
http://thread.gmane.org/gmane.comp.web.dojo.devel/14580

but I will summarize and provide some background here for AMD folks
not familiar with path resolution of module IDs.

Background
----------------

For an AMD loader to work in the browser efficiently, it should only
do one URL lookup for the module. This is different from traditional
CommonJS Modules 1.1 spec loaders on the server, where they normally
use a require.paths array to search for modules.

AMD supports the CommonJS type of module IDs that look like
'some/module', which gets mapped to an URL using some configuration in
the loader.

This normally involves using a "baseUrl" and appending the module ID
plus '.js' to create the path. So, for 'some/module', the URL ends up
being baseUrl + 'some/module.js'.

There are a few AMD implementations, requirejs, curl, bdload/dojo
loader that allow mapping *part* of a module ID to a different path
than what would normally be used if just using the baseUrl.

For example, in requirejs, this configuration call would result in an
URL of baseUrl + 'thirdparty/special/some/module.js':

require({
paths: {
'some': 'thirdparty/special/some'
}
});

Some loaders, like requirejs and dojo, support configuring the
location of a set of modules according to the CommonJS Packages spec's
use of 'main' and 'lib' properties:
http://requirejs.org/docs/api.html#packages

I now believe the 'main' and 'lib' properties of the CommonJS Packages
spec to be bad ideas. Node 0.4 does support the 'main' property.
However Node removed support for 'lib' in 0.4 because of the extra
path magic it required. I agree with this conclusion, as I believe it
contributes to this relative ID problem.

Summary of Issue:
----------------

If a module does a require('foo') and then foo.js has a
require('./bar'), how should './bar' be resolved to an URL?

Options:
----------------

1) "Relative to module ID": resolve the relative module ID, './bar',
relative to the reference ID, 'foo'. So, the relative module ID gets
converted to 'foo/bar' and then the path/package configuration is used
to resolve it.

2) "Relative to module URL": Resolve the relative ID './bar' relative
to the reference ID's URL. So, in the simple case: URL prefix +
'foo.js' + './bar.js' (but of course cleaned up to be a real path).

Problems:
----------------

There are many configurations where #1 and #2 result in the same
value, so it does not matter. However, there are two cases where it
makes a difference:

1) CommonJS Package's 'lib' directory is in play. In the example, if
'foo' is a CommonJS package, and its main: 'foo.js' and 'lib' is set
to 'scripts', then the paths get resolved to:

Option 1: foo/scripts/bar.js
Option 2: foo/bar.js

I suggest we follow Node's lead here and discourage AMD
implementations from implementing 'lib' support. This removes the
problem with this discrepancy, and makes implementation easier all
around.

2) paths: configuration. There could be this type of configuration set up:

require({
paths: {
'foo/bar': 'overrides/foo/bar'
}
});

The resulting URLs:

Option 1: baseUrl + 'overrides/foo/bar.js'
Option 2: baseUrl + 'foo/bar.js'

I believe Option 1 is the desirable behavior, because this allows
proper overriding of modules. This is particularly useful if you are
building a toolkit, and you know that you can deploy a different
version of 'arrayExtras' depending on the deployment scenario (if only
webkit, then map 'arrayExtras' to an empty file). If option 2 is used,
you will *not* be able to override the path used if './arrayExtras' is
used.

Summary
----------------

Those were the issues I were able to come up with, but I may have
missed something so please point it out. However, based on this
information, I favor using Option 1, 'relative to module ID'.

This is actually a change for RequireJS, and I will update the code to
reflect the change if other implementers agree. As part of that
update, I would remove CommonJS Package 'lib' support too.

The original reason RequireJS supports Option 2 'relative to module
URL' was because of this issue:
https://github.com/jrburke/requirejs/issues/51

but I believe with the removal of 'lib' support issue #51 goes away.

James

Ben Hockey

unread,
May 10, 2011, 4:41:44 PM5/10/11
to amd-im...@googlegroups.com

Options:
----------------

1) "Relative to module ID": resolve the relative module ID, './bar',
relative to the reference ID, 'foo'. So, the relative module ID gets
converted to 'foo/bar' and then the path/package configuration is used
to resolve it.

i think you have a typo here and getting clarification is important...

the module with id 'foo' declares a dependency as './bar'.  if i'm not mistaken, this does not resolve to 'foo/bar' but rather <parent of foo>/bar.  a large part of the issue here is what is <parent of foo>?

some options for this are:
 a) <parent of foo> is always baseUrl and the resolved id for './bar' would be the same is if it had been declared as 'bar'
 b) <parent of foo> is the url of the directory containing foo
   so, if 'foo' had a mapping of 'my/long/name/foo' then './bar' would resolve to baseUrl + 'my/long/name/bar'
 
ben...

Ben Hockey

unread,
May 10, 2011, 4:47:56 PM5/10/11
to amd-im...@googlegroups.com
to clarify option b) a little further - if there is no path/package mapping for 'foo' then a) and b) are equal since <parent of foo> would already be baseUrl.

James Burke

unread,
May 10, 2011, 4:51:56 PM5/10/11
to amd-im...@googlegroups.com
On Tue, May 10, 2011 at 1:41 PM, Ben Hockey <neonst...@gmail.com> wrote:
>
>> Options:
>> ----------------
>>
>> 1) "Relative to module ID": resolve the relative module ID, './bar',
>> relative to the reference ID, 'foo'. So, the relative module ID gets
>> converted to 'foo/bar' and then the path/package configuration is used
>> to resolve it.
>
> i think you have a typo here and getting clarification is important...
> the module with id 'foo' declares a dependency as './bar'.  if i'm not
> mistaken, this does not resolve to 'foo/bar' but rather <parent of foo>/bar.
>  a large part of the issue here is what is <parent of foo>?

Hmm, I do not think so? The example I have was that something asks for
require('foo'). This could be done like so too: define(['foo'],...) or
require(['foo']).

In that case, foo is a top-level ID. The module asking for it does not
matter for the path resolution?

James

Ben Hockey

unread,
May 10, 2011, 5:04:01 PM5/10/11
to amd-im...@googlegroups.com
the module asking for it does matter but normal file path resolution would be that from a file at <somewhere>/foo.js a reference to './bar.js' is then intended to mean <somewhere>/bar.js and not <somewhere>/foo/bar.js - the only difference here is that for AMD we take ".js" off everything.  if foo was a directory then it would mean <somewhere>/foo/bar.js

paths: {
    'one': 'foo/bar/one.js'
}

if load 'one' and it has the dependency './two' - which file will it get?  it won't be one/two.js but it will be <parent of one>/two.js


fwiw, i like option b.


ben...

Ben Hockey

unread,
May 10, 2011, 5:05:37 PM5/10/11
to amd-im...@googlegroups.com
correction - remove '.js', it was a cut/paste error and i don't want it to cloud the example

paths: {
    'one': 'foo/bar/one'
}

James Burke

unread,
May 10, 2011, 5:23:07 PM5/10/11
to amd-im...@googlegroups.com
On Tue, May 10, 2011 at 2:04 PM, Ben Hockey <neonst...@gmail.com> wrote:
> the module asking for it does matter but normal file path resolution would
> be that from a file at <somewhere>/foo.js a reference to './bar.js' is then
> intended to mean <somewhere>/bar.js and not <somewhere>/foo/bar.js - the
> only difference here is that for AMD we take ".js" off everything.  if foo
> was a directory then it would mean <somewhere>/foo/bar.js

Hmm, I got lost here. Module IDs are not actual URL paths. They can be
used in a path lookup, and you can normally guess where the module is
at based on the ID, but for instance, in a built file, there is no
path lookup because multiple modules are included in one file.

I feel like there is a disconnect, I still do not see how the module
that require's 'foo' affects the module resolution in either option,
given that 'foo' is a non-relative ID.

> consider this
> file https://github.com/jrburke/requirejs/blob/master/tests/relative/foo/bar/one.js and
> lets say https://github.com/jrburke/requirejs/blob/master/tests/relative/ is
> my baseURl and i had an entry in the path map for
> paths: {
>     'one': 'foo/bar/one.js'
> }
> if load 'one' and it has the dependency './two' - which file will it get?
>  it won't be one/two.js but it will be <parent of one>/two.js
> option a)
> - https://github.com/jrburke/requirejs/blob/master/tests/relative/two.js
> option b)
> - https://github.com/jrburke/requirejs/blob/master/tests/relative/foo/bar/two.js
> fwiw, i like option b.

That specific example has a problem because
tests/relative/foo/bar/one.js defines a named module, 'foo/bar/one',
so mapping 'one' to 'foo/bar/one' would cause an error then asking for
require('one') will cause an error.

But it does point out why using module ID instead of URL is useful in
a built situation: in that case, 'foo/bar/one' module asking for
'./two' means it can be found at the ID 'foo/bar/two', which would
work with a built file.

What am I missing?

James

Ben Hockey

unread,
May 10, 2011, 5:40:58 PM5/10/11
to amd-im...@googlegroups.com
agreed there seems to be some kind of disconnect and i'm willing to admit that it could be on my end if it turns out to be so.  if anyone else reading is able to see where the disconnect is then feel free to try to bring some clarity.

looks like i picked a bad file as an example - everything was right about it except that the module was not anonymous :)

hopefully later tonight i'll work on a concrete example and provide a link to demonstrate what i mean and help remove the disconnect.

ben...

Rawld Gill

unread,
May 10, 2011, 6:11:33 PM5/10/11
to amd-im...@googlegroups.com

I agree.

>
> Summary of Issue:
> ----------------
>
> If a module does a require('foo') and then foo.js has a
> require('./bar'), how should './bar' be resolved to an URL?
>
> Options:
> ----------------
>
> 1) "Relative to module ID": resolve the relative module ID, './bar',
> relative to the reference ID, 'foo'. So, the relative module ID gets
> converted to 'foo/bar' and then the path/package configuration is used
> to resolve it.

I'm confused and have the same observation that Ben has made. Let's consider a
specific foo.js:

// begin foo.js
define(["./bar", require], function(bar, require) {
// the foo module factory...
// stuff...


bar2= require("./bar");

require(["./bar"], funciton(bar3) {
// at this point bar===bar2===bar3
});

// more stuff...
});
// end foo.js

I believe:

1. The loader should deliver the same module for bar, bar2, and bar3. Those
require's are not required and bad form...they are there solely to demonstrate
this point.

2. The absolute module id implied by "./bar" for Option 1 is "bar". This is
because modules have an implied root, so "foo" is, at least heuristically,
equivalent to "/foo" in the module id space (that's not a URL), and "." is the
parent module of the reference module (note: I don't like that, but I lost
that argument months ago).


>
> 2) "Relative to module URL": Resolve the relative ID './bar' relative
> to the reference ID's URL. So, in the simple case: URL prefix +
> 'foo.js' + './bar.js' (but of course cleaned up to be a real path).

So, here are you saying "." is the directory that holds foo?

>
> Problems:
> ----------------
>
> There are many configurations where #1 and #2 result in the same
> value, so it does not matter. However, there are two cases where it
> makes a difference:
>
> 1) CommonJS Package's 'lib' directory is in play. In the example, if
> 'foo' is a CommonJS package, and its main: 'foo.js' and 'lib' is set
> to 'scripts', then the paths get resolved to:
>
> Option 1: foo/scripts/bar.js
> Option 2: foo/bar.js
>
> I suggest we follow Node's lead here and discourage AMD
> implementations from implementing 'lib' support. This removes the
> problem with this discrepancy, and makes implementation easier all
> around.

I advocate the following rule:

"When you specify a relative module id, it is relative to the reference
module's id. No further mapping of the module id is applied"

Since the loader must compute a non-relative module id for every module, the
reference module id is well-defined.

Since no further mapping is applied, loaders could still understand the lib
and main package config variables...with respect to *non-relative* module ids.

>
> 2) paths: configuration. There could be this type of configuration set up:
>
> require({
> paths: {
> 'foo/bar': 'overrides/foo/bar'
> }
> });
>
> The resulting URLs:
>
> Option 1: baseUrl + 'overrides/foo/bar.js'
> Option 2: baseUrl + 'foo/bar.js'
>
> I believe Option 1 is the desirable behavior, because this allows
> proper overriding of modules. This is particularly useful if you are
> building a toolkit, and you know that you can deploy a different
> version of 'arrayExtras' depending on the deployment scenario (if only
> webkit, then map 'arrayExtras' to an empty file). If option 2 is used,
> you will *not* be able to override the path used if './arrayExtras' is
> used.

You can still solve this problem with option 1. If you are building a toolkit,
then the toolkit will have a top-level module id, say "dojo". Then, for
example, from the reference module "dojo/lang" you could specify the relative
module "./array", which resolves to "dojo/array", which can be mapped in paths
as usual to a different URL.

Here's why relative ids should be relative to the reference module's id, not
the reference module's url....

The mapped version of array (to continue the example) may have the same deps
vector as the unmapped version of array--in fact, it almost always will. Let's
say it has the deps ["./kernel"].

If the mapped version of array is located in the same directory as the
unmapped version, then either method will work.

But if the mapped version is located someplace else and the loader tries to
resolve "./kernel" with respect to the somplace else URL, then the system will
fail. otoh, if the loader resolves "./kernel" with respect to the reference
module id, in this case still "dojo/array" irrespective of its actual URL,
then the proper kernel module will be used.


>
> Summary
> ----------------
>
> Those were the issues I were able to come up with, but I may have
> missed something so please point it out. However, based on this
> information, I favor using Option 1, 'relative to module ID'.
>
> This is actually a change for RequireJS, and I will update the code to
> reflect the change if other implementers agree. As part of that
> update, I would remove CommonJS Package 'lib' support too.
>
> The original reason RequireJS supports Option 2 'relative to module
> URL' was because of this issue:
> https://github.com/jrburke/requirejs/issues/51

With the clarification given above, option 1 can support issues/51.


I support option 1.

Best,
Rawld

unscriptable

unread,
May 10, 2011, 6:12:52 PM5/10/11
to amd-implement
> Options:
> ----------------
>
> 1) "Relative to module ID": resolve the relative module ID, './bar',
> relative to the reference ID, 'foo'. So, the relative module ID gets
> converted to 'foo/bar' and then the path/package configuration is used
> to resolve it.
>
> 2) "Relative to module URL": Resolve the relative ID './bar' relative
> to the reference ID's URL. So, in the simple case: URL prefix +
> 'foo.js' + './bar.js' (but of course cleaned up to be a real path).

Hey James, can you expand on this part? Because I come to the same
conclusion as Ben when I read it. You're skipping over quite a lot of
steps/definitions in these options.

FWIW, curl.js implements the "main" and "lib" path magic, too. It's a
bit of a mind bender to users, I agree.

-- John

unscriptable

unread,
May 10, 2011, 6:33:47 PM5/10/11
to amd-implement
Hey Rawld, by this, I assume you mean:
Since the loader must compute a non-relative module id for every
*dependent* module, the reference module id is well-defined.

What exactly do you mean by "well-defined"?


>
> Since no further mapping is applied, loaders could still understand the lib
> and main package config variables...with respect to *non-relative* module ids.

Can you expand on this, too?
I'm having trouble determining if the following is supported by option
1:

require({
paths: {
"array": "dojo/array"
}
});

require(["array"], function (array) {
// do something with array module
});


or this:

require({
paths: {
"core/array": "dojo/array"
}
});

require(["core/array"], function (array) {
// do something with array module
});

Thanks!

-- J

James Burke

unread,
May 10, 2011, 6:59:10 PM5/10/11
to amd-im...@googlegroups.com
On Tue, May 10, 2011 at 3:11 PM, Rawld Gill <rg...@altoviso.com> wrote:
> On Tuesday 10 May 2011 13:25:49 James Burke wrote:
>> Options:
>> ----------------
>>
>> 1) "Relative to module ID": resolve the relative module ID, './bar',
>> relative to the reference ID, 'foo'. So, the relative module ID gets
>> converted to 'foo/bar' and then the path/package configuration is used
>> to resolve it.
>
> I'm confused and have the same observation that Ben has made.
[snip]

> 1. The loader should deliver the same module for bar, bar2, and bar3. Those
> require's are not required and bad form...they are there solely to demonstrate
> this point.
>
> 2. The absolute module id implied by "./bar" for Option 1 is "bar". This is
> because modules have an implied root, so "foo" is, at least heuristically,
> equivalent to "/foo" in the module id space (that's not a URL), and "." is the
> parent module of the reference module (note: I don't like that, but I lost
> that argument months ago).

Yeah, so I *really* messed up on that example: require('./bar') would
resolve to just 'bar', since 'foo' is a top-level ID, *not* as I said,
'foo/bar'. Very sorry for muddying the waters.

>>
>> 2) "Relative to module URL": Resolve the relative ID './bar' relative
>> to the reference ID's URL. So, in the simple case: URL prefix +
>> 'foo.js' + './bar.js' (but of course cleaned up to be a real path).
>
> So, here are you saying "." is the directory that holds foo?

My bad again. The resolved URL for './bar' would be URL prefix +
'bar.js', which would be the same as option 1 unless a paths:
configuration for 'bar' was in play.

> I advocate the following rule:
>
> "When you specify a relative module id, it is relative to the reference
> module's id.  No further mapping of the module id is applied"

I think the "no further mapping of the module ID is applied" is a bit
confusing, or at least does not add any more guidance for relative ID
resolution: it seems enough to say "resolve the relative ID relative
to the reference module ID". Then, that resolved module ID gets the
configuration mapping applied to generate its URL.

> Since the loader must compute a non-relative module id for every module, the
> reference module id is well-defined.
>
> Since no further mapping is applied, loaders could still understand the lib
> and main package config variables...with respect to *non-relative* module ids.

When "lib" configuration is in play, this does not work out, and this
is the cause of the issue #51 in requirejs. Consider a package named
pkgName that has main: 'main.js' and lib: 'lib'.

* Something does a require('pkgName')
* inside pkgName/main.js there is a require('./lib/support')

With Option 1, the resolved relative module ID would be
'pkgName/lib/support'. However, given 'lib' rules, this would be
mapped to a 'pkgName/lib/lib/support.js' URL, which is wrong.

But this is just evidence to me that lib is a bad idea, and we should
not support it anymore.

> Here's why relative ids should be relative to the reference module's id, not
> the reference module's url....
>
> The mapped version of array (to continue the example) may have the same deps
> vector as the unmapped version of array--in fact, it almost always will. Let's
> say it has the deps ["./kernel"].
>
> If the mapped version of array is located in the same directory as the
> unmapped version, then either method will work.
>
> But if the mapped version is located someplace else and the loader tries to
> resolve "./kernel" with respect to the somplace else URL, then the system will
> fail. otoh, if the loader resolves "./kernel" with respect to the reference
> module id, in this case still "dojo/array" irrespective of its actual URL,
> then the proper kernel module will be used.

Agreed, however, I think the tricky situation is the one Ben
originally raised in the dojo-contributors list is this:

I want to create an 'array' module. However I have a paths:
configuration with 'array': 'dojo/array', and 'dojo/array' has a
dependency on './kernel'.

So, in my project, when I do require('array'), the relative ID
'./kernel' in Option 1 will be resolved to 'kernel' (a top-level ID)
which does not exist. With Option 2, it would find the right
'dojo/kernel.js' file.

However, with Option 2, the developer could *never* use paths: config
to map 'dojo/kernel' to another file, since the relative ID is
resolved via the URL -- paths config cannot be used as part of that
URL resolution.

I believe Ben's case should be solved by having his 'array' to just
delegate to dojo/array:

//array.js:
define('[dojo/array'], function (array){
return array;
});

Because losing the ability to map 'dojo/kernel' to some other file is
a greater loss in capability, and when considering built/optimized
files with more than one module in it, those modules use module IDs to
name themselves, not URL paths.

Rawld, which I think you agree with, but just trying to illustrate the
tradeoffs better relative to Ben's original question.

James

Rawld Gill

unread,
May 10, 2011, 7:13:16 PM5/10/11
to amd-im...@googlegroups.com

Right.

>
> What exactly do you mean by "well-defined"?

Just that it has an unabiguous name known to the loader: given a module-id,
it can refer to exactly one module within any context.

From my perspective:

* both are fine.

* the exsitence or not of paths doesn't affect the discussion.

I would rephrase the options as follows:

1. A relative module id is resolved with respect to the reference module's id.

2. A relative module id is resolved with respect to the reference module's
URL.

I strongly support [1].


Best,
Rawld

Ben Hockey

unread,
May 10, 2011, 10:35:45 PM5/10/11
to amd-im...@googlegroups.com

Yeah, so I *really* messed up on that example: require('./bar') would
resolve to just 'bar', since 'foo' is a top-level ID, *not* as I said,
'foo/bar'. Very sorry for muddying the waters.

i was afraid i was losing it for a while there...  

i've built a test file showing the issue i'm talking about.  you can see it here http://neonstalwart.dojotoolkit.org/amd/relative/src/index.html and i've also made a build at http://neonstalwart.dojotoolkit.org/amd/relative/build/index.html - view source on both and take a look at the files involved.  the overview is this:
index.html loads require.js with a data-main of "main".  main.js contains a path configuration that maps "one" to "foo/bar/one" and then declares a dependency for "one" and the factory logs the value of one.two

in foo/bar/one.js, it has a dependency of "./two" and the factory returns { two: two };
foo/bar/two.js has no dependencies and returns the string "foo/bar/two"
two.js (sibling of main.js) has no dependencies and returns the string "two"

in practice, foo/bar/two.js is loaded as the "./two" dependency from foo/bar/one.js - below is part of the build output:

./main.js
----------------
./foo/bar/./two.js
./foo/bar/one.js
./main.js


the interesting part is after doing a build.  the module that is inlined with an explicit id of "two" is the foo/bar/two.js module.  this perhaps reveals that the intention is to resolve the "./two" dependency as baseUrl + two.js (because the top-level id "two" is the id given by the build) but in practice the foo/bar/two.js module is what is used in src and build.  i had never really paid attention to the ids given by the build system (they just worked) and so in my mind, it was intentional to resolve relative ids as urls.  however, if i had paid close attention to the build output i may have been able to see that the intention was to resolve ids relative to module names/ids and it is probably a bug in requirejs that it resolves relative to the url.


<snip> 

I believe Ben's case should be solved by having his 'array' to just
delegate to dojo/array:

//array.js:
define('[dojo/array'], function (array){
    return array;
});

this was already my plan if ids are to be resolved in a way that necessitates it.  my project that consumes array might pull in all its dependencies like util/array, util/lang, etc and then i can map the top level util to a path that contains a collection of adapters that do something like what you're suggesting.  if someone wanted to use YUI instead of dojo then they would map util to another collection that adapted YUI libraries to the API my project expected.
 

Because losing the ability to map 'dojo/kernel' to some other file is
a greater loss in capability, and when considering built/optimized
files with more than one module in it, those modules use module IDs to
name themselves, not URL paths.

i can see now (based on the build output of all things!) that james you probably intended to resolve ids relative to the module name but in reality the implementation resolves ids relative to the url.  the question is now, do you fix this "bug"?

ben...

unscriptable

unread,
May 11, 2011, 12:05:14 AM5/11/11
to amd-implement

> "When you specify a relative module id, it is relative to the reference
> module's id.  No further mapping of the module id is applied"

I can't accept this rule unless somebody gives me a clear use case
explaining why "no further mapping" is a bad idea. I don't get it.
What does it break exactly? Why can't we map the module id?


> But if the mapped version is located someplace else and the loader tries to
> resolve "./kernel" with respect to the somplace else URL, then the system will
> fail.

Of course this should fail. Why would anybody separate a module from
its dependencies? This makes no sense to me.


In a truly interoperable, modular world, this should work:

// folder structure:
js/
some/
hawt/
array.js
kernel.js

// config:
require({
baseUrl: "js",
paths: {
"array": "some/hawt/array"
}
});

// "main" module:
require(["array"], function (array) {
// use my standardized array abstraction in here
// notice in my dep list I am not referring to some/hawt/array
// or dojo/array or yui/array etc.
});

// js/some/hawt/array.js
define(["./kernel"], function (kernel) {
// use kernel and return a standardized array abstraction
// (like dojo does already)
});

If "No further mapping of the module id is applied", then the above
can't work because the loader will look for kernel.js at "js/
kernel.js" (no mapping) when it is actually located at "js/some/hawt/
kernel.js". If we apply the existing mapping rules (as found in
RequireJS 0.24 and curl.js 0.4.3), kernel.js will be found at "js/some/
hawt/kernel.js" where we would expect it.

Tomorrow, when I discover I want to start using dojo instead of the
some/hawt framework, I can change my paths config:

require({ paths: { "array": "dojo/array" }, baseUrl: "js" });

At JSCONF 2011, I talked about being able to write your code to a
standardized API using this straightforward method of mapping
standardized abstractions (such as "dojo/array" or "dojo/Sizzle") into
implementation-agnostic names (e.g. "array" or "querySelectorAll") --
and even quickly demoed it. People got very, very excited by this.
Rebecca also showed a code example that exemplified this and advocated
for it.

This is what most developers want from modules -- well, the developers
that aren't zealots or fanboys for a single framework want this.

And frankly, I still don't know what the problem is. I have yet to
see a use case that forces this: "No further mapping of the module id
is applied". Every use case or code snippet I've seen in this thread
so far works perfectly fine in curl.js.

What isn't working today? Why do we need these new rules that break
current, desirable functionality?

Please help me understand.

Thanks,

- John

Rawld Gill

unread,
May 11, 2011, 12:05:32 AM5/11/11
to amd-im...@googlegroups.com
On Tuesday 10 May 2011 15:59:10 James Burke wrote:
> On Tue, May 10, 2011 at 3:11 PM, Rawld Gill <rg...@altoviso.com> wrote:
snip

>
> > I advocate the following rule:
> >
> > "When you specify a relative module id, it is relative to the reference
> > module's id. No further mapping of the module id is applied"
>
> I think the "no further mapping of the module ID is applied" is a bit
> confusing, or at least does not add any more guidance for relative ID
> resolution: it seems enough to say "resolve the relative ID relative
> to the reference module ID". Then, that resolved module ID gets the
> configuration mapping applied to generate its URL.

I was trying to say that the module id is not further transformed. It may or
may not be used as an argument to find the URL that addresses the resource
that defines the module given by the module id. For example, maybe the module
has already been published to the loader directly by some method other than
having the loader go retrieve the resource.


>
> > Since the loader must compute a non-relative module id for every module,
> > the reference module id is well-defined.
> >
> > Since no further mapping is applied, loaders could still understand the
> > lib and main package config variables...with respect to *non-relative*
> > module ids.
>
> When "lib" configuration is in play, this does not work out, and this
> is the cause of the issue #51 in requirejs. Consider a package named
> pkgName that has main: 'main.js' and lib: 'lib'.
>
> * Something does a require('pkgName')
> * inside pkgName/main.js there is a require('./lib/support')
>
> With Option 1, the resolved relative module ID would be
> 'pkgName/lib/support'. However, given 'lib' rules, this would be
> mapped to a 'pkgName/lib/lib/support.js' URL, which is wrong.

Well, we're making up the rules here...right?:)

So, just for the sake of argument, apply my rule:

"When you specify a relative module id, it is relative to the reference
module's id. No further mapping of the module id is applied"

1. The reference module id is "pkgName/main".
2. The relative module id with respect to [1] is "./lib/support"

Therefore the absolute module id implied by [2] is:
"pkgName/main/.././lib/support" =>
"pkgName/lib/support"
which is exactly what we want.

>
> But this is just evidence to me that lib is a bad idea, and we should
> not support it anymore.

I will never argue in favor of lib. I, also, strongly dislike main and lib and
don't see the point. I agree with Kris Zyp...just use paths...keep it simple.

>
> > Here's why relative ids should be relative to the reference module's id,
> > not the reference module's url....
> >
> > The mapped version of array (to continue the example) may have the same
> > deps vector as the unmapped version of array--in fact, it almost always
> > will. Let's say it has the deps ["./kernel"].
> >
> > If the mapped version of array is located in the same directory as the
> > unmapped version, then either method will work.
> >
> > But if the mapped version is located someplace else and the loader tries
> > to resolve "./kernel" with respect to the somplace else URL, then the
> > system will fail. otoh, if the loader resolves "./kernel" with respect
> > to the reference module id, in this case still "dojo/array" irrespective
> > of its actual URL, then the proper kernel module will be used.
>
> Agreed, however, I think the tricky situation is the one Ben
> originally raised in the dojo-contributors list is this:
>
> I want to create an 'array' module. However I have a paths:
> configuration with 'array': 'dojo/array', and 'dojo/array' has a
> dependency on './kernel'.
>
> So, in my project, when I do require('array'), the relative ID
> './kernel' in Option 1 will be resolved to 'kernel' (a top-level ID)
> which does not exist. With Option 2, it would find the right
> 'dojo/kernel.js' file.

I think that's a defective configuration...if that's what Ben really wants (I
think what he really wants is a non-ambiguous way to do rational mappings).

I think he is asking, "how do a replace a single module, for example
"dojo/array", in the middle of a hierarchy of modules?"

If that's the question then my answer is paths looks like:

paths:{
"dojo/array":"path/to/replacement/module"
}

Relative paths in the replacement module are relative to module id, not the
url, so, e.g., "./kernel" resolves to the dojo/kernel in the dojo hierarchy,
not in the replacement hierarchy.


>
> However, with Option 2, the developer could *never* use paths: config
> to map 'dojo/kernel' to another file, since the relative ID is
> resolved via the URL -- paths config cannot be used as part of that
> URL resolution.

Agree.

To put a fine point on in, if the replacement wants to reference modules from
the original hierarchy, just specify a relative path (e.g., "./kernel"). If
the replacement also wants some modules outside of the original hierachy, then
say so, explicitly (e.g., "myReplacement/kernel"). Of course you can further
map myReplacement with paths.


>
> I believe Ben's case should be solved by having his 'array' to just
> delegate to dojo/array:
>
> //array.js:
> define('[dojo/array'], function (array){
> return array;
> });

Agree...strongly


>
> Because losing the ability to map 'dojo/kernel' to some other file is
> a greater loss in capability, and when considering built/optimized
> files with more than one module in it, those modules use module IDs to
> name themselves, not URL paths.
>
> Rawld, which I think you agree with, but just trying to illustrate the
> tradeoffs better relative to Ben's original question.


Best,
Rawld

unscriptable

unread,
May 11, 2011, 12:08:54 AM5/11/11
to amd-implement


> I believe Ben's case should be solved by having his 'array' to just
> delegate to dojo/array:
>
> //array.js:
> define('[dojo/array'], function (array){
>     return array;
>
> });

Sure, this works, but is a lot more work than using a paths config
mapping.

Rawld Gill

unread,
May 11, 2011, 12:20:33 AM5/11/11
to amd-im...@googlegroups.com
On Tuesday 10 May 2011 19:35:45 Ben Hockey wrote:
[big snip]

>
> i can see now (based on the build output of all things!) that james you
> probably intended to resolve ids relative to the module name but in reality
> the implementation resolves ids relative to the url. the question is now,
> do you fix this "bug"?

It's important to remember that all of the requirejs tests work correctly
either way [I'm pretty sure about this]. I know James if very good about
adding a test case for each of these kinds of decisions. In other words, this
hasn't been a big issue until now.

I'd even argue that James hasn't committed--certainly not strongly by
specifying a test to demonstrate correctness of a particular algorithm--to
either option so far. If you buy that argument, then nobody should complain
about clarifying an undecided area.

Lastly, the *great* part of this mailing list is that we have several
implementers. I'm committed to making dojo and bdLoad compat with RequireJS.
As long as we make rational, justifiable decisions, it seems we have enough
critical mass that others will follow.

Best,
Rawld

Rawld Gill

unread,
May 11, 2011, 1:36:09 AM5/11/11
to amd-im...@googlegroups.com
On Tuesday 10 May 2011 21:05:14 unscriptable wrote:
> > "When you specify a relative module id, it is relative to the reference
> > module's id. No further mapping of the module id is applied"
>
> I can't accept this rule unless somebody gives me a clear use case
> explaining why "no further mapping" is a bad idea. I don't get it.
> What does it break exactly? Why can't we map the module id?

Before I respond. Can you confirm that you want to further map the module id
to another module id? Or are you really asking about transforming the module
id to a url?

>
> > But if the mapped version is located someplace else and the loader tries
> > to resolve "./kernel" with respect to the somplace else URL, then the
> > system will fail.
>
> Of course this should fail. Why would anybody separate a module from
> its dependencies? This makes no sense to me.

Well, that's exactly how requirejs works today.

Again, the question is...is "./kernel" resolved with respect to the reference
module id or module url?

You are making the assumption that array is the only module that depends on
kernel (or, put another way, that kernel only serves array). This may not be
the case. If you are trying to map/replace a single module in a hierarchy,
this is not the case.

If you are trying to map a hierarchy, that's a package. Just point the package
location to a different place if you want a different package.

A package is not about lib and main...those are side issues. A package is a
hierarchy of interdependent modules.

>
>
> In a truly interoperable, modular world, this should work:
>
> // folder structure:
> js/
> some/
> hawt/
> array.js
> kernel.js
>
> // config:
> require({
> baseUrl: "js",
> paths: {
> "array": "some/hawt/array"
> }
> });


I consider this to be a defective configuration because you're mapping a top-
level name (array) into a module in the middle of a package. hawt is a package
because it's clearly a tree of interdependent modules.


>
> // "main" module:
> require(["array"], function (array) {
> // use my standardized array abstraction in here
> // notice in my dep list I am not referring to some/hawt/array
> // or dojo/array or yui/array etc.
> });
>
> // js/some/hawt/array.js
> define(["./kernel"], function (kernel) {
> // use kernel and return a standardized array abstraction
> // (like dojo does already)
> });
>
> If "No further mapping of the module id is applied", then the above
> can't work because the loader will look for kernel.js at "js/
> kernel.js" (no mapping) when it is actually located at "js/some/hawt/
> kernel.js". If we apply the existing mapping rules (as found in
> RequireJS 0.24 and curl.js 0.4.3), kernel.js will be found at "js/some/
> hawt/kernel.js" where we would expect it.

No, I wouldn't expect it...but I wouldn't give the defective config either.
Also, dojo, one of the original Javascript module systems wouldn't expect
that. More below.

>
> Tomorrow, when I discover I want to start using dojo instead of the
> some/hawt framework, I can change my paths config:
>
> require({ paths: { "array": "dojo/array" }, baseUrl: "js" });
>
> At JSCONF 2011, I talked about being able to write your code to a
> standardized API using this straightforward method of mapping
> standardized abstractions (such as "dojo/array" or "dojo/Sizzle") into
> implementation-agnostic names (e.g. "array" or "querySelectorAll") --
> and even quickly demoed it. People got very, very excited by this.
> Rebecca also showed a code example that exemplified this and advocated
> for it.

OK. But if your standardized abstraction requires a set of interdependent
modules, that's a package. If you want to use a single name (like array), then
that's the purpose of the package main config variable...

change your config to...

require({
baseUrl: "js",
packages:[{
name:"array",
main:"./array",
location:"some/hawt"
}]
})


> This is what most developers want from modules -- well, the developers
> that aren't zealots or fanboys for a single framework want this.

Everybody wants a good module system. Module systems are hard to get right and
have trade-offs. Any module system that I've had to use over the years has
areas that people--smart people, not just zealots--strongly disagree with.
Also, conference people getting excited in not evidence of good design.


>
> And frankly, I still don't know what the problem is. I have yet to
> see a use case that forces this: "No further mapping of the module id
> is applied". Every use case or code snippet I've seen in this thread
> so far works perfectly fine in curl.js.
>
> What isn't working today? Why do we need these new rules that break
> current, desirable functionality?


Its an established cannon of computer science that good design places a high
value on conceptual integrity. You are advocating a system where some kinds of
ids in deps vectors (relative ids) don't really specify a module id, but
rather a url, while others (absolute ids) don't specify a url, but rather a
module id. To me, that lacks conceptual integrity. Heck, it's possible that
some loader/systems may never resolve to URLs...then what's the rule?

And, tell me, what is the module id of "./kernel" with respect to the module
array. Give me a rule/process to compute it. So far, with option 2, I've only
seen a rule to find it's URL.

I advocate:

* ids--all ids--in deps vectors are module ids
* the purpose of a loader is to map a module id to a value

This, to my eye, has conceptual integrity.

Let me solve the problem you gave above and point out a few things along the
way.

The "hawt" tree clearly indicates a set of modules that

* are interdependent
* publish some public, perhaps interchangeable, API

That's a package, not a module. I would name the package "hawt" and configure
accordingly.

require({
baseUrl:"js",
packages:[{
name:"hawt",
location:"some/hawt"
}]
});

array.js is defined as you described above.

To use it, you require(["hawt/array"], //...

If you hate saying "hawt/array" and just want to say "array", then use the
main config value as I showed above.


Everything is peachy until the library "cool" is published which is way, way
better than hawt. But, you've got 300 source files that specify hawt...don't
want to change that. Well, you don't have too...just change the config and
you're good to go.

require({
baseUrl:"js",
packages:[{
name:"hawt",
location:"some/cool"
}]
});

Life is good again. But then a client needs an ever so slight extension to the
array impl in the cool lib. What's worse, the client is using--and wants to
continue to use--the cool library from a cdn that you can't touch. What you
need to do is somehow just replace one resource in cool library's hierarchy.
No problem, copy cool/array.js, make the changes, put it on your server and
then change the config to...


require({
baseUrl:"js",
packages:[{
name:"hawt",
location:"some/cool"
}],
paths:{
"hawt/array":"path/to/my/server"
}
});

Notice that the replacement will *still use the rest of cool's hierarchy*
without any further mapping.

Now, of course you could solve this if you use the "relative ids are with
respect to the reference module's url" option, but you'd have to rewrite and
then map the replacement array's dependency vector.

> What isn't working today? Why do we need these new rules that break
> current, desirable functionality?
>
> Please help me understand.

Actually, nothing isn't working today. Ben has brought up a spec ambiguity and
we're trying to agree on a resolution. Further, for most use cases, both
option 1 and option 2 end up having the same effect (though option 2 does not
specify how what the module id is of the resolved relative module)

But, this is important to decide. It is really foundational about what a
module loader is...does it provide a map from module id to module value with
machinery to instantiate modules as demanded by module id? If we say that URLs
can be specified in deps vectors (and that appears to be what you're
advocating) then I don't understand the abstraction we're trying to model.

Best,
Rawld

Rawld Gill

unread,
May 11, 2011, 1:38:46 AM5/11/11
to amd-im...@googlegroups.com

Depending on what your trying to do (map a module or a tree), each of option 1
or 2 is more or less work. It also depends on how you configure (as per my
other response).

To me, it's not about more/less work, but rather it's a question about what is
the abstraction we are modeling.

James Burke

unread,
May 11, 2011, 5:20:57 PM5/11/11
to amd-im...@googlegroups.com

Right, to echo Rawld's point, this is an edge-case issue, for most
things it all works no matter what option you choose. However, this
edge case has highlighted how we think of module IDs and how they are
mapped to URLs. And I believe relative IDs are relative module IDs not
relative path constructs.

As to the case that breaks with "map relative to URL":

If dojo/array.js is:

define(['./kernel'], function (){})

In the case where you wanted to replace "dojo/kernel" with a different
implementation, you will not be able to do this with paths: config --
dojo/array.js will always use dojo/kernel.js, it will not look up
"dojo/kernel" for a path mapping and then generate the URL, it will
just use dojo/kernel.js. You would have to manually overwrite
dojo/kernel.js with a new file to replace the implementation. This
breaks one of the fundamental use cases for paths: configurations.

John, Ben: how does that sound? I know it places more work on making
adapters, but not being able to paths: map modules that are referenced
via relative ID seem to be a bigger loss. Not using module IDs means
the mapping rules are not applied uniformly and will cause confusion
-- it will result in tricky to track down runtime errors ("why isn't
dojo/array using my replacement for dojo/kernel, but this other module
does?").

If that sounds OK, the actions items I would like to do for requirejs:

* remove "lib" support for packages.
* make sure relative module IDs are resolved relative to the parent module ID.
* Create at least one test that confirms the change.
* This should also fix any weird build output that Ben discovered, but
if not, make sure that uses module IDs too.

James

Miller Medeiros

unread,
May 11, 2011, 6:26:13 PM5/11/11
to amd-im...@googlegroups.com

On May 11, 2011, at 6:20 PM, James Burke wrote:

> John, Ben: how does that sound? I know it places more work on making
> adapters, but not being able to paths: map modules that are referenced
> via relative ID seem to be a bigger loss. Not using module IDs means
> the mapping rules are not applied uniformly and will cause confusion
> -- it will result in tricky to track down runtime errors ("why isn't
> dojo/array using my replacement for dojo/kernel, but this other module
> does?").
>

> * make sure relative module IDs are resolved relative to the parent module ID.

is this the desired result?

require({
paths: {
"dojo/kernel" : "other_lib/kernel"
}
});

"dojo/kernel" -> "other_lib/kernel"
"foo/../dojo/kernel" -> "other_lib/kernel"
"dojo/./kernel" -> "other_lib/kernel"
"dojo/bar/../kernel" -> "other_lib/kernel"
"dojo/kernel/more" -> "other_lib/kernel/more" [?]

I like the idea of always mapping relative to parent module IDs but I think the last option "dojo/kernel/more" shouldn't be mapped since it doesn't match the "whole path ID", maybe add an option to add wildcards to paths IDs like "dojo/kernel/*" that way anything inside "dojo/kernel" would be mapped to "other_lib/kernel" and last option would make total sense.. it could be an easy way to replace a whole library by simply using adapters.

James Burke

unread,
May 11, 2011, 6:48:52 PM5/11/11
to amd-im...@googlegroups.com
On Wed, May 11, 2011 at 3:26 PM, Miller Medeiros
<lis...@millermedeiros.com> wrote:
> is this the desired result?
>
> require({
>  paths: {
>    "dojo/kernel" : "other_lib/kernel"
>  }
> });
>
> "dojo/kernel" -> "other_lib/kernel"
yes

> "foo/../dojo/kernel" -> "other_lib/kernel"
This would not match since the ID is foo/dojo/kernel, path matches
start from the beginning of the string.

> "dojo/./kernel" -> "other_lib/kernel"
yes

> "dojo/bar/../kernel" -> "other_lib/kernel"
Would not match, same reason as foo/dojo/kernel

> "dojo/kernel/more" -> "other_lib/kernel/more" [?]

Would match the paths config.

> I like the idea of always mapping relative to parent module IDs but I think the last option "dojo/kernel/more" shouldn't be mapped since it doesn't match the "whole path ID", maybe add an option to add wildcards to paths IDs like "dojo/kernel/*" that way anything inside "dojo/kernel" would be mapped to "other_lib/kernel" and last option would make total sense.. it could be an easy way to replace a whole library by simply using adapters.

I have thought about supporting something like:

require({
paths: {
//Only applied if module name is exactly 'dojo/kernel'
'dojo/kernel': 'some/thing',
//Does not match 'dojo/kernel', but does match 'dojo/kernel/util'
'dojo/kernel/': 'other/dir'
}
});

So the presence of the slash on the end would signify only matching a
"directory name". I was preferring to not use a wildcard, since the
ending slash should be enough, but not sure if it is too subtle, and
not sure this is important enough to support vs. causing confusion on
typos.

James

Dustin Machi

unread,
May 11, 2011, 7:11:36 PM5/11/11
to amd-im...@googlegroups.com
The key here could be a regex instead of being somewhat ambiguous. This gives pretty sophisticated mapping ability. The matched regex's can be replaced with the expression in the value. James mentioned that requirejs is working on the individual path components, but I would think this matching work could be done before the module id gets split up into components, and that should be fairly easy to make consistent across implementations.

Dustin

Ben Hockey

unread,
May 11, 2011, 7:42:36 PM5/11/11
to amd-im...@googlegroups.com
James,

foo/../bar is bar

Ben...



Sent from my Palm Pre on AT&T


C Snover

unread,
May 11, 2011, 8:14:14 PM5/11/11
to amd-implement
I feel like lack of concrete examples are clouding my understanding of
this topic, so I really appreciate having “if you configure it like x
and require y, you resolve z” illustrations like these. Thanks, Miller
and James.

To continue along that vein, and to help support my understanding of
what the proposal is here, if someone configures an AMD-compliant
loader like:

require({
baseUrl: 'foo/',
paths: {
i18n: 'dojo/i18n',
'my/nls/app': 'other/nls/app',
'my/template': 'other/template',
'my/template.html': 'other/template2.html'
}
})

What will 'i18n' in require([ 'i18n!my/nls/app' ]) resolve to (both
module ID and path)?

What will dojo/i18n’s relative dependencies (like './main') resolve to
(both module ID and path)?

What will 'my/nls/app' resolve to in this example (both module ID and
path)?

What will 'dojo/text!my/template.html' resolve to in this example
(both module ID and path)?

What will 'dojo/text!./my/template.html' resolve to in this example
(both module ID and path)?

What will 'dojo/text!../my/template.html' resolve to in this example
(both module ID and path)?

I don’t mean to imply by any means that all of these behaviours should
definitely be defined by the spec (since as far as I am aware, any
data after the exclamation point is arbitrary and plugin-specific),
but I think it would good to be able to 1. explicitly define what, if
anything, is *outside* the spec in terms of resolving paths, and 2.
make sure that (if nothing else) there is some sort of de facto
agreement about how AMD plugins ought to handle common path/module
resolution cases, since these examples are all areas in which I’ve
experienced some wildly contradictory behaviour lately in existing
implementations.

Cheers,

Ben Hockey

unread,
May 11, 2011, 8:40:50 PM5/11/11
to amd-im...@googlegroups.com
there's probably just as much weight in the argument that addresses the case that breaks with "map relative to name/ID":

using the same dojo/array, i can't use it as a top-level module.  this breaks another fundamental use case for paths: configurations and that is to abstract my declared dependencies (declare a dependency on "array") and dynamically map if to different modules at different times.

personally i'm at a bit of a stalemate within myself.  i can see myself wanting to do both of these things at some point and i don't know which one i want to give up.  i came into this discussion with the abstract dependencies use case in mind but i could just as easily want to replace dojo/kernel at some point.
 

John, Ben: how does that sound? I know it places more work on making
adapters, but not being able to paths: map modules that are referenced
via relative ID seem to be a bigger loss. Not using module IDs means
the mapping rules are not applied uniformly and will cause confusion
-- it will result in tricky to track down runtime errors ("why isn't
dojo/array using my replacement for dojo/kernel, but this other module
does?").

i can't decide which is a bigger loss but i can see that the one with the simplest workaround is what you're suggesting - ie it's easier to make an adapter than to replace a physical file in a potentially external location.
 

If that sounds OK, the actions items I would like to do for requirejs:

* remove "lib" support for packages.

+ 1 - long term i think i'm ok with it.  i know the short term transition might hurt a little but i don't think anything is a blocker.

* make sure relative module IDs are resolved relative to the parent module ID.

+0 - ultimately, i don't have much of a dog in this fight (i'm not an amd loader implementer).  i respect all of you who are loader implementers and i trust that as long as there is enough agreement between you for the decision to reach critical mass then i'm sure that the result will provide a platform that i can manage to work with.

* Create at least one test that confirms the change.

+1 - feel free to use any of my code at http://neonstalwart.dojotoolkit.org/amd/relative/src/ if you'd like

* This should also fix any weird build output that Ben discovered, but
if not, make sure that uses module IDs too.

+1 - i think the build output will be ok once the modules resolve as you intended.  the build output was using the ids that match what you're proposing but it was including the modules that would be used if resolution was based on urls.  i would assume that this will probably fix itself once you get the loader to resolve based on module id.

James

Ben Hockey

unread,
May 11, 2011, 9:14:17 PM5/11/11
to amd-im...@googlegroups.com
i'll provide an answer to the best of my knowledge based on resolving relative ids as being relative to the module id.  i'm open to correction of course and it would help if someone would confirm that i'm right as well if that's the case.


On Wednesday, May 11, 2011 8:14:14 PM UTC-4, C Snover wrote:
I feel like lack of concrete examples are clouding my understanding of
this topic, so I really appreciate having “if you configure it like x
and require y, you resolve z” illustrations like these. Thanks, Miller
and James.

To continue along that vein, and to help support my understanding of
what the proposal is here, if someone configures an AMD-compliant
loader like:

require({
  baseUrl: 'foo/',
  paths: {
    i18n: 'dojo/i18n',
    'my/nls/app': 'other/nls/app',
    'my/template': 'other/template',
    'my/template.html': 'other/template2.html'
  }
})

What will 'i18n' in require([ 'i18n!my/nls/app' ]) resolve to (both
module ID and path)?
i18n has an entry in the paths config so it will map to baseUrl + value from path config
module id: i18n
path: foo/dojo/i18n.js
 

What will dojo/i18n’s relative dependencies (like './main') resolve to
(both module ID and path)?
dojo/i18n was retrieved via a module id of 'i18n' so this will be our point of reference for resolving relative module ids
the module id for ./main will resolve to <relative module id> + ../ + ./main
    -> i18n + ../ + ./main
    -> main (let me know if i skipped too many steps at once for you to follow this)
for the module id main there is no path config at play here so the path is just baseUrl + module id
so... from i18n ./main is
module id: main
path: foo/main.js
 

What will 'my/nls/app' resolve to in this example (both module ID and
path)?
my/nls/app is the module id and it has an entry in the path config so the value in the map will be used in combination with the baseUrl
module id: my/nls/app
path: foo/other/nls/app.js
 

What will 'dojo/text!my/template.html' resolve to in this example
(both module ID and path)?
as you mention below, everything after the ! is up to the plugin to resolve.  the plugin has require.toUrl at its disposal so we can discuss what would happen if require.toUrl was used because the behavior of require.toUrl for a plugin has been specified.  see http://wiki.commonjs.org/wiki/Modules/LoaderPlugin#require.toUrl - however i'm unclear if require.toUrl('my/template.html') should use "my/template" as the module id or "my/template.html" and i see that you've loaded your example to highlight this ambiguity.

first, the plugin with module id dojo/text would need to be resolved
module id: dojo/text
path: foo/dojo/text.js

next, if dojo/text calls require.toUrl with "my/template.html" then i'm uncertain about the outcome.  the 2 options are
module id: my/template.html
path returned to the plugin: foo/other/template2

module id: my/template
path returned to the plugin: foo/other/template
 

What will 'dojo/text!./my/template.html' resolve to in this example
(both module ID and path)?
i'd like to provide an answer but this is impossible to say what will happen without knowing which module declared this relative dependency.  i can say what should *not* happen - the relative id will not be relative to the plugin but rather it will be relative to the module that declared this dependency.  without knowing the point of reference we can't figure out module id or path.
 

What will 'dojo/text!../my/template.html' resolve to in this example
(both module ID and path)?
as above
 

I don’t mean to imply by any means that all of these behaviours should
definitely be defined by the spec (since as far as I am aware, any
data after the exclamation point is arbitrary and plugin-specific),
but I think it would good to be able to 1. explicitly define what, if
anything, is *outside* the spec in terms of resolving paths, and 2.
make sure that (if nothing else) there is some sort of de facto
agreement about how AMD plugins ought to handle common path/module
resolution cases, since these examples are all areas in which I’ve
experienced some wildly contradictory behaviour lately in existing
implementations.

Cheers,

i believe that you've had the same shock that i've had recently when trying to load code with dojo's loader that was previously loaded with the requirejs loader.  i concur that "wildly contradictory behaviour" describes what i observed.  

here's the executive summary of what's happened so far:
it seems that james is proposing that requirejs will change to match dojo's behavior.

the results i've given you in the examples above are based on my understanding of this behavior.

ben...

unscriptable

unread,
May 11, 2011, 10:25:56 PM5/11/11
to amd-implement


On May 11, 1:36 am, Rawld Gill <rg...@altoviso.com> wrote:
> On Tuesday 10 May 2011 21:05:14 unscriptable wrote:
>
> > > "When you specify a relative module id, it is relative to the reference
> > > module's id.  No further mapping of the module id is applied"
>
> > I can't accept this rule unless somebody gives me a clear use case
> > explaining why "no further mapping" is a bad idea.  I don't get it.
> > What does it break exactly?  Why can't we map the module id?
>
> Before I respond. Can you confirm that you want to further map the module id
> to another module id? Or are you really asking about transforming the module
> id to a url?

I want to map the module id to another module id. To me, the paths
config parameter's purpose is to alias module ids. The concept that
paths are just a way to map module ids to urls is not a universal
given. In fact, out in the wild, it's being used to serve both
purposes.

I can think of two valid use cases for mapping module ids off the top
of my head:

1) To shorten module ids when your third-party libraries are verbose:

"dojo" <=> "third-party/dojo-1.6.0/dojo"

2) To remove explicit library names from common, standardized modules:

"array" <=> "dojo/_base/array" // as the array module is named in dojo
1.6


I can think of one use case for mapping a module id to a url:

1) To map to a remote location:

"mylib" <=> "http://fastcdn.com/libs/mylib"


Both of these scenarios need to work. No, creating an abstraction
layer as a work-around for mapping module ids is not an option.



>
>
>
> > > But if the mapped version is located someplace else and the loader tries
> > > to resolve "./kernel" with respect to the somplace else URL, then the
> > > system will fail.
>
> > Of course this should fail.  Why would anybody separate a module from
> > its dependencies?  This makes no sense to me.
>
> Well, that's exactly how requirejs works today.

I disagree. This is how the dojo loader behaves in the scenario I had
painted. We must be miscommunicating about this point.


>
> Again, the question is...is "./kernel" resolved with respect to the reference
> module id or module url?

No. There is a middle ground: module ids may be mapped / aliased.

>
> You are making the assumption that array is the only module that depends on
> kernel (or, put another way, that kernel only serves array).

Again, no. See 2 comments below.

> This may not be
> the case. If you are trying to map/replace a single module in a hierarchy,
> this is not the case.

This is not the use case I am describing. This use case doesn't make
sense. If the module is defective, fix it.

>
> If you are trying to map a hierarchy, that's a package. Just point the package
> location to a different place if you want a different package.
>
> A package is not about lib and main...those are side issues. A package is a
> hierarchy of interdependent modules.
>
> > In a truly interoperable, modular world, this should work:
>
> > // folder structure:
> > js/
> >     some/
> >         hawt/
> >             array.js
> >             kernel.js
>
> > // config:
> > require({
> >     baseUrl: "js",
> >     paths: {
> >         "array": "some/hawt/array"
> >     }
> > });
>
> I consider this to be a defective configuration because you're mapping a top-
> level name (array) into a module in the middle of a package. hawt is a package
> because it's clearly a tree of interdependent modules.

Stepping back for a bit...

Frameworks, such as dojo, may have public and private modules. The
collection of public modules forms the public API of the package.

In an interoperable world, public modules must be able to be used
without the burden of the entire framework, but also must be able to
have shared, private dependencies. (dojo/array having a dojo/kernel
dependency is the example we've been using.)

If you don't agree that dojo/array should be able to be used without
the rest of the dojo package (i.e. it's a "defective configuration"),
then there's a bigger problem to discuss since the dojo package (as
implied by package.json) is the entire dojo framework. You may
believe that a developer should embrace dojo as a whole, and maybe
there is a lot of merit in doing so.

However, I believe the developer should decide whether she/he wants
the convenience of using and entire framework or cherry-picking the
best modules. IMHO, she/he should be able to do this in an abstract
way that doesn't require explicit package names (concrete dependencies
infiltrating the code) or writing an abstraction layer (lots of work).

As it stands right now, the only way to do this with dojo 1.6/
RequireJS is to alias module ids ("array" <=> "dojo/_base/array"). If
you remove the ability to alias module ids in the dojo 1.7 loader,
then we're stuck writing an abstraction layer which sucks.
You're proposed changes will break our valid use cases. Clearly,
there must be a way to resolve the edge cases and not break our
existing code. Does it make sense to separate the concepts of module
id aliasing and module url mapping?

-- John


C Snover

unread,
May 11, 2011, 10:39:50 PM5/11/11
to amd-implement
Replies inline

On May 11, 8:14 pm, Ben Hockey <neonstalw...@gmail.com> wrote:
> > What will dojo/i18n’s relative dependencies (like './main') resolve to
> > (both module ID and path)?
>
> dojo/i18n was retrieved via a module id of 'i18n' so this will be our point
> of reference for resolving relative module ids
> the module id for ./main will resolve to <relative module id> + ../ + ./main
>     -> i18n + ../ + ./main
>     -> main (let me know if i skipped too many steps at once for you to
> follow this)
> for the module id main there is no path config at play here so the path is
> just baseUrl + module id
> so... from i18n ./main is
> module id: main
> path: foo/main.js

Hmm. Alright, so, this makes sense in the case where you’re replacing
some functionality with some other new functionality with its own set
of relative dependencies, but it then seems to become impossible to
provide simple aliases which would be desirable for easily swapping
common/interoperable functionality like i18n/text plugins. I believe
this “swappability” is something that is desired as well, see e.g.
<http://bugs.dojotoolkit.org/ticket/12672#comment:124>. Right now, as
far as I can tell, there’s no possible way to alias 'dojo/i18n' or
'dojo/text' directly to 'i18n' and 'text' module IDs because they’ll
try to load dependencies relative to the wrong path and fail. If
interchangeable module implementations or enabling lazier typing are
considered noble goals, it would probably be good to add some option
to do this.

> > What will 'dojo/text!./my/template.html' resolve to in this example
> > (both module ID and path)?
>
> i'd like to provide an answer but this is impossible to say what will happen
> without knowing which module declared this relative dependency. i can say
> what should *not* happen - the relative id will not be relative to the
> plugin but rather it will be relative to the module that declared this
> dependency.

I imagine you’ve answered my question anyway, but just for the sake of
being thorough, if you would be so kind, assume that these are being
required by 'my/app' and provide some examples of what you think the
module ID and path resolution should be. :)

Also, this brings up another interesting/annoying point, which is that
one could do something like this (naïve example here, but you
hopefully get the point):

define(…, function () {
return {
getBar: function (callback) {
require([ 'text!../my/bar.html' ], callback);
}
};
});

…at which point the required relative file path would not have any
association to a module ID except for the plugin. Which is icky, but
also something that someone will inevitably try to do, so defining
what should happen in this case might be important. In any case, I
think I’ve observed three different ways this has been resolved at
various points in time: 1. relative to baseUrl, 2. relative to the
path of the loader script, and 3. relative to the path of the plugin.
Even if this particular edge case isn’t defined, having a consistent
decision mechanism for resolving file paths when the dependent module
ID *is* known would seem important to interop and not having to
remember 3-4 different resolution methods that change depending upon
the plugin author.

Regards,

Miller Medeiros

unread,
May 11, 2011, 10:50:00 PM5/11/11
to amd-im...@googlegroups.com
On May 11, 2011, at 10:14 PM, Ben Hockey wrote:

i'll provide an answer to the best of my knowledge based on resolving relative ids as being relative to the module id.  i'm open to correction of course and it would help if someone would confirm that i'm right as well if that's the case.

On Wednesday, May 11, 2011 8:14:14 PM UTC-4, C Snover wrote:
I feel like lack of concrete examples are clouding my understanding of
this topic, so I really appreciate having “if you configure it like x
and require y, you resolve z” illustrations like these. Thanks, Miller
and James.

To continue along that vein, and to help support my understanding of
what the proposal is here, if someone configures an AMD-compliant
loader like:

require({
  baseUrl: 'foo/',
  paths: {
    i18n: 'dojo/i18n',
    'my/nls/app': 'other/nls/app',
    'my/template': 'other/template',
    'my/template.html': 'other/template2.html'
  }
})



What will 'dojo/text!my/template.html' resolve to in this example
(both module ID and path)?
as you mention below, everything after the ! is up to the plugin to resolve.  the plugin has require.toUrl at its disposal so we can discuss what would happen if require.toUrl was used because the behavior of require.toUrl for a plugin has been specified.  see http://wiki.commonjs.org/wiki/Modules/LoaderPlugin#require.toUrl - however i'm unclear if require.toUrl('my/template.html') should use "my/template" as the module id or "my/template.html" and i see that you've loaded your example to highlight this ambiguity.

first, the plugin with module id dojo/text would need to be resolved
module id: dojo/text
path: foo/dojo/text.js

next, if dojo/text calls require.toUrl with "my/template.html" then i'm uncertain about the outcome.  the 2 options are
module id: my/template.html
path returned to the plugin: foo/other/template2

I think this is the correct outcome, just missing the ".html" file extension on the path, should be: 'foo/other/template2.html'


 

What will 'dojo/text!./my/template.html' resolve to in this example
(both module ID and path)?
i'd like to provide an answer but this is impossible to say what will happen without knowing which module declared this relative dependency.  i can say what should *not* happen - the relative id will not be relative to the plugin but rather it will be relative to the module that declared this dependency.  without knowing the point of reference we can't figure out module id or path.
 

this is a tricky case... if called on the root folder it would be the same as 'my/template.html' if inside a folder called 'ipsum' it would be:
module id: ipsum/my/template.html  (note that module ID should get ID of parent module minus last path)
path: foo/ipsum/my/template.html

since ID resolve to a value that isn't present on the paths config note that path and module ID aren't replaced.



What will 'dojo/text!../my/template.html' resolve to in this example
(both module ID and path)?


if inside 'ipsum' it would act as 'my/template.html' (id = 'my/template.html', path = 'foo/other/template2.html') 

if inside 'ipsum/dolor' it would act as previous example (id = 'ipsum/my/template.html', path = 'foo/ipsum/my/template.html')



 

I don’t mean to imply by any means that all of these behaviours should
definitely be defined by the spec (since as far as I am aware, any
data after the exclamation point is arbitrary and plugin-specific),
but I think it would good to be able to 1. explicitly define what, if
anything, is *outside* the spec in terms of resolving paths, and 2.
make sure that (if nothing else) there is some sort of de facto
agreement about how AMD plugins ought to handle common path/module
resolution cases, since these examples are all areas in which I’ve
experienced some wildly contradictory behaviour lately in existing
implementations.

Cheers,

i believe that you've had the same shock that i've had recently when trying to load code with dojo's loader that was previously loaded with the requirejs loader.  i concur that "wildly contradictory behaviour" describes what i observed.  


I agree that what happens after the exclamation point should be plugin-specific but I also think it should have a default action like RequireJS does with normalize ( http://requirejs.org/docs/plugins.html#apinormalize )



# Based on the examples described on this e-mail the rules would be: #

rule for ID resolution:  
  1. normalize relative paths (identify parent module id and removes last path)
  2. <absolute-path-to-parent-folder>/<module-name>

rule for PATH resolution: 
 1a. if `id` in `paths` config:  <baseUrl>/<path>.js
 1b. else: <baseUrl>/<module-id>.js


Rawld Gill

unread,
May 12, 2011, 8:25:16 AM5/12/11
to amd-im...@googlegroups.com
On Wednesday 11 May 2011 19:25:56 unscriptable wrote:
> On May 11, 1:36 am, Rawld Gill <rg...@altoviso.com> wrote:
> > On Tuesday 10 May 2011 21:05:14 unscriptable wrote:
[snip]

>
> I want to map the module id to another module id. To me, the paths
> config parameter's purpose is to alias module ids. The concept that
> paths are just a way to map module ids to urls is not a universal
> given. In fact, out in the wild, it's being used to serve both
> purposes.

That's interesting. I never considered paths anything but a URL mapping
feature...I guess because of its name. But, I tend to agree, there is no
"authority" to say what paths is really supposed to be doing.


>
> I can think of two valid use cases for mapping module ids off the top
> of my head:
>
> 1) To shorten module ids when your third-party libraries are verbose:
>
> "dojo" <=> "third-party/dojo-1.6.0/dojo"

How is that different than the package.location config variable?

>
> 2) To remove explicit library names from common, standardized modules:
>
> "array" <=> "dojo/_base/array" // as the array module is named in dojo
> 1.6

Hmmm, to what standardization body are you referring? I'm not aware of any
"standardized modules".

>
>
> I can think of one use case for mapping a module id to a url:
>
> 1) To map to a remote location:
>
> "mylib" <=> "http://fastcdn.com/libs/mylib"
>
>
> Both of these scenarios need to work. No, creating an abstraction
> layer as a work-around for mapping module ids is not an option.

I'm not sure I see that mapping module ids isn't an abstraction layer...but
it's not worth a big discussion...let's just put this one on the "we may not
see it the same but it doesn't matter " stack.

>
> > > > But if the mapped version is located someplace else and the loader
> > > > tries to resolve "./kernel" with respect to the somplace else URL,
> > > > then the system will fail.
> > >
> > > Of course this should fail. Why would anybody separate a module from
> > > its dependencies? This makes no sense to me.
> >
> > Well, that's exactly how requirejs works today.
>
> I disagree. This is how the dojo loader behaves in the scenario I had
> painted. We must be miscommunicating about this point.

fwiw, I've implemented a has feature that lets the dojo loader do it either
way.

>
> > Again, the question is...is "./kernel" resolved with respect to the
> > reference module id or module url?
>
> No. There is a middle ground: module ids may be mapped / aliased.

OK. I agree with this...and have this feature in dojo. I just haven't talked
about it because we already are having a tough time with paths. It seems like
part of our (the group as a whole...not John and Rawld) problem is vocabular
and assumptions. I never considered paths a module id map...if that's what you
are proposing then my opinion changes.

[snip]


>
> Stepping back for a bit...
>
> Frameworks, such as dojo, may have public and private modules. The
> collection of public modules forms the public API of the package.
>
> In an interoperable world, public modules must be able to be used
> without the burden of the entire framework, but also must be able to
> have shared, private dependencies. (dojo/array having a dojo/kernel
> dependency is the example we've been using.)
>
> If you don't agree that dojo/array should be able to be used without
> the rest of the dojo package (i.e. it's a "defective configuration"),
> then there's a bigger problem to discuss since the dojo package (as
> implied by package.json) is the entire dojo framework. You may
> believe that a developer should embrace dojo as a whole, and maybe
> there is a lot of merit in doing so.
>
> However, I believe the developer should decide whether she/he wants
> the convenience of using and entire framework or cherry-picking the
> best modules. IMHO, she/he should be able to do this in an abstract
> way that doesn't require explicit package names (concrete dependencies
> infiltrating the code) or writing an abstraction layer (lots of work).
>
> As it stands right now, the only way to do this with dojo 1.6/
> RequireJS is to alias module ids ("array" <=> "dojo/_base/array"). If
> you remove the ability to alias module ids in the dojo 1.7 loader,
> then we're stuck writing an abstraction layer which sucks.

I understand and agree with your point. One of the issues is that a public
module, say dojo/array, may need to bring some private stuff along with it. So
we need a way to cherry pick dojo/array and have the private stuff come, but
not the rest of the public stuff. Is that what you're getting at?

[snip]


>
> You're proposed changes will break our valid use cases. Clearly,
> there must be a way to resolve the edge cases and not break our
> existing code. Does it make sense to separate the concepts of module
> id aliasing and module url mapping?

Though I'm not convinced of your assertion here, the idea that paths is a
module map, not a url map, changes the discussion significantly. I need to
think some more.

Best,
Rawld

James Burke

unread,
May 12, 2011, 6:52:07 PM5/12/11
to amd-im...@googlegroups.com
On Wed, May 11, 2011 at 4:42 PM, Ben Hockey <neonst...@gmail.com> wrote:
> On May 11, 2011 18:48, James Burke <jrb...@gmail.com> wrote:
> On Wed, May 11, 2011 at 3:26 PM, Miller Medeiros
> <lis...@millermedeiros.com> wrote:
>> "dojo/bar/../kernel" -> "other_lib/kernel"
> Would not match, same reason as foo/dojo/kernel

As Ben pointed out, this one would match, since dojo/bar/../kernel
resolves to dojo/kernel. Sorry, when I read it I only parsed a single
dot, not both dots.

James

James Burke

unread,
May 12, 2011, 11:50:41 PM5/12/11
to amd-im...@googlegroups.com
On Wed, May 11, 2011 at 7:25 PM, unscriptable <jo...@e-numera.com> wrote:
> You're proposed changes will break our valid use cases.  Clearly,
> there must be a way to resolve the edge cases and not break our
> existing code.  Does it make sense to separate the concepts of module
> id aliasing and module url mapping?

I'm not sure how this would work, allowing both kinds of mappings.
Maybe you could try to illustrate it, how you would apply the rules.

I am also unclear how you expect others to use your 'array' that is
mapped to 'dojo/array' -- it would require the end developer to set up
all the paths: mappings in their config to use it?

James

Ben Hockey

unread,
May 13, 2011, 10:08:11 AM5/13/11
to amd-im...@googlegroups.com

I am also unclear how you expect others to use your 'array' that is
mapped to 'dojo/array' -- it would require the end developer to set up
all the paths: mappings in their config to use it?

for my use case - yes.  it is my desire and intention for the end user to set up the paths mappings to point to their favorite "array" implementation.  conceptually, AMD is kind of one form of inversion of control (IOC) container that uses dependency injection (DI).  in my use case, my goal is to write modules that consume an abstract API.  i declare my dependency on that API as "array".  after that, it's up to the AMD loader / IOC container to inject the concrete implementation of that API into my module's factory.  this leaves the end user with the responsibility/privilege of configuring how to get the concrete implementations of the APIs that my modules will depend on.  (btw, i just read this http://briancavalier.com/presentations/wirejs-2011-jsconf/#0 AFTER writing this paragraph).  i am also working on an IOC container with DI (WIP - https://github.com/neonstalwart/twine) that builds on top of AMD and it seems john (and brian) and i have exactly the same concept in mind (build an IOC container that is independent of a specific toolkit) and i came to my conclusions independently of them which indicates to me that we might be on to something that's useful.

micro frameworks are a buzz right now and nearly all of them suffer from the same problem - they either have a dependency on a specific toolkit (usually jQuery) or they duplicate code (like array extras) that probably already exists in another part of the application.  ideally they should decouple from a specific toolkit and not repeat code that is going to be typically available already.  my solution to this is to use a module format that i believe will be widely accepted (AMD) and declare abstract dependencies (such as "array", "promise", "EventEmitter", etc) that will be up to the end user to fulfill.  i may even move away from top-level ids to avoid collisions so that rather than just "array" it might be "myName/array" but that's beside the point.  what i outline is here is what also motivates my proposal to have base modules in dojo return the API that they implement but that's also outside the scope of this conversation.

by trying to use minimal dependencies it's likely that modules from inside a collection of modules will be mapped to ids outside of the collection that those modules came from ("dojo/array" -> "array" or "myName/array").  if those modules (such as "array") have relative dependencies then those dependencies need to resolve to modules that the "array" module expects.  up until seeing dojo's loader, this was how things worked with AMD loaders.

the last piece of the equation is to try and define some standard APIs that toolkits can work towards providing so that the need for adapters can be minimized.  does anyone know of work being done to produce some common js APIs ;)

ben...

James Burke

unread,
May 13, 2011, 2:09:54 PM5/13/11
to amd-im...@googlegroups.com
On Fri, May 13, 2011 at 7:08 AM, Ben Hockey <neonst...@gmail.com> wrote:
> IOC container that is independent of a specific toolkit) and i came to my
> conclusions independently of them which indicates to me that we might be on
> to something that's useful.

This is a good point, being able to use an IOC approach. Although, it
means it only works for top-level IDs. You cannot use an IOC model
inside a set of modules (commonly referred to as a package). Example:

Take the dojo package. Ideally relative modules IDs and anonymous
modules are used everywhere to allow the whole package to
renamed/moved to other directory name, for example, "myName". By using
relative IDs internally and using anonymous modules, the renaming
works.

However, if you wanted to build special versions of dojo that
substituted dojo/kernel, it would not be possible using URL
resolution, since all the modules use './kernel' as a dependency. The
primary use case for this internal mapping I would expect may be to do
browser-targeted builds, or to monkey-patch a problem with a module.

So I wonder if the package renaming of "dojo" to "myName" that is
supported is enough to get the IOC benefits you talk about?

A counterpoint may be that module mapping inside a package is not
really needed, although I have used that in the past with Dojo, so not
sure if it holds up as a counterpoint. But I'm open to the possibility
it is not.

> the last piece of the equation is to try and define some standard APIs that
> toolkits can work towards providing so that the need for adapters can be
> minimized.  does anyone know of work being done to produce some common js
> APIs ;)

I would expect that if there were common APIs every made, they would
be placed under a "package" name, like "commonjs"? If dojo supported
them, then it would be a matter of mapping "commonjs" : "dojo".

James

unscriptable

unread,
May 13, 2011, 9:23:12 PM5/13/11
to amd-implement
On May 13, 10:08 am, Ben Hockey <neonstalw...@gmail.com> wrote:
> > I am also unclear how you expect others to use your 'array' that is
> > mapped to 'dojo/array' -- it would require the end developer to set up
> > all the paths: mappings in their config to use it?
>
> for my use case - yes.  it is my desire and intention for the end user to
> set up the paths mappings to point to their favorite "array" implementation.
>  conceptually, AMD is kind of one form of inversion of control (IOC)
> container that uses dependency injection (DI).  in my use case, my goal is
> to write modules that consume an abstract API.  i declare my dependency on
> that API as "array".  after that, it's up to the AMD loader / IOC container
> to inject the concrete implementation of that API into my module's factory.
>  this leaves the end user with the responsibility/privilege of configuring
> how to get the concrete implementations of the APIs that my modules will
> depend on.  (btw, i just read thishttp://briancavalier.com/presentations/wirejs-2011-jsconf/#0 AFTER writing
> this paragraph).  i am also working on an IOC container with DI (WIP -https://github.com/neonstalwart/twine) that builds on top of AMD and it
> seems john (and brian) and i have exactly the same concept in mind (build an
> IOC container that is independent of a specific toolkit) and i came to my
> conclusions independently of them which indicates to me that we might be on
> to something that's useful.

Yes! Being able to create a framework-agnostic ioc container is our
primary goal.

There are at least two other javascript ioc containers that we saw
last week at JSCONF. One of these is interested in AMD.

> the last piece of the equation is to try and define some standard APIs that
> toolkits can work towards providing so that the need for adapters can be
> minimized.  does anyone know of work being done to produce some common js
> APIs ;)

This is somewhat off-topic, but may be worth mentioning. We've
already got a very good head start on standard APIs. If you look at
the major micro-libs and frameworks, most of them (except jQuery, of
course) implement one of two APIs:

a) an abstraction/wrapper that mimics the javascript native API with
an additional initial context parameter. For instance:
dojo.forEach(contextArray, lambda, lambdaContext); // also: map,
every, some, etc.

which looks like the native Array.prototype.forEach (minus the
contextArray):
contextArray.forEach(lambda, lambdaContext);

underscore.js has _.forEach(contextArray, lambda, lambdaContext); //
also: map, every, some, etc.

b) an extension/shim to the native prototypes to make them look as if
they support javascript 1.8. Prototype does this as does FuseJS.

IMHO, either of these is a good approach (if used correctly).

I admit that in some cases we'd have to build adapters, either via aop
or abstraction layers, to handle the cases where frameworks stray from
one of these "standard APIs". however, in the ideal case, we wouldn't
have to build an abstraction over an abstraction -- especially when
the two abstraction layers are so similar.

Formalizing these "standard APIs" is the second major reason why I'd
like to keep the capability to alias "array" ==> "dojo/array".

-- J

unscriptable

unread,
May 13, 2011, 9:54:17 PM5/13/11
to amd-implement
On May 12, 8:25 am, Rawld Gill <rg...@altoviso.com> wrote:
> Hmmm, to what standardization body are you referring? I'm not aware of any
> "standardized modules".

There is no official standard, but I am hoping we can initiate one by
showing several existing implementations besides just dojo.
underscore is another such implementation.

> I understand and agree with your point. One of the issues is that a public
> module, say dojo/array, may need to bring some private stuff along with it. So
> we need a way to cherry pick dojo/array and have the private stuff come, but
> not the rest of the public stuff. Is that what you're getting at?

Yes.

> Though I'm not convinced of your assertion here, the idea that paths is a
> module map, not a url map, changes the discussion significantly. I need to
> think some more.

Thanks for pondering it.

It's true that we could build an abstraction layer in code, but I'd
really like to be able to use aliasing instead.

Regards,

-- John

James Burke

unread,
May 20, 2011, 12:44:35 AM5/20/11
to amd-im...@googlegroups.com
On Tue, May 10, 2011 at 1:25 PM, James Burke <jrb...@gmail.com> wrote:
> Options:
> ----------------
>
> 1) "Relative to module ID": resolve the relative module ID, './bar',
> relative to the reference ID, 'foo'. So, the relative module ID gets
> converted to 'foo/bar' and then the path/package configuration is used
> to resolve it.
>
> 2) "Relative to module URL": Resolve the relative ID './bar' relative
> to the reference ID's URL. So, in the simple case: URL prefix +
> 'foo.js' + './bar.js' (but of course cleaned up to be a real path).

I want to check in with everyone on this, and I would like to get it
resolved. I still favor resolving relative to module ID for the
following reasons:

1) It is still possible to provide the IOC/API mapping that
John/Brian/Ben bring up, as long as the mapping is restricted to a
"package" level mapping: So to use "dojo/array" as the implementation
for some common API, the common API needs to be scoped to a package.
So, it is possible to map "common/array" to "dojo/array" and things
still work by just mapping "common": "dojo".

2) It still allows single file mappings. So things like
"microlibraries" that just implement one small function without
complex dependencies can be aliased to other files.

3) It allows replacing the implementation of one module within a
package. Useful for building targeted builds of a package for
different environments/uses, or for monkey patching a single module in
a package.

Using URL mapping means:

Gives some more flexibility for #1 at the expense of not being able to
do #3. #2 works in both since it does not really involve relative IDs,
since microlibs probably would specify "top level" dependencies vs. a
relative one.

John/Brian/Ben: how does this sound? Any other feedback on the tradeoffs?

James

Rawld Gill

unread,
May 20, 2011, 4:09:23 AM5/20/11
to amd-im...@googlegroups.com
On Thursday 19 May 2011 21:44:35 James Burke wrote:
> On Tue, May 10, 2011 at 1:25 PM, James Burke <jrb...@gmail.com> wrote:
> > Options:
> > ----------------
> >
> > 1) "Relative to module ID": resolve the relative module ID, './bar',
> > relative to the reference ID, 'foo'. So, the relative module ID gets
> > converted to 'foo/bar' and then the path/package configuration is used
> > to resolve it.

I think "./bar" relative to "foo" is "bar", and further...

"./B" relative to "A" => "A/.././B" => B
"./C" relative to "A/B" => "A/B/.././C" => "A/C"
"../C" relative to "A/B" => "A/B/../../C" => "C"

Best,
Rawld

Ben Hockey

unread,
May 20, 2011, 9:53:33 AM5/20/11
to amd-im...@googlegroups.com
james,

first, i agree that this is an accurate summary of the pros/cons of both approaches so i feel that there's no need for further discussion to outline the use cases.  from the very start my ultimate desire has been for a resolution that brings consistency between loaders.  i argued in favor of resolution relative to the url but i can see that you and rawld are in agreement for resolution relative to the module id.

i would like to get this resolved quickly so that we can all move on and with that in mind i have no further feedback to add.  all i ask is that the loaders work consistently and i'll do my best to find a way to get my code to work with them.  

i haven't had time to look over the test suite in detail yet but i would like to see a test that is able to show if a loader has properly implemented relative resolution.  fyi, the tests in requirejs (as of last week) were not able to show which method of relative resolution a loader had implemented.  https://gist.github.com/982913 would be along the lines of what i think should be adequate (note: foo_bar_one.js and foo_bar_two.js should be foo/bar/one.js and foo/bar/two.js but gist doesn't allow that).  you would just need to add an assertion in main.js to ensure the right module was resolved.

ben...

James Burke

unread,
May 20, 2011, 1:31:19 PM5/20/11
to amd-im...@googlegroups.com

Right, sorry, I replied to the first message that had my very bad example.

James

James Burke

unread,
May 20, 2011, 1:36:57 PM5/20/11
to amd-im...@googlegroups.com

Definitely, if we get agreement on the resolution policy, I will add a
pull request for the amd-tests repo with a valid test.

Looking at the CommonJS Modules 1.1.1 page:
http://wiki.commonjs.org/wiki/Modules/1.1.1

it looks like they also specify in the Module Specifiers section:
"Relative identifiers are resolved relative to the identifier of the
module in which "require" is written and called. "

So for me that is more impetus to go with module ID resolution.

I would like to hear from John Hann to get his current view on it
given that some IOC types of mappings are possible, and stand-alone
microlib mapping work in either case.

James

ben hockey

unread,
May 20, 2011, 1:45:06 PM5/20/11
to amd-im...@googlegroups.com

> it looks like they also specify in the Module Specifiers section:
> "Relative identifiers are resolved relative to the identifier of the
> module in which "require" is written and called. "
>
haha... this is where rawld and i started. the spec doesn't actually
specify what is to be used as the module identifier and it would not be
outside the letter or intention of the spec to use the url as the module
identifier - its left to implementers to determine what will be used as
a module identifier. we have now come full circle and the spec is still
not going to be any more useful now than when we started :)

ben...

John Hann

unread,
May 25, 2011, 10:50:26 PM5/25/11
to amd-im...@googlegroups.com

I am fine with this. This makes sense, is predictable, seems to
handle many use cases, and is consistent with how I interpret the
arguably loose CommonJS Modules/1.1 spec.

However, I am also contemplating a feature in curl.js that allows Ben
-- as well as my team -- to create pseudo-packages. These look like
top-level packages even if they're really embedded deep in another
package.

Actually, I'm leaning towards removing the whole concept of "path
mappings" and implementing "package mappings" instead. I think
calling them "paths" is misleading if we take the approach we've
agreed to here. I was under the impression that "paths" allowed me to
map anything I wanted: packages, modules, plugins, resources. (Indeed
RequireJS works this way now, as does curl.js.)

What's really funny (to me) is that several of you have been thinking
just the opposite: remove the concept of packages in favor of paths,
IIUC. For example, there was talk to remove the "lib" property of
packages since it's really just a path. If you do this, how does
somebody specify that the A/B module is actually at A/lib/B or A1.6.1/
src/B ?

-- J

James Burke

unread,
May 25, 2011, 11:35:05 PM5/25/11
to amd-im...@googlegroups.com
On Wed, May 25, 2011 at 7:50 PM, John Hann <jo...@e-numera.com> wrote:
> However, I am also contemplating a feature in curl.js that allows Ben -- as
> well as my team -- to create pseudo-packages.  These look like top-level
> packages even if they're really embedded deep in another package.

To me, this just sounds like the paths mapping is just lower: so
instead of 'foo': 'fooPkg', it is 'foo': 'fooPkg/some/other/dir'. I am
interested in learning more of your approach though.

In the meantime, I'll be changing requirejs to do the relative module
resolution, and I'll work up a test for the amdjs-tests to exercise
that rule.

> Actually, I'm leaning towards removing the whole concept of "path mappings"
> and implementing "package mappings" instead.  I think calling them "paths"
> is misleading if we take the approach we've agreed to here.  I was under the
> impression that "paths" allowed me to map anything I wanted: packages,
> modules, plugins, resources.  (Indeed RequireJS works this way now, as does
> curl.js.)

Those mappings are still related to paths. The relative ID resolution
means that the ID is resolved to the parent ID then the paths mappings
are consulted to generate the final path.

I can see were "paths" could be misleading, maybe "mappings" is a
better one? Although that might cause some confusion with a
package.json idea of "mappings", which can point to zip/tar.gz files.
That said, I could support putting an alias for "paths" to be
"mappings" and then gradually deprecating "paths" in favor of
"mappings" if you want to coordinate on common configuration values.
I'm open to other descriptive names for this functionality.

> What's really funny (to me) is that several of you have been thinking just
> the opposite: remove the concept of packages in favor of paths, IIUC.  For
> example, there was talk to remove the "lib" property of packages since it's
> really just a path.  If you do this, how does somebody specify that the A/B
> module is actually at A/lib/B or A1.6.1/src/B ?

With requirejs "paths", I would expect a mapping of 'A':
'A1.6.1/src/B' and expect the "main" module to be at A1.6.1/src/B.js.
The packages config seems like too much configuration for very little
payoff, more configuration to burden an app user with. But I'm
interested to know if you see it another way and how it might work.

James

Rawld Gill

unread,
May 26, 2011, 7:16:18 PM5/26/11
to amd-im...@googlegroups.com
On Wednesday 25 May 2011 20:35:05 James Burke wrote:
> On Wed, May 25, 2011 at 7:50 PM, John Hann <jo...@e-numera.com> wrote:
> > However, I am also contemplating a feature in curl.js that allows Ben --
> > as well as my team -- to create pseudo-packages. These look like
> > top-level packages even if they're really embedded deep in another
> > package.
>
> To me, this just sounds like the paths mapping is just lower: so
> instead of 'foo': 'fooPkg', it is 'foo': 'fooPkg/some/other/dir'. I am
> interested in learning more of your approach though.

Agree.

>
> In the meantime, I'll be changing requirejs to do the relative module
> resolution, and I'll work up a test for the amdjs-tests to exercise
> that rule.

I will do the same in dojo and bdLoad.

>
> > Actually, I'm leaning towards removing the whole concept of "path
> > mappings" and implementing "package mappings" instead. I think calling
> > them "paths" is misleading if we take the approach we've agreed to here.
> > I was under the impression that "paths" allowed me to map anything I
> > wanted: packages, modules, plugins, resources. (Indeed RequireJS works
> > this way now, as does curl.js.)
>
> Those mappings are still related to paths. The relative ID resolution
> means that the ID is resolved to the parent ID then the paths mappings
> are consulted to generate the final path.

Agree.

>
> I can see were "paths" could be misleading, maybe "mappings" is a
> better one? Although that might cause some confusion with a
> package.json idea of "mappings", which can point to zip/tar.gz files.
> That said, I could support putting an alias for "paths" to be
> "mappings" and then gradually deprecating "paths" in favor of
> "mappings" if you want to coordinate on common configuration values.
> I'm open to other descriptive names for this functionality.
>
> > What's really funny (to me) is that several of you have been thinking
> > just the opposite: remove the concept of packages in favor of paths,
> > IIUC. For example, there was talk to remove the "lib" property of
> > packages since it's really just a path. If you do this, how does
> > somebody specify that the A/B module is actually at A/lib/B or
> > A1.6.1/src/B ?
>
> With requirejs "paths", I would expect a mapping of 'A':
> 'A1.6.1/src/B' and expect the "main" module to be at A1.6.1/src/B.js.
> The packages config seems like too much configuration for very little
> payoff, more configuration to burden an app user with. But I'm
> interested to know if you see it another way and how it might work.
>

I agree with John that there are some other mapping issues. But I'll hold you
all in suspense until I get time to do a good writeup on another thread.

I suggest we keep this thread restricted to "Relative module ID resolution".
And, iiuc, {RequireJS, curl, dojo, bdLoad} are now in agreement.

Best,
Rawld

James Burke

unread,
May 27, 2011, 6:03:43 PM5/27/11
to amd-im...@googlegroups.com
On Fri, May 20, 2011 at 6:53 AM, Ben Hockey <neonst...@gmail.com> wrote:
> i haven't had time to look over the test suite in detail yet but i would
> like to see a test that is able to show if a loader has properly implemented
> relative resolution.

I have pull request for the amdjs-tests repo that adds an explicit
test for this module resolution:
https://github.com/amdjs/amdjs-tests/pull/1

Unless there are objections or requests for more time to review, I
will likely merge this on Sunday.

James

Daniel Dotsenko

unread,
Dec 17, 2012, 9:20:55 PM12/17/12
to amd-im...@googlegroups.com
Re: Relative resource resolution in named define

With the idea of "relative IDs are resolved against container module ID" in mind in the following set up:

    /foo/
        bar.js contents:
            define('a/b/c',['./resource'], function (resource) { }); 
            define('e/f/g',['./resource'], function (resource) { });
            define('h/i/j',['./resource'], function (resource) { });

What should be the resolved module IDs for the resources in the above?

`foo/resource` for all?

or

'a/b/resource'
'e/f/resource'
'h/i/resource'

In other words, do you say a proper behavior is to tread an inception of a particular named define as start of a "module" and resolve the requirement array against the named ID, or treat the named define call as part of the `foo/bar` module and resolve that define's relative resources against `foo/bar`?

Thanks.

Daniel

On Tuesday, May 10, 2011 1:25:49 PM UTC-7, James Burke wrote:
This came up on the dojo-contributors list, full thread here:
http://thread.gmane.org/gmane.comp.web.dojo.devel/14580

but I will summarize and provide some background here for AMD folks
not familiar with path resolution of module IDs.

Background
----------------

For an AMD loader to work in the browser efficiently, it should only
do one URL lookup for the module. This is different from traditional
CommonJS Modules 1.1 spec loaders on the server, where they normally
use a require.paths array to search for modules.

AMD supports the CommonJS type of module IDs that look like
'some/module', which gets mapped to an URL using some configuration in
the loader.

This normally involves using a "baseUrl" and appending the module ID
plus '.js' to create the path. So, for 'some/module', the URL ends up
being baseUrl + 'some/module.js'.

There are a few AMD implementations, requirejs, curl, bdload/dojo
loader that allow mapping *part* of a module ID to a different path
than what would normally be used if just using the baseUrl.

For example, in requirejs, this configuration call would result in an
URL of baseUrl + 'thirdparty/special/some/module.js':

require({
    paths: {
       'some': 'thirdparty/special/some'
    }
});

Some loaders, like requirejs and dojo, support configuring the
location of a set of modules according to the CommonJS Packages spec's
use of 'main' and 'lib' properties:
http://requirejs.org/docs/api.html#packages

I now believe the 'main' and 'lib' properties of the CommonJS Packages
spec to be bad ideas. Node 0.4 does support the 'main' property.
However Node removed support for 'lib' in 0.4 because of the extra
path magic it required. I agree with this conclusion, as I believe it
contributes to this relative ID problem.

Summary of Issue:
----------------

If a module does a require('foo') and then foo.js has a
require('./bar'), how should './bar' be resolved to an URL?

Options:
----------------

1) "Relative to module ID": resolve the relative module ID, './bar',
relative to the reference ID, 'foo'. So, the relative module ID gets
converted to 'foo/bar' and then the path/package configuration is used
to resolve it.

2) "Relative to module URL": Resolve the relative ID './bar' relative


to the reference ID's URL. So, in the simple case: URL prefix +
'foo.js' + './bar.js' (but of course cleaned up to be a real path).

Problems:
----------------

There are many configurations where #1 and #2 result in the same
value, so it does not matter. However, there are two cases where it
makes a difference:

1) CommonJS Package's 'lib' directory is in play. In the example, if
'foo' is a CommonJS package, and its main: 'foo.js' and 'lib' is set
to 'scripts', then the paths get resolved to:

Option 1: foo/scripts/bar.js
Option 2: foo/bar.js

I suggest we follow Node's lead here and discourage AMD
implementations from implementing 'lib' support. This removes the
problem with this discrepancy, and makes implementation easier all
around.

2) paths: configuration. There could be this type of configuration set up:

require({
    paths: {
       'foo/bar': 'overrides/foo/bar'
    }
});

The resulting URLs:

Option 1: baseUrl + 'overrides/foo/bar.js'
Option 2: baseUrl + 'foo/bar.js'

I believe Option 1 is the desirable behavior, because this allows
proper overriding of modules. This is particularly useful if you are
building a toolkit, and you know that you can deploy a different
version of 'arrayExtras' depending on the deployment scenario (if only
webkit, then map 'arrayExtras' to an empty file). If option 2 is used,
you will *not* be able to override the path used if './arrayExtras' is
used.

Summary
----------------

Those were the issues I were able to come up with, but I may have
missed something so please point it out. However, based on this
information, I favor using Option 1, 'relative to module ID'.

This is actually a change for RequireJS, and I will update the code to
reflect the change if other implementers agree. As part of that
update, I would remove CommonJS Package 'lib' support too.

The original reason RequireJS supports Option 2 'relative to module
URL' was because of this issue:
https://github.com/jrburke/requirejs/issues/51

but I believe with the removal of 'lib' support issue #51 goes away.

James

James Burke

unread,
Dec 17, 2012, 9:28:41 PM12/17/12
to amd-im...@googlegroups.com
'./resource is resolved relative to the ID of the module requesting
it. So none of those are resolved relative to foo/bar.js, but the
'a/b/c', 'e/f/g', 'h/i/j' references respectively, since those are the
modules requesting the relative module.

James

John Hann

unread,
Dec 17, 2012, 10:57:05 PM12/17/12
to amd-im...@googlegroups.com
Yes:
'a/b/resource'
'e/f/resource'
'h/i/resource'
Reply all
Reply to author
Forward
0 new messages