package.json mappings

312 views
Skip to first unread message

Kris Zyp

unread,
Apr 9, 2010, 10:52:37 AM4/9/10
to comm...@googlegroups.com
I wanted to put forth the idea of defining a "mappings" property for
package.json. Currently, the package spec defines a "dependencies"
property indicating the needed packages. However, without a consistent
way to reference modules from other packages, this is basically useless.
An "installed" package is of no value if you don't know how to reference
the modules in it. We have had a lot of debates about how require could
be augmented to reference packages. Narwhal (and maybe some others)
install packages directly into the top level of the module id namespace,
and a require2 has been suggested for disambiguation. The require2
certainly does not have consensus, and requiring everyone to install
packages at the top level seems unreasonable and very prone to
conflicts. Various other schemes have been suggested for ids to
reference packages. I would suggest that instead, packages themselves
should get to decide how they will reference modules from other packages.

Another problem with the "dependencies" property is that it provides no
information on where to actually get the package.

Building on the concept that URLs are the foundational mechanism for
referencing modules, but recognizing that it is still important to
provide easy aliases to modules from other packages, I suggest a
"mappings" property for package.json that lists an array of mapping
hashes that define how an module ids are mapped to the a canonical
locator. The key properties in the mapping hash would be a "from" and
"to" property indicating how the ids are mapped. For example, we could
define:
"mappings": [
{"from": "package/perstore/", "to":
"http://github.com/kriszyp/perstore/raw/master/lib/"}
]

And this would guarantee that if I used a
require("package/perstore/model") in my package, that it would load the
module that has the canonical locator of
http://github.com/kriszyp/perstore/raw/master/lib/model.js. Now this
does not imply that all require impls need to do an HTTP download for
each require. The model.js (and its containing package) file may be
installed by a package manager, it could be downloaded on demand, it
could be cached, the mechanism is beyond the scope of the package.json
definition. But it does provide a clear way for a package to reference
and use a module in a reliable manner, regardless of how the module gets
installed or accessed.

This also does not preclude us from using packages/modules that were
written with the expectation of modules from other packages existing at
the top level. They can simply define a mapping like
{"from": "", "to":
"http://github.com/kriszyp/perstore/raw/master/lib/"},

One could also add support for have "named" packages without locators
(although without a "to", this lacks information about how to locate the
package):
{"from": "package/perstore", "package": "perstore"},

The mapping definition could also be extended to deal with another
challenge we face, cross-engine code forking. The reality of writing
cross-engine modules is that it involves forking into various different
based on the environment. We could provide conditional mappings for
dealing with engine-specific modules:
{"from": "http-client", "if": "node", "to": "node-http-client"},

The condition could be as simple as an engine, it could be a version
comparison, or maybe even support more complex feature detection based
expressions.

And we could further extend mapping with compilation information
{"from": "cs-modules", "extension": ".coffee", "compiler":
"coffee-script"}

These extra properties are probably aren't as important, but show how
this simple model could be used to solve a number of issues with
packages. The key properties would be "from" and "to". Also, I think it
may be reasonable to define "from" and "to" be regular expressions than
just strings to match.

--
Thanks,
Kris

Ash Berlin

unread,
Apr 9, 2010, 11:04:14 AM4/9/10
to comm...@googlegroups.com
On 9 Apr 2010, at 15:52, Kris Zyp wrote:
> I wanted to put forth the idea of defining a "mappings" property for
> package.json.


As a general thing (having no relation on the rest of your email) once we get onto a new version of packages.json I think having a meta-version field in the json to signify which version of the spec this package.json file relates to is a very good idea, even if things are 'backwards compatible'. This means that compliant readers of a package.json should ignore fields that aren't in the version - this would work around possible compat issues with some systems using fields that later come to mean something (subtly) different when standardized.

Oh and no 'meta-version' field means 'somewhere around Packages/1.0'

-ash

Christoph Dorn

unread,
Apr 9, 2010, 12:28:57 PM4/9/10
to comm...@googlegroups.com
Kris Zyp wrote:
> Building on the concept that URLs are the foundational mechanism for
> referencing modules, but recognizing that it is still important to
> provide easy aliases to modules from other packages, I suggest a
> "mappings" property for package.json that lists an array of mapping
> hashes that define how an module ids are mapped to the a canonical
> locator. The key properties in the mapping hash would be a "from" and
> "to" property indicating how the ids are mapped. For example, we could
> define:
> "mappings": [
> {"from": "package/perstore/", "to":
> "http://github.com/kriszyp/perstore/raw/master/lib/"}
> ]
>
> And this would guarantee that if I used a
> require("package/perstore/model") in my package, that it would load the

The mapping feature you describe would mean that

require("package/perstore/model")

from one package would not necessarily yield the same module as

require("package/perstore/model")

from another package.

This is counter-intuitive to me as IMO require() is designed to load
modules from one top-level namespace consistent across all system
packages in a program.


> module that has the canonical locator of
> http://github.com/kriszyp/perstore/raw/master/lib/model.js. Now this
> does not imply that all require impls need to do an HTTP download for
> each require. The model.js (and its containing package) file may be
> installed by a package manager, it could be downloaded on demand, it
> could be cached, the mechanism is beyond the scope of the package.json
> definition. But it does provide a clear way for a package to reference
> and use a module in a reliable manner, regardless of how the module gets
> installed or accessed.
>
> This also does not preclude us from using packages/modules that were
> written with the expectation of modules from other packages existing at
> the top level. They can simply define a mapping like
> {"from": "", "to":
> "http://github.com/kriszyp/perstore/raw/master/lib/"},

Based on [1]:

package.json ~ {
"dependencies": [
"narwhal",
"domain.net/path/to/package/1" // put package's modules onto
system namespace
],
"packages": {
"theirPackage": { // alias
"location": "http://domain.net/path/to/package/1.3.2.zip",
"api": "1"
}
}
}


> One could also add support for have "named" packages without locators
> (although without a "to", this lacks information about how to locate the
> package):
> {"from": "package/perstore", "package": "perstore"},
>
> The mapping definition could also be extended to deal with another
> challenge we face, cross-engine code forking. The reality of writing
> cross-engine modules is that it involves forking into various different
> based on the environment. We could provide conditional mappings for
> dealing with engine-specific modules:
> {"from": "http-client", "if": "node", "to": "node-http-client"},

Possible solution based on [1]:

package.json ~ {
"enginePackages": { // engine-specific packages
"theirPackage": { // alias
"node": { // if condition
// locator
}
}
}
}


> The condition could be as simple as an engine, it could be a version
> comparison, or maybe even support more complex feature detection based
> expressions.
>
> And we could further extend mapping with compilation information
> {"from": "cs-modules", "extension": ".coffee", "compiler":
> "coffee-script"}

I don't think this belongs in the dependency declaration. I see this as
the responsibility of the module being required or a more specialized
loader module.


> These extra properties are probably aren't as important, but show how
> this simple model could be used to solve a number of issues with
> packages. The key properties would be "from" and "to". Also, I think it
> may be reasonable to define "from" and "to" be regular expressions than
> just strings to match.

IMO we need require2 or alternative to make non-system packages
interoperable and loading modules from these intuitive.

Christoph

[1] - http://github.com/cadorn/pinf/blob/master/docs/Design/Foundation.md

Kris Zyp

unread,
Apr 9, 2010, 1:15:39 PM4/9/10
to comm...@googlegroups.com

On 4/9/2010 10:28 AM, Christoph Dorn wrote:
> Kris Zyp wrote:
>> Building on the concept that URLs are the foundational mechanism for
>> referencing modules, but recognizing that it is still important to
>> provide easy aliases to modules from other packages, I suggest a
>> "mappings" property for package.json that lists an array of mapping
>> hashes that define how an module ids are mapped to the a canonical
>> locator. The key properties in the mapping hash would be a "from" and
>> "to" property indicating how the ids are mapped. For example, we could
>> define:
>> "mappings": [
>> {"from": "package/perstore/", "to":
>> "http://github.com/kriszyp/perstore/raw/master/lib/"}
>> ]
>>
>> And this would guarantee that if I used a
>> require("package/perstore/model") in my package, that it would load the
>
> The mapping feature you describe would mean that
>
> require("package/perstore/model")
>
> from one package would not necessarily yield the same module as
>
> require("package/perstore/model")
>
> from another package.
>

Exactly. The notion that every package must have exactly the same view
of the top level namespace, even for aliases, is the cause of the
collision woes that we have been attempting to avert with require2 and
alternatives discussion (that have been pretty unfruitful, AFAICT). It
is important to note that these are aliases though, the canonical id/URI
of the modules can stay consistent across all packages.

This seems a lot more complicated than a simple mapping object, it is
still not clear how to the modules from theirPackage get referenced.
Hopefully not stuffed in the same top level namespace as everything else.

It doesn't seem that require2 is going to ever have broad support, and
there certainly are issues with it, like no counterpart in
require.ensure, require.define, and complications with browser loading
of modules from a server that require disambiguation. And putting
require2 as part of module packages module seems to couple
packages/modules to a package manager, which I wouldn't want.

--
Thanks,
Kris

Kris Kowal

unread,
Apr 9, 2010, 2:57:13 PM4/9/10
to comm...@googlegroups.com
On Fri, Apr 9, 2010 at 7:52 AM, Kris Zyp <kri...@gmail.com> wrote:
> I wanted to put forth the idea of defining a "mappings" property for
> package.json.

I think this is an intriguing idea, functionally similar to require2,
but without the API disruption. I think I would need to play with it.
However, I know Ihab well enough, I think, to know he will complain
about the name-space overlay and the hazard it presents: a
mis-configuration could silently effect a security hole. I am not
personally bothered by the idea, but I'd like to hear the proverbial
argument from the horse's mouth.

Kris Zyp, it would be good to see this proposal in a concrete form. I
think the necessary feature subset is an "id" to module URL mapping,
an "id" to package URL mapping, and something for cryptographically
verifying the content, taking a nod perhaps from Ihab's require2
proposal. I'm not sure how versioning fits. I would also like some
time to ponder how to implement this.

Kris Kowal

Irakli Gozalishvili

unread,
Apr 9, 2010, 2:57:52 PM4/9/10
to comm...@googlegroups.com
Kris I do support you on this one even though I do think Christoph has a point - It's not predictable from a module source what require("foo/bar") will return. If mapping will be optional and we'll have a default like: { from:"."  to:"packages/<packageName>/" } I'm up for it.

--
Irakli Gozalishvili
Web: http://rfobic.wordpress.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands



--
You received this message because you are subscribed to the Google Groups "CommonJS" group.
To post to this group, send email to comm...@googlegroups.com.
To unsubscribe from this group, send email to commonjs+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/commonjs?hl=en.


Christoph Dorn

unread,
Apr 9, 2010, 3:01:43 PM4/9/10
to comm...@googlegroups.com

I agree that the discussions thus far have not been very fruitful. I
would not conclude however that it is already a lost cause.

Based on my point of view I do not agree with aliasing packages into
require(). It necessitates that every implementation provides a wrapped
require for each package which I do not think all implementations do.

I see plain require() as a means to stitch together modules irrespective
of packages. Once you bring packages into the mix you degrade the
simplest use-case.

It is a mapping + package locator. I could be convinced that package
locators and mappings should be held in two separate properties but
don't think that is necessary.


> still not clear how to the modules from theirPackage get referenced.
> Hopefully not stuffed in the same top level namespace as everything else.

The modules from theirPackage cannot be referenced without specifying a
package top-level ID or an alias:

require("module", "domain.net/path/to/package/1") or
require("module", "theirPackage")

or

require("pm").require("module", "domain.net/path/to/package/1") or
require("pm").require("module", "theirPackage")

The syntax clearly discloses a pointer (ID or alias) to a package which
I think is critical for static analysis and intuitive programming.

That is a matter of further exploration which I have not gotten around
to yet. It is certainly doable.


> require2 as part of module packages module seems to couple
> packages/modules to a package manager, which I wouldn't want.

I don't want that either.

I am not so much concerned with the syntax of requiring packaged modules
(except for layering it on top of plain require()) than having a
standardized package.json that all package managers use.

I suggest we put forward a few concrete proposals for an enhanced
package.json that we can compare, contrast and vote on. What do you think?

Christoph

Irakli Gozalishvili

unread,
Apr 9, 2010, 3:04:21 PM4/9/10
to comm...@googlegroups.com
BTW to make my position clear, my preferences are in order:

1. Single strict package to URI mapping like in my proposal http://github.com/Gozala/github/tree/bespin or the approach that was taken by sproutecore tiki.
2. This proposal with an optional and default mapping

I'm totally against require2


--
Irakli Gozalishvili
Web: http://rfobic.wordpress.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands


ihab...@gmail.com

unread,
Apr 9, 2010, 3:14:46 PM4/9/10
to comm...@googlegroups.com
Hi everyone,

On Fri, Apr 9, 2010 at 7:52 AM, Kris Zyp <kri...@gmail.com> wrote:

> "mappings": [
>    {"from": "package/perstore/",  "to":
> "http://github.com/kriszyp/perstore/raw/master/lib/"}
> ]
>
> And this would guarantee that if I used a
> require("package/perstore/model") in my package, that it would load the
> module that has the canonical locator of
> http://github.com/kriszyp/perstore/raw/master/lib/model.js.

This looks fine to me. The main criteria that make it fine imho are:

1. A package is in control of its own namespace; it has sovereignty
(to use a favorite word of Kris Kowal's);

2. To the extent that a package may rely on catalogs to define
mappings for it, it chooses to put these catalogs into its "reliance
set" (so to speak);

3. There is a place to stand for some lint tool or whatever to warn me
when I (or catalogs I depend on) have mapped "package/perstore" to two
different things;

4. There is the possibility to rely on a catalog via a checksum (or
other "strong" reference) so that, if I'm really worried about things,
I can verify that the mappings, including those defined by my
catalogs, will *never* conflict; and

5. Absent such errors, any package I depend on cannot "reach up" and
pollute namespace regions I have allocated to other packages.

If this is a preferred alternative to require2, then I'd say go for it.

Ihab

--
Ihab A.B. Awad, Palo Alto, CA

Dean Landolt

unread,
Apr 9, 2010, 3:30:13 PM4/9/10
to comm...@googlegroups.com
I'd just like to say thank you for this very clear list, Ihab. It makes a lot of sense and (at least in my head) clarifies the rationale behind many of your prior objections.

Dustin Machi

unread,
Apr 9, 2010, 4:10:34 PM4/9/10
to comm...@googlegroups.com
Here is my simplified take on it based the stuff i've been working on to get commonjs/perstore/pintura loading in Titanium (JSCore). I've had quite a bit of fun trying to normalize things in the various packages to try to take advantage of them. In general, especially since my use cases are desktop application oriented (as opposed to server or browser oriented), I'm not a fan of placing a crapload of things on the search path to make this whole system work. It seems slow to me, and i think that while there are ways and things people have done to help avoid that, people do see this during startup times and during development restarts frequently. I would much prefer in my framework to specify a few paths where packages can be found rather than a ton of search paths within packages themselves. It is much easier to understand and implement, and requires less checking.

I treat the system as such:

- require("foo/bar") is an 'absolute' request for bar within package foo. I essentially then look for searchPath + id. To note here is that I treat the first part of the id as the package and insert "/lib/" between. So basically given this id I will look for searchPath + "/foo/lib/bar.js" for each of the search paths. While that /lib/ is hard coded, I will ultiimately pull that info from package.json of course to determine the subpath.

- require("foo") is also an absolute request. However in this case it is usually referring to the package itself. I'm not sure if it is a good idea, but for the time being I'm recognizing this request for a package instead of a file, and automatically adding in /lib/foo.js. In short require('foo")==require("foo/foo");

- require("./foo) or require("../foo") - These are relative requests within a package. I do not allow relative requests to leave a specific package.

This is pretty simple and easy to implement and doesnt' have to do too much 'searching'. However, sometimes we of course need to use a specific package or branch of a package and we need to deal with knowing where code for alternative engines can be found. Here is my simplified version of what KrisZ proposed.


{
...rest of package.json...

paths: [
"engine/${engine}",
"lib"
],

packageAliases: {
"perstore": "myApp/perstore"
}

}

the "paths" section basically defines paths to search for modules within the package itself. When the require() happens, and does the search above as described, it simply goes for the first match within the paths definition. Another package, say running titanium, that does require("foo/bar") will ultimately look in each searchPath + "foo/engine/titanium/bar.js". If that doesnt' exist require can move on to looking in searchPath + "foo/lib/bar.js". The aliases are for aliasing one external package (not file within the package) to another as KrisZ described with his mapping idea. In this version any absolute requests within this package for perstore or perstore/anything, will rewritten to be an absolute request to myApp/perstore/anything. Otherwise the same searching path rules described a moment ago apply.

Obviously I'm relatively new to this group and could be missing situations or things that have already been described, so please do point out any errors or problems.

Thanks,
Dustin

Kris Zyp

unread,
Apr 9, 2010, 4:31:18 PM4/9/10
to comm...@googlegroups.com

On 4/9/2010 12:57 PM, Irakli Gozalishvili wrote:
> Kris I do support you on this one even though I do think Christoph has
> a point - It's not predictable from a module source what
> require("foo/bar") will return.

If it can come from any package is that more predictable?

However, I do want to point that this proposal does not necessarily need
to preclude an implementation from using a single uniform top level
namespace. I believe one could take the mappings from every package and
apply all of them to every require. But, I would think that would be
very undesirable, and would increase chances of collisions (although
probably less collisions than just having every package in the top level
namespace, at least packages would have a fighting chance at voluntarily
disambiguating their references). Applying all mappings to all requires
would also violate Ihab's principles of package namespace sovereignty.
Just wanted to point out that it is possible though, not recommending it.

> If mapping will be optional and we'll have a default like: { from:"."
> to:"packages/<packageName>/" } I'm up for it.

This looks like a mapping from a relative reference. I don't think you
would ever want disrupt how relative references work. I am not exactly
sure what you mean by default mapping, but if you mean that you should
be able to do require("packages/someOtherPackage/someModule") without
any mappings, I think that would be out of scope for package.json to be
defining.

On 4/9/2010 12:57 PM, Kris Kowal wrote:
> [snip]


> Kris Zyp, it would be good to see this proposal in a concrete form. I
> think the necessary feature subset is an "id" to module URL mapping,
> an "id" to package URL mapping, and something for cryptographically
> verifying the content, taking a nod perhaps from Ihab's require2
> proposal. I'm not sure how versioning fits. I would also like some
> time to ponder how to implement this.
>

Yes, I can put together a proposal.

--
Thanks,
Kris

ihab...@gmail.com

unread,
Apr 9, 2010, 7:01:54 PM4/9/10
to comm...@googlegroups.com
On Fri, Apr 9, 2010 at 1:31 PM, Kris Zyp <kri...@gmail.com> wrote:
> Applying all mappings to all requires would also violate Ihab's principles
> of package namespace sovereignty.

In the spirit of completeness, there is one possible misfeature of
this sovereignty:

Let's say there is a library "framebuffer.js" that expects to be used
as the shared, singleton frame buffer by everyone in a CommonJS
sandbox.

Now let's say two modules, A and B, in the same sandbox require() that
module. However, they map the name to (slightly?) different versions
of the code from (slightly?) different places.

From the viewpoint of A's and B's sovereignty, we have to give A and B
the exact versions they asked for.

From the viewpoint of sandbox unity, A and B should get some singleton
instance of "framebuffer.js", or else the assumptions under which
"framebuffer.js" is constructed will be false.

What gives? For the CommonJS module instantiation model, it means that
the unit of "name sovereignty" should be a given _sandbox_, not a
_package_. So a sandbox should be constructed with a catalog that maps
--

package/framebuffer -> http://foo.com/frbufHelpers.zip
package/collections -> http://zoop.org/collections-v3.1.2.4.1.zip
package/... -> ...

and then _by agreement_, as part of being a member of the CommonJS
_community_, a package does a --

require('package/framebuffer/...');

and expects that (a) this "short, readable" identifier is well-known;
and (b) the sandbox has a mapping that maps it to concrete code.

Nothing says that *everything* in the *system* needs to be a single
top-level namespace. However, each sandbox must contain a mapping with
the desired properties.

In this way, each sandbox must have something very similar to an RPM
or apt-get (or whatever) system configuration. It's a bunch of
packages that are known to *all* work together.

The way to make that zero-admin would be to provide, at some site,
pre-configured mappings from (well-known) short names to specific
versioned code.

Does this make sense?

Irakli Gozalishvili

unread,
Apr 9, 2010, 7:04:44 PM4/9/10
to comm...@googlegroups.com
Kris sorry I have not illustrated it properly. Anyway while thinking of this proposal further I do see some potential mess and I would like to get your opinions on that.

Potential problem is that each package you will work with will have it's own view of what require("foo/bar") means and this spaghetti can be dangerous for a brain. Just try to think of a situation where you happened to have a many dependencies in your app and 5 of them do contain a mapping { from: "package/foo" to: "<x>" } where <x> is different for each of those. You will have a hard time reading the code. Besides this proposal is very friendly with a conflicting package names that most likely will happen and will cause issues described in this example.  Hopefully this example is described well enough to illustrate why IMO mapping per package is bad idea.

Hopefully we can take this discussion further and will be able to find a better solution. That being said I think it worth to look at a [proposal I wrote](http://github.com/Gozala/require-js/blob/gh-pages/docs/design/foundation.md) which I do believe addresses this problem nicely while still relies on mapping. I think we should be able to mix this two proposals and have something out of it.

I would be very happy if guys will take a few moments to look at and comment since I spend quite a while on writing it.

Thanks

--
Irakli Gozalishvili
Web: http://rfobic.wordpress.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands


Thanks,
Kris

Kris Kowal

unread,
Apr 9, 2010, 7:21:19 PM4/9/10
to comm...@googlegroups.com
On Fri, Apr 9, 2010 at 4:04 PM, Irakli Gozalishvili <rfo...@gmail.com> wrote:
> Hopefully we can take this discussion further and will be able to find a
> better solution. That being said I think it worth to look at a [proposal I
> wrote](http://github.com/Gozala/require-js/blob/gh-pages/docs/design/foundation.md)
> which I do believe addresses this problem nicely while still relies on
> mapping. I think we should be able to mix this two proposals and have
> something out of it.

This proposal is not tractable because it depends on a central package
naming authority, or depends upon multiple mutually exclusive package
naming authorities. This is the problem solved by require2 and
mappings; both give the package author sovereignty and permit packages
to be hosted on arbitrary URLs, thus deferring the name collision
space to the web where ownership of names is well managed and
verifiable.

The goal of making the behavior of require consistent across packages
is inconsistent with the goal of making packages sovereign entities,
with the ability to determine where their modules come from. Kris
Zyp's mappings proposal addresses my concerns that were addressed by
require2, but also addresses your concerns with extending the require
syntax. We appear to still not agree on where to draw the name space
sovereignty line. Your proposal draws it at the set of installed
packages, but multiple voices here suggest that it needs to be a
smaller scope that gives package authors and maintainers more control
over their invariants. It might even be necessary to provide a
spectrum of solutions.

Kris Kowal

Irakli Gozalishvili

unread,
Apr 9, 2010, 8:36:30 PM4/9/10
to comm...@googlegroups.com
@KrisK I do think you right. I think we all see pros and cons with both solutions & it might be a good idea to make a list of them so we all understand what we are choosing from. That being said I'm not totally against Kris's proposal only thing is that model I'm pushing had been successfully used in other languages without a big issues or at list I have not heard of them. While I don't know of real life analog to a model that Kris suggests, there for I do prefer to be on a safe path.


--
Irakli Gozalishvili
Web: http://rfobic.wordpress.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands



--

Kris Zyp

unread,
Apr 10, 2010, 10:50:40 PM4/10/10
to comm...@googlegroups.com
Here is my proposal for package.json defined mappings:
http://wiki.commonjs.org/wiki/Packages/Mappings

I attempted to keep it fairly minimal in scope. Getting into the
details, there are certainly a number different approaches that could be
taken. Below are the questions I thought were debatable. For all these I
chose the simplest option, to keep the spec as lightweight and as easy
to implement as possible. If more sophistication is needed, so be it.

Questions:
Should all mappings be applied even if there is a match, creating a set
of possible alternate module ids to use to look up a module, or should
it translate to single id/URI? For example, if a mapping was defined:
{
"foo/bar/": "http://bar.com/",
"foo/": "http://foo.com/"
}
then should require("foo/bar/1") induce a lookup for a module with the
URI http://foo.com/bar/1.js if there is no module with the URI of
http://bar.com/1.js? I suggest that it should not, I believe it
simplifies the specification and implementations if there is a maximum
of one translation/URI for any given module id, and require("foo/bar/1")
would map unambiguously to http://bar.com/1.js. However, this does
preclude the possibility of packages setting an expectation that all
other package modules are at the top level by creating multiples
mappings from "" to the packages URIs of the dependencies. If we were to
go this route, the structure of the mappings should be changed to be an
array so that multiple identical "from" values could be used in a set of
mappings.

Should we use an "if" property instead of "overlay" for engine-specific
mappings? I used "overlay" to be consistent since it is already part of
the packages specification. However, if the mappings was changed to an
array (with multiple identical "from" values), and "if" property would
be quite convenient.

Should we be using the jar URI schemes, or should we create a
multi-property mechanism for defining a URL and/or the zip location and
path inside the URL? I prefer using the jar scheme since it provides
nice elegant mappings.

Should we define the from/prefix and the "to" replacement to be a
regular expression and regex replacement instead of plain strings? This
would be more flexible, but makes the mappings a little more complex.

Thanks,
Kris

Irakli Gozalishvili

unread,
Apr 11, 2010, 8:21:37 AM4/11/10
to comm...@googlegroups.com
I still do think this approach is not browser friendly it also hard to imagine it being adopted for an extension development and is it's two important platforms we might end up with several versions of the same package which is very bad for an ecosystem. I think if we'll have a default mapping mechanism + optional package specific mapping overlaying  as suggested here we'll get best of two worlds. In regard how to avoid name collisions first that comes to my mind is a github repo with a la symlinks `package-name -> http://my.repo.com/package/` everyone can fork and contribute to the repo. We can even setup server that will perform redirects based on that repo. `http://get.js/package-name -> http://my.repo.com/package/`

--
Irakli Gozalishvili
Web: http://rfobic.wordpress.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands


Kris Zyp

unread,
Apr 11, 2010, 10:59:02 PM4/11/10
to comm...@googlegroups.com

On 4/11/2010 6:21 AM, Irakli Gozalishvili wrote:
> I still do think this approach is not browser friendly it also hard to
> imagine it being adopted for an extension development and is it's two
> important platforms we might end up with several versions of the same
> package which is very bad for an ecosystem.

I am not sure why package defined mappings are would cause problems for
the browser-based module loaders. The different approaches to browser
based module loading that I am aware of:

* For packages that come prewrapped with a module transport (like
transport/D) and are intended for cross-domain consumption (such as from
a CDN), there are two possible ways to provide the modules:
** The modules can be wrapped with full absolute URLs, and utilize
relative references or full URLs for dependencies/requires. This is
approach can beneficial because it requires zero client side
configuration and basically guarantees collision avoidance. Package
mappings does not need to be used for this, but package mappings does
imply a legitimization of the concept of modules having URLs.
** The modules can be wrapped with an aliased top level ids. In this
situation, a client side configuration of how the top level ids map to
the cross-domain URLs is needed. This type of configuration is exactly
what mappings-aware client provides. While a client side loader would
not necessarily need to use a package.json to create these mappings, it
is convenient and consistent that this mapping mechanism is the same as
the package.json mapping. Essentially this can be solved with a
package.json mappings aware client or configuring with the same type of
mapping capability.

* The other major use case for the browser is on-the-fly module
wrappers, that take CommonJS modules and wrap them with the module
transport. Ideally one would like to avoid rewriting require statements
inside the module body, and only wrap (only prepend and postpend the
module body). This can be achieved in the majority of cases by having a
using a union of all package mappings (that will be used by the client)
for looking up modules from HTTP requests (on the server). As long as
the union doesn't create any ambiguities (and following reasonable
conventions of using package names as prefixes would avoid ambiguities
as well as any other proposal), than we can simply wrap.

If mappings conflict (perhaps due to conflicting package names), the
module package.json mappings preserve the requisite information for us
to do require rewrites to resolve the conflicts and disambiguate module
ids prior to delivery to the browser.

AFAICT, in all situations, package.json mappings is either beneficial
for the browser targeted modules, or does not hinder other mechanisms.

> I think if we'll have a default mapping mechanism + optional package
> specific mapping overlaying as suggested here we'll get best of two
> worlds.

As mentioned in my proposal, the package.json mappings doesn't say
anything about or preclude default package/module referencing. You could
certainly do that.

> In regard how to avoid name collisions first that comes to my mind is
> a github repo with a la symlinks `package-name ->

> http://my.repo.com/package/` <http://my.repo.com/package/%60> everyone


> can fork and contribute to the repo. We can even setup server that
> will perform redirects based on that repo. `http://get.js/package-name

> -> http://my.repo.com/package/` <http://my.repo.com/package/%60>

I am not sure I understand this, but it doesn't sound compatible with a
decentralized approach.

--
Thanks,
Kris

Irakli Gozalishvili

unread,
Apr 12, 2010, 5:50:35 AM4/12/10
to comm...@googlegroups.com
Thanks Kris for a reply, but I do find that approaches that enforce a wrapping boilerplate or a tools to do that are browser unfriendly we can do better. There is another concern I have regarding unreadability of a code, where same module id can mean completely different thing across different packages and will crate make a mess. Just try to think of a case where you have 5 foo named packages and try to explain which one do you use, this should be enough to illustrate my concern.

It seems that I'm the only one with this concerns so I will just let you move forward, hoping that I'm wrong


--
Irakli Gozalishvili
Web: http://rfobic.wordpress.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands


Thanks,
Kris

Reply all
Reply to author
Forward
0 new messages