[CommonJS] packages

24 views
Skip to first unread message

Irakli Gozalishvili

unread,
May 3, 2010, 6:54:16 PM5/3/10
to CommonJS
Hi

I thought I'll try to push this topic once again, since we are pretty close to an agreement (I think), since all the existing suggestions are more or less compatible with mapping proposal from KrisZ.

- Is there any point of keeping package dependency listings in package.json? Mapping expresses dependencies explicitly I believe.

- Can mapping be used for providing an overlays as well for example
mappings: {'foo/bar': 'http://myhost.com/lib/bar.js'}   // where require('foo/bar') requires http://myhost.com/lib/bar.js

- @Charles can we make seedjs compatible with Kris's proposal ? Cause one issue I think of is illustrated below
mappings: {'foo:': 'http://myhost.com/lib/'}  // to require http://myhost.com/lib/bar.js
seed will expect require('foo:bar) while other platforms will expect require('foo:/bar)

Is there any other open questions ?

KrisZ Mind writing proposal since you're an author (Sorry if you already did so, can't find it on a wiki) ?
--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

--
You received this message because you are subscribed to the Google Groups "CommonJS" group.
To post to this group, send email to comm...@googlegroups.com.
To unsubscribe from this group, send email to commonjs+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/commonjs?hl=en.

Charles Jolley

unread,
May 3, 2010, 7:03:38 PM5/3/10
to comm...@googlegroups.com
- @Charles can we make seedjs compatible with Kris's proposal ? Cause one issue I think of is illustrated below
mappings: {'foo:': 'http://myhost.com/lib/'}  // to require http://myhost.com/lib/bar.js
seed will expect require('foo:bar) while other platforms will expect require('foo:/bar)

I don't mind making seed compatible, but I have some concern still about using / as the package separator.  The problem is:

require('foo/bar/baz') 

Is ambiguous.  This could be:

'foo/bar/baz' module from the current package
'baz' module from the 'foo/bar' package
'bar/baz' from the 'foo' package
'foo/bar/baz' from the underlying system (i.e. to remain compatible with node for example, you have to fallback to looking for a simple module outside of packages)

Even the mappings as proposed by Kris doesn't solve this.  What if I have defined:

mappings: {
  "foo":  "...",
  "foo/bar": "..."
}

We basically end up back where we started - looking at a require() statement it is not clear what should be loaded (and any rules we come up with in the spec are going to be complex enough that implementors could easily screw them up).

On the other hand:

require('foo:bar/baz');
require('foo/bar:baz');
require('foo/bar/baz');

is far less ambiguous.  The last one is technically still ambiguous but less likely to be so since package cases are removed.

How would you address this ambiguous scenario with mappings?

-C




Is there any other open questions ?

KrisZ Mind writing proposal since you're an author (Sorry if you already did so, can't find it on a wiki) ?
--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

--
You received this message because you are subscribed to the Google Groups "CommonJS" group.
To post to this group, send email to comm...@googlegroups.com.
To unsubscribe from this group, send email to commonjs+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/commonjs?hl=en.

Kris Zyp

unread,
May 3, 2010, 7:12:26 PM5/3/10
to comm...@googlegroups.com, Irakli Gozalishvili


On 5/3/2010 4:54 PM, Irakli Gozalishvili wrote:
Hi

I thought I'll try to push this topic once again, since we are pretty close to an agreement (I think), since all the existing suggestions are more or less compatible with mapping proposal from KrisZ.

- Is there any point of keeping package dependency listings in package.json? Mapping expresses dependencies explicitly I believe.

- Can mapping be used for providing an overlays as well for example
mappings: {'foo/bar': 'http://myhost.com/lib/bar.js'}   // where require('foo/bar') requires http://myhost.com/lib/bar.js
Yes, and there is an example of using overlays in the proposal:
http://wiki.commonjs.org/wiki/Packages/Mappings


- @Charles can we make seedjs compatible with Kris's proposal ? Cause one issue I think of is illustrated below
mappings: {'foo:': 'http://myhost.com/lib/'}  // to require http://myhost.com/lib/bar.js
seed will expect require('foo:bar) while other platforms will expect require('foo:/bar)
This should already be compatible. With the mapping you defined, require('foo:bar')  should map to http://myhost.com/lib/bar.js. Doing require('foo:/bar') would map to http://myhost.com/lib//bar.js.



Is there any other open questions ?

I guess Charles is asking them right now...


KrisZ Mind writing proposal since you're an author (Sorry if you already did so, can't find it on a wiki) ?


--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands
--
You received this message because you are subscribed to the Google Groups "CommonJS" group.
To post to this group, send email to comm...@googlegroups.com.
To unsubscribe from this group, send email to commonjs+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/commonjs?hl=en.

-- 
Thanks,
Kris

Tim Schaub

unread,
May 3, 2010, 7:20:32 PM5/3/10
to comm...@googlegroups.com
Charles Jolley wrote:
>> - @Charles can we make seedjs compatible with Kris's proposal ? Cause one issue I think of is illustrated below
>> mappings: {'foo:': 'http://myhost.com/lib/'} // to require http://myhost.com/lib/bar.js
>> seed will expect require('foo:bar) while other platforms will expect require('foo:/bar)
>
> I don't mind making seed compatible, but I have some concern still about using / as the package separator. The problem is:
>
> require('foo/bar/baz')
>
> Is ambiguous. This could be:
>
> 'foo/bar/baz' module from the current package
> 'baz' module from the 'foo/bar' package
> 'bar/baz' from the 'foo' package
> 'foo/bar/baz' from the underlying system (i.e. to remain compatible with node for example, you have to fallback to looking for a simple module outside of packages)
>
> Even the mappings as proposed by Kris doesn't solve this. What if I have defined:
>
> mappings: {
> "foo": "...",
> "foo/bar": "..."
> }
>

How about specifying that aliases are a single term (only word chars)?

And (with apologies if this has been covered), why not put the mappings
right in the dependency member (assuming the order is not significant)?

"dependencies": {
"package-alias": "package-identifier"
}

Or, to allow more detail to be conveyed:

"dependencies": {
"narwhal-lib": {
"id": "http://github.com/kriskowal/narwhal-lib",
"version": "0.2"
}
}


Tim

Kris Kowal

unread,
May 3, 2010, 7:21:54 PM5/3/10
to comm...@googlegroups.com
On Mon, May 3, 2010 at 4:03 PM, Charles Jolley <cha...@sproutit.com> wrote:
> How would you address this ambiguous scenario with mappings?

I would describe this problem not as "ambiguous" but as "flexible", as
long as the behavior is well defined by the specification. My
understanding is that the mappings would be tried first, in order, for
the container package, falling back to the unspecified native module
search. It might be good to change the mappings to an array for that
reason, although object literals can be depended upon in JavaScript to
be enumerated in definition order — this would not be the case for
alternate language tooling with the same JSON.

Kris Kowal

Kris Zyp

unread,
May 3, 2010, 7:24:13 PM5/3/10
to comm...@googlegroups.com


On 5/3/2010 5:03 PM, Charles Jolley wrote:
- @Charles can we make seedjs compatible with Kris's proposal ? Cause one issue I think of is illustrated below
mappings: {'foo:': 'http://myhost.com/lib/'}  // to require http://myhost.com/lib/bar.js
seed will expect require('foo:bar) while other platforms will expect require('foo:/bar)

I don't mind making seed compatible, but I have some concern still about using / as the package separator.  The problem is:

require('foo/bar/baz') 

Is ambiguous.  This could be:

'foo/bar/baz' module from the current package
'baz' module from the 'foo/bar' package
'bar/baz' from the 'foo' package
'foo/bar/baz' from the underlying system (i.e. to remain compatible with node for example, you have to fallback to looking for a simple module outside of packages)

Even the mappings as proposed by Kris doesn't solve this.  What if I have defined:

mappings: {
  "foo":  "...",
  "foo/bar": "..."
}

We basically end up back where we started - looking at a require() statement it is not clear what should be loaded (and any rules we come up with in the spec are going to be complex enough that implementors could easily screw them up).

Actually, this is explicitly defined in the spec:
"The mapping with the longest property in the mappings object that matches should be applied, and subsequent mapping objects may then be ignored."
There is no ambiguity in how mappings are to be calculated. Of course you can argue that one can create confusing mappings ;).

On the other hand:

require('foo:bar/baz');
require('foo/bar:baz');
require('foo/bar/baz');

is far less ambiguous.  The last one is technically still ambiguous but less likely to be so since package cases are removed.

How would you address this ambiguous scenario with mappings?

As I have noted before, package mappings are completely with the seedjs convention. It is perfectly legal to do:
mappings:{
  "foo:": "...",
  "foo/bar:": "...",
 ..
The use of slashes was just what I used in the example because I thought it was the most likely convention. I'd be happy to change the examples if we want a different convention. I've noted my concern with the use of colon as a package delimiter being confusing with URI protocol syntax, but this is technically overcome-able, and I could live with colons if need be.

However, I think what you are really getting at is that users shouldn't be expected to include the package delimiter in the mapping key, and that the package delimiter should always be inferred. This is a reasonable objection. But, I believe this greatly limits the flexibility of simple string substitution or regex based model. I think the current design is elegant in that it can be used for package mapping, overlay mapping, mapping to nested folders within packages, and even mapping single path/identifers to nested paths. All of these various forms of mappings I am currently enjoying being able to use with the mappings impl I wrote [1]. If all we cared about was just mapping to the top level of packages, inferring package delimiters would be fine, but there is so much more we can do with mappings.

[1] http://github.com/kriszyp/nodules


-- 
Thanks,
Kris

Kris Zyp

unread,
May 3, 2010, 7:34:27 PM5/3/10
to comm...@googlegroups.com
And to address another open question that I had asked earlier on...
Having implemented the package mappings proposal, I have come to the
conclusion that treating the mappings as regex replacements (with
inferred ^ start string anchor) rather than plain string substitutions
is more flexible, and actually slightly easier to implement. In
particular this makes it safer to write mappings for single identifier
module ids:
mappings:{
"model$": "..." # only matches "model", not "model/something",
"model/": "..." # matches anything that starts with "model/",

Kris

Kris Kowal

unread,
May 3, 2010, 7:36:57 PM5/3/10
to comm...@googlegroups.com
On Mon, May 3, 2010 at 4:34 PM, Kris Zyp <kri...@gmail.com> wrote:
> And to address another open question that I had asked earlier on...
> Having implemented the package mappings proposal, I have come to the
> conclusion that treating the mappings as regex replacements (with
> inferred ^ start string anchor) rather than plain string substitutions
> is more flexible, and actually slightly easier to implement. In
> particular this makes it safer to write mappings for single identifier
> module ids:

This however mixes poorly with the "longest expression first" rule.

Kris Kowal

Kris Zyp

unread,
May 3, 2010, 7:44:45 PM5/3/10
to comm...@googlegroups.com


On 5/3/2010 5:36 PM, Kris Kowal wrote:
> On Mon, May 3, 2010 at 4:34 PM, Kris Zyp <kri...@gmail.com> wrote:
>
>> And to address another open question that I had asked earlier on...
>> Having implemented the package mappings proposal, I have come to the
>> conclusion that treating the mappings as regex replacements (with
>> inferred ^ start string anchor) rather than plain string substitutions
>> is more flexible, and actually slightly easier to implement. In
>> particular this makes it safer to write mappings for single identifier
>> module ids:
>>
> This however mixes poorly with the "longest expression first" rule.
>

You are probably right, and I can think of some contrived cases where
longer regex strings would be less specific, but do you have an example
of how this would be really problematic/confusing in useful mappings?

--
Thanks,
Kris

James Burke

unread,
May 3, 2010, 8:32:53 PM5/3/10
to comm...@googlegroups.com
On Mon, May 3, 2010 at 4:12 PM, Kris Zyp <kri...@gmail.com> wrote:
> http://wiki.commonjs.org/wiki/Packages/Mappings

Kris Zyp, how would names be matched in this scenario?

mappings: {
"foo": "some/path/to/foo"
}

then I do a require("foo") and a require("foobar")? Does the mapping
above apply to both?

I am hoping it only applies to require("foo") and not
require("foobar"). I have found it useful to have a mapping like so:

mappings: {
"foo": "some/path/v2/foo"
}

where I want just require("foo") to be resolved to
"some/path/v2/foo.js". I would not want require("foobar") mapping to
"some/path/v2/foobar.js"

I also like being able to map all modules "under" foo to a directory:

require("foo/baz") -> "some/path/v2/foo/baz.js"
require("foobar") -> "possible/other/path/foobar.js"

I appreciate I could do the last one with mappings for a "foo/", but I
liked in general that a value without a slash indicated a first level
"segment" in a module ID. It is also not clear how I would do the case
where I want "foo" to be mapped to "foo.js" and "foo/bar" mapped to
"foo/bar.js" with just one mapping entry. My first impression of
needing a separate rule that uses a regexp for a "$" on the end
strikes me as odd, but then I am coming from the way we did mappings
in Dojo too.

My general question could be restated as, why not use the mapping
algorithm that Dojo uses (except use "/" instead of ".")?

James

Kris Kowal

unread,
May 3, 2010, 10:27:46 PM5/3/10
to comm...@googlegroups.com
Perhaps we should go back to the simplest form of this proposal and
figure out what we buy by generalizing, and see whether the
generalization is worth paying for.

Suppose that mappings map ID "directories" to package lib directories
and nothing more. Thus "foo/bar/baz" could be affected by either a
"foo" or a "foo/bar" mapping, but no other. Mappings would not employ
regular expressions. We can use the longest-pattern-applies-first
algorithm without any uncertainty or hazards. Subtrees can be
subverted transparently, if needed.

The weakness of this proposal (and even more flexible proposals) is
that the package prefix does not map to any particular module. NPM
resolves this by associating the package prefix with an "index.js" in
the root of the package. For example, if "foo/bar" is mapped to the X
package, X/index.js would be returned by require("foo/bar"). I am
indifferent to this approach.

If we use regexen, we open the possibility of using any delimiter at
all to distinguish self-packaged modules from sub-packaged modules. I
don't see an advantage of introducing arbitrary delimiters,
particularly since this might reduce our options for standardizing
extensions to the module identifier name space.

I'm, in any case, in favor of keeping things simple. I could vote
"yes" on mapping dirs to dirs with an implied "/" delimiter between
right now. I would take some convincing for regexen or alternate
delimiters.

Kris Kowal

Charles Jolley

unread,
May 4, 2010, 12:20:39 AM5/4/10
to comm...@googlegroups.com
Based on my experience writing the SproutCore build tools three times now (including the dependency system), I have found that dependencies are an endless source of grief for most developers. Therefore it is important to keep the rules very simple and easy to understand.

For example:

require('foo/bar/baz') will get the bar/baz module from whatever package is mapped to foo unless you have listed foo/bar in your mappings, in which case it will get the 'baz' module from whatever packages is mapped to foo/bar. It will load the latest version of the package mapped to foo or foo/bar in the mappings hash as listed in the dependencies hash.

Even the simplest proposal for mapping is pretty complicated!

Maybe the proposal that we extend the dependencies hash (as proposed by Tim Schaub) and limit package names to the first term is the best? i.e.

require('foo/bar/baz') will get the bar/baz module from the packaged mapped to 'foo' in the dependencies hash. You can name a package, location, and version in the dependencies hash.

Seems pretty simple. Does this still meet the requirements for mappings?

-C

Kris Kowal

unread,
May 4, 2010, 1:12:58 AM5/4/10
to comm...@googlegroups.com
On Mon, May 3, 2010 at 9:20 PM, Charles Jolley <cha...@sproutit.com> wrote:

> Maybe the proposal that we extend the dependencies hash (as proposed
> by Tim Schaub) and limit package names to the first term is the
> best? i.e.
>
> require('foo/bar/baz') will get the bar/baz module from the packaged
> mapped to 'foo' in the dependencies hash.  You can name a package,
> location, and version in the dependencies hash.
>
> Seems pretty simple.  Does this still meet the requirements for
> mappings?

I think that it would be a good start. It's certainly the option we
would be least likely to regret. We could surely extend the system in
a future version as we collectively gain more experience.

Irakli Gozalishvili

unread,
May 4, 2010, 7:55:27 AM5/4/10
to comm...@googlegroups.com
Please let's exclude fallback scenarios since it's browser platform unfriendly. I do understand that this is flexibility but it means either you use tooling or server support for browser development or you cannot rely on commonjs packages. Think that tooling enforcement is not something that commonjs should do. So disambiguity would be nice.

I also would like to see merge of mappings with dependencies


--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands


Irakli Gozalishvili

unread,
May 4, 2010, 8:36:59 AM5/4/10
to comm...@googlegroups.com
Cool I also like @Tim's proposal a lot!! It also addresses all the concerns I had with other proposals.

I think proposal can be extended further by making it like this, where either id or uri should be presented.
 
"dependencies": {
    "narwhal-lib": { // alias following same naming convention as module id's except `/`
        "uri": "http://github.com/kriskowal/narwhal-lib/",  // explicit mapping
        "version": "0.2"                                    // does verision makes sence with expliciting mapping ?
    },
    "foo": {    // alias
        "id": "foo-bar-pack",   // package installed in platform with the matching name
        "version": ["1.0", "2.0"] //(optional ?
platform preference) version
    }
}

In this case we'll end up with a simple extension to a current packages




--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands


Kris Zyp

unread,
May 4, 2010, 9:41:03 AM5/4/10
to comm...@googlegroups.com


On 5/3/2010 6:32 PM, James Burke wrote:
> On Mon, May 3, 2010 at 4:12 PM, Kris Zyp <kri...@gmail.com> wrote:
>
>> http://wiki.commonjs.org/wiki/Packages/Mappings
>>
> Kris Zyp, how would names be matched in this scenario?
>
> mappings: {
> "foo": "some/path/to/foo"
> }
>
> then I do a require("foo") and a require("foobar")? Does the mapping
> above apply to both?
>
Yes, with the current specified algorithm it would apply to both.
> I am hoping it only applies to require("foo") and not
> require("foobar"). I have found it useful to have a mapping like so:
>
> mappings: {
> "foo": "some/path/v2/foo"
> }
>
> where I want just require("foo") to be resolved to
> "some/path/v2/foo.js". I would not want require("foobar") mapping to
> "some/path/v2/foobar.js"
>
> I also like being able to map all modules "under" foo to a directory:
>
> require("foo/baz") -> "some/path/v2/foo/baz.js"
> require("foobar") -> "possible/other/path/foobar.js"
>
> I appreciate I could do the last one with mappings for a "foo/", but I
> liked in general that a value without a slash indicated a first level
> "segment" in a module ID. It is also not clear how I would do the case
> where I want "foo" to be mapped to "foo.js" and "foo/bar" mapped to
> "foo/bar.js" with just one mapping entry. My first impression of
> needing a separate rule that uses a regexp for a "$" on the end
> strikes me as odd, but then I am coming from the way we did mappings
> in Dojo too.
>
> My general question could be restated as, why not use the mapping
> algorithm that Dojo uses (except use "/" instead of ".")?
>

Hmm, the Dojo algorithm, that sounds like something I should know about
:). Could you share it?

Anyway, I totally agree that we need to meet the use cases you shared.
So far I haven't heard any alternates to the search & replace (plain
string or regex) algorithm that would meet them (Tim's proposal can't
map to individual modules or won't work with ad-hoc folder structures,
so it won't do). However, I certainly don't have my heart set on the
current algorithm, so I would be eager to hear other ideas.

--
Thanks,
Kris

Christoph Dorn

unread,
May 4, 2010, 12:25:44 PM5/4/10
to comm...@googlegroups.com

Suppose we limit the package aliases to the first term:

"require('foo/bar/baz') will get the bar/baz module from the package
mapped to 'foo'"

We can map packages with:

"mappings": {
"foo": { // alias
"uri": "http://github.com/foo/...",
"version": "0.2"
}
}

To map to a specific module you could now do:

"mappings": {
"foo": { // alias
"uri": "http://github.com/foo/...",
"version": "0.2",
"module": "specific-foo"
}
}

require("foo")

This could be used to overwrite system modules as well.

I too have found the need to map to a module in package.json and think
it is a necessary feature.

I am adamant about needing a clear separation between the package
component and module component in a module ID. To achieve this for
one-arg require the package must *always* be prefixed even for system
modules.

require("system/system")
require("commonjs/system")
require("foo/bar")
require("foo") -> special case (mapped to module as described above)

This would necessitate the platform providing some default packages such
as "system" or "commonjs" or the package.json needs to map *all* packages.

The reason I like the two-arg require is that it provides in my mind
more flexibility.

require("system") -> the "system" module from the system namespace
require("system", "commonjs") -> the "system" module from the "commonjs"
package

*Any* package can install modules into the system namespace by declaring
them in the "dependencies" package.json property while "packages" get
defined in the "packages" property.

Mappings in this case would be used to map modules in the system
namespace. This can be used to organize modules for transport to the
browser with a browser loader that re-maps them to the two-arg require
when they arrive at the client. I have not had a chance to play with
that but believe it is possible and fairly trivial.

I see the module system at several levels:

Level 0: bootstrapping
Level 1: 1 arg require global namespace module system
Level 2: 2 arg require or other aliased package-based module systems
Level 3: package spaces with dependencies mapped in package.json

Level 0 and 1 would be platform specific while level 2 and 3 would be
interoperable across all platforms.

I presume we all have the goal of writing programs split into packages
that run on a variety of platforms. I am also observing that
implementors have their own ideas of how they believe packages should
operate on their platforms.

Maybe our focus should remain at level 1 with the exception of
specifying a common package.json format that works for all package
managers and loaders. I think we are close in that regard. In my mind
this should suffice to achieve interoperability because when a package
loads it can load a module loader to load its own dependencies from
whatever package ecosystem it desires. I see this as no different to a
website running prototype, jquery and dojo in parallel each with their
own plugin/module systems.

Over time I see certain package loaders succeeding likely primarily
based on the strength of the community surrounding them, not based on
the specifics of how they operate or their require syntax.

Christoph



Irakli Gozalishvili wrote:
> Cool I also like @Tim's proposal a lot!! It also addresses all the
> concerns I had with other proposals.
>
> I think proposal can be extended further by making it like this, where
> either id or uri should be presented.
>
> *"dependencies": {
> "narwhal-lib": { // alias following same naming convention as module
> id's except `/`
> "uri": "http://github.com/kriskowal/narwhal-lib/", // explicit
> mapping
> "version": "0.2" // does
> verision makes sence with expliciting mapping ?
> },
> "foo": { // alias
> "id": "foo-bar-pack", // package installed in platform with
> the matching name
> "version": ["1.0", "2.0"] //(optional ? **platform preference**)
> version
> }
> }
> *
> <mailto:comm...@googlegroups.com>.
> To unsubscribe from this group, send email to
> commonjs+u...@googlegroups.com
> <mailto:commonjs%2Bunsu...@googlegroups.com>.

Kris Zyp

unread,
May 4, 2010, 12:45:03 PM5/4/10
to comm...@googlegroups.com, Christoph Dorn


On 5/4/2010 10:25 AM, Christoph Dorn wrote:

Suppose we limit the package aliases to the first term:

"require('foo/bar/baz') will get the bar/baz module from the package mapped to 'foo'"

We can map packages with:

"mappings": {
    "foo": { // alias
        "uri": "http://github.com/foo/...",
        "version": "0.2"
    }
}

To map to a specific module you could now do:

"mappings": {
    "foo": { // alias
        "uri": "http://github.com/foo/...",
        "version": "0.2",
        "module": "specific-foo"
    }
}

require("foo")

This could be used to overwrite system modules as well.

I too have found the need to map to a module in package.json and think it is a necessary feature.


This strategy sounds good, although I think we can do all of this without adding new properties:
"mapping":{
  "base64": "http://github.com/kriskowal/narwhal-lib/raw/narwhal-lib/lib/base64.js",
  "jack": http://github.com/tlrobinson/jack/raw/0.3/lib/jack/",
 ...

We can determine that the first mapping defines a module, so we can do require("base64"). The second one ends with a slash, so we could determine that it is mapping to a folder, so we could infer the package delimiter in the mapping, to do require("jack/querystring") (or require("jack:querysting") to follow Charles preferred package delimiter). A couple other important things to notice about the second mapping. First, Jack has all of its modules in side it's lib/jack/ directory. If we can only map to packages themselves, we would have to write jack twice for each require (like require("jack/jack/querystring")) in every module which is far more error prone than even the most complicated rule set for a single package.json mapping. It is critical that we are mapping to URIs is the foundational concept, packages work within that scheme. Also, URIs easily allow us to indicate versioning, specifically in a way that indicates how to locate the specified version. In the example, I am linking to version 0.3 (branch).


I am adamant about needing a clear separation between the package component and module component in a module ID. To achieve this for one-arg require the package must *always* be prefixed even for system modules.

require("system/system")
require("commonjs/system")
require("foo/bar")
require("foo") -> special case (mapped to module as described above)

This would necessitate the platform providing some default packages such as "system" or "commonjs" or the package.json needs to map *all* packages.

The reason I like the two-arg require is that it provides in my mind more flexibility.

require("system") -> the "system" module from the system namespace
require("system", "commonjs") -> the "system" module from the "commonjs" package

*Any* package can install modules into the system namespace by declaring them in the "dependencies" package.json property while "packages" get defined in the "packages" property.

Mappings in this case would be used to map modules in the system namespace. This can be used to organize modules for transport to the browser with a browser loader that re-maps them to the two-arg require when they arrive at the client. I have not had a chance to play with that but believe it is possible and fairly trivial.

I see the module system at several levels:

Level 0: bootstrapping
Level 1: 1 arg require global namespace module system
Level 2: 2 arg require or other aliased package-based module systems
Level 3: package spaces with dependencies mapped in package.json

Level 0 and 1 would be platform specific while level 2 and 3 would be interoperable across all platforms.

I presume we all have the goal of writing programs split into packages that run on a variety of platforms. I am also observing that implementors have their own ideas of how they believe packages should operate on their platforms.

Maybe our focus should remain at level 1 with the exception of specifying a common package.json format that works for all package managers and loaders. I think we are close in that regard. In my mind this should suffice to achieve interoperability because when a package loads it can load a module loader to load its own dependencies from whatever package ecosystem it desires. I see this as no different to a website running prototype, jquery and dojo in parallel each with their own plugin/module systems.

I agree, separating a package.json-aware module loader from the bootstrap module system is very reasonable.
-- 
Thanks,
Kris

Tim Schaub

unread,
May 4, 2010, 2:40:17 PM5/4/10
to comm...@googlegroups.com
Right. My thinking previously (that didn't go over well) was
require("js/file") etc.

The advantage of this is that a package could specify what flavor of js
they depend on.

If going with require("package-alias/module-id") is too verbose, I
assume a module loader could allow a default package to be specified
(e.g. narwhal for narwhal). Then require("file") would load the file
module from the default package if a package with the alias "file" is
not specified in the current package dependencies.

The default package (or root or base or whatever) could also be
specified in package.json.

Tim

Christoph Dorn

unread,
May 4, 2010, 3:00:07 PM5/4/10
to comm...@googlegroups.com
> <http://github.com/tlrobinson/jack/raw/master/lib/jack/>
> ...
>
> We can determine that the first mapping defines a module, so we can do
> require("base64"). The second one ends with a slash, so we could
> determine that it is mapping to a folder, so we could infer the package
> delimiter in the mapping, to do require("jack/querystring") (or
> require("jack:querysting") to follow Charles preferred package
> delimiter). A couple other important things to notice about the second
> mapping. First, Jack has all of its modules in side it's lib/jack/
> directory. If we can only map to packages themselves, we would have to
> write jack twice for each require (like
> require("jack/jack/querystring")) in every module which is far more
> error prone than even the most complicated rule set for a single
> package.json mapping. It is critical that we are mapping to URIs is the
> foundational concept, packages work within that scheme. Also, URIs
> easily allow us to indicate versioning, specifically in a way that
> indicates how to locate the specified version. In the example, I am
> linking to version 0.3 (branch).

Ok, I think I finally got the idea behind the "mapping" property. Keep
it simple and for pointing to URIs only and use the expanded package
locators for the "dependencies" and "packages" properties with
significant more flexibility so you can do for example:

{
"packages": {
"jack": { // a package locator object
"locator": "http://github.com/tlrobinson/jack/zipball/master/",
"prefix": "jack"
}
}
}

Which assumes jack's package.json specifies modules are in the "lib"
directory and "prefix" is added before module ids are resolved as one
example so you can do:

require("jack:querystring");

instead of

require("jack:jack/querystring");


> I agree, separating a package.json-aware module loader from the
> bootstrap module system is very reasonable.

So now the question is what properties the package locators should
support to allow for interoperability between package managers.

Christoph

James Burke

unread,
May 6, 2010, 1:28:40 PM5/6/10
to comm...@googlegroups.com
On Tue, May 4, 2010 at 6:41 AM, Kris Zyp <kri...@gmail.com> wrote:
> Hmm, the Dojo algorithm, that sounds like something I should know about
> :). Could you share it?

Dojo, and RequireJS, use a "longest match wins" but only allow
segmented paths (in CommonJS-land, the paths are segmented by "/"), no
regexps, and the mappings start from the beginning of the module IDs:

mappings: {
"foo/bar": "path/to/foo",
"foo/bar/baz": "different/path/to/baz",
"bam": "path/to/bam"
}

require("foo/bar") -> path/to/foo/bar.js
require("foo") -> path/to/foo.js
require("foo/bar/baz") -> different/path/to/baz.js
require("foo/bar/baz/far") -> different/path/to/baz/far.js
require("foo/bar/bam") -> path/to/foo/bar/bam.js
require("bam/blur") -> /path/to/bam/blur.js

From what I understand of the conversation in this thread, it sounds
like the desire to only map one segment, the first one, is being
advocated. That would work with the approach described above, just
specify the first segment only.

I find the above approach fairly flexible and allows mapping a single
module ID to a single file, which has been useful in practice -- it
allows an easy swap of a module without modifying code that depends on
it. That may be out of context for the package mapping discussion
here, so feel free to ignore that.

James

Kris Zyp

unread,
May 6, 2010, 2:59:40 PM5/6/10
to comm...@googlegroups.com, James Burke
Assuming the first mapping was supposed to be "foo": ..., this makes
total sense, and I think we are generally in agreement that this is what
we want. I would propose a slight alteration. I think that if target
path ends with a slash, we should only match prefixes, and not exact
matches. Ending with a slash should be a way to provide explicit path
only conversion. For example:

mappings:{
"json": "http://example.com/json-package/lib/"
}
Then:
// this should not go to http://example.com/json-package/lib.js, it
should go directly to the default lookup path
require("json");
// this should go to http://example.com/json-package/lib/schema.js
require("json/schema");

But I think the primary question that is still in contention is what the
mapping delimiter character should be. In these examples we are using
slashes. Charles has suggested colons (for less confusion with paths),
so with the previous example:
require("json:schema");

Maybe we should just take this to a vote? What would you prefer for a
mapping delimiter character:
a. '/'
b. ':'
c. Something else, '!' or ';' anyone?

--
Thanks,
Kris

Dean Landolt

unread,
May 6, 2010, 3:08:03 PM5/6/10
to comm...@googlegroups.com
Dean Landolt: +1 (until someone can make a stronger case for confusion)
 
b. ':'
c. Something else, '!' or ';' anyone?

--
Thanks,
Kris

--
You received this message because you are subscribed to the Google Groups "CommonJS" group.
To post to this group, send email to comm...@googlegroups.com.
To unsubscribe from this group, send email to commonjs+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/commonjs?hl=en.

Kris Kowal

unread,
May 6, 2010, 3:30:52 PM5/6/10
to comm...@googlegroups.com
On Thu, May 6, 2010 at 11:59 AM, Kris Zyp <kri...@gmail.com> wrote:
> Maybe we should just take this to a vote? What would you prefer for a
> mapping delimiter character:
> a. '/'
> b. ':'
> c. Something else, '!' or ';' anyone?

a. "/"

Kris Kowal

Irakli Gozalishvili

unread,
May 10, 2010, 6:03:25 PM5/10/10
to comm...@googlegroups.com
I'm fine with both a) and b) even though I see advantages of having different delimiter. I can also suggest
@ delimiter

--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands


Irakli Gozalishvili

unread,
May 10, 2010, 6:07:33 PM5/10/10
to comm...@googlegroups.com, James Burke
KrisZ I believe seedjs has one more thing which I personally won't miss much, but some might. From your mapping example seedjs would:

require("json"); // gets http://example.com/json-package/lib/index.js



--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands


Kris Zyp

unread,
May 10, 2010, 6:18:12 PM5/10/10
to comm...@googlegroups.com


On 5/10/2010 4:07 PM, Irakli Gozalishvili wrote:
KrisZ I believe seedjs has one more thing which I personally won't miss much, but some might. From your mapping example seedjs would:

require("json"); // gets http://example.com/json-package/lib/index.js


I don't have any objections to the use of index.js as the default module for a folder, but shouldn't that be handled by the web server, just like index.html? Your browser doesn't request http://somesite.com/index.html for you when you type in http://somesite.com/.

-- 
Thanks,
Kris

Charles Jolley

unread,
May 10, 2010, 6:21:05 PM5/10/10
to comm...@googlegroups.com
It is handled by the module loader.  So when you require('json') it finds json/index (and if not that, looks for json/json).

This fits with the current node.js behavior.

-C

Irakli Gozalishvili

unread,
May 10, 2010, 6:27:14 PM5/10/10
to comm...@googlegroups.com
I would love to eliminate fallback, since it will just complicate things on browser platform (which does not relies on build / backend).


--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands


Kris Zyp

unread,
May 10, 2010, 6:26:49 PM5/10/10
to comm...@googlegroups.com, Charles Jolley


On 5/10/2010 4:21 PM, Charles Jolley wrote:
> It is handled by the module loader. So when you require('json') it
> finds json/index (and if not that, looks for json/json).
>
> This fits with the current node.js behavior.

I'm assuming by "module loader" in this context you are referring to the
lookup on the file system, which is fine. A URI on the otherhand (which
is what the package mapping deals with), shouldn't to know what you are
using for your default module name, and how that maps to the system. The
web server or file loader implementation can certainly choose to handle
folder URIs with index.js, but I think that should be beyond the scope
of the concerns of package mapping specification.

Irakli Gozalishvili

unread,
May 10, 2010, 6:29:02 PM5/10/10
to comm...@googlegroups.com
Maybe I miss something here, but can we:

merge dependencies and mappings ?
can we have relative mappings for nested packages ?


--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands


Kris Zyp

unread,
May 10, 2010, 6:30:13 PM5/10/10
to comm...@googlegroups.com, Irakli Gozalishvili


On 5/10/2010 4:27 PM, Irakli Gozalishvili wrote:
> I would love to eliminate fallback, since it will just complicate
> things on browser platform (which does not relies on build / backend).
+1. Multiple lookups can easily move into exponential permutations and
terrible performance. The package mapping specification has been very
carefully crafted to unambiguously produce only a single URI for any
given module id.

Kris Zyp

unread,
May 10, 2010, 6:43:02 PM5/10/10
to comm...@googlegroups.com, Irakli Gozalishvili


On 5/10/2010 4:29 PM, Irakli Gozalishvili wrote:
> Maybe I miss something here, but can we:
>
> merge dependencies and mappings ?

Maybe, although I don't really know how. They are operated on completely
different concepts right now. "dependencies" works on named packages vs
and mappings works with URIs. Mappings don't really indicate any
package-wide dependency. If a mapping is not used (no require is made
that matches a particular mapping), there is no reason that the target
of the mapping needs to be available. Mappings explicitly specify how
the target can be located and how it can be required. Mappings don't
necessarily correspond to any package (could be to just a random URL on
the web, a nested folder in a package, etc). I'm open to ideas how they
could be merged, but I wouldn't want to complicate mappings too much.

> can we have relative mappings for nested packages ?

Following the standard rules for relative URIs (mapping targets can be
relative URIs and module ids follow the relative paths part), that
should be easy to do right now.

Charles Jolley

unread,
May 10, 2010, 6:47:52 PM5/10/10
to comm...@googlegroups.com
To me mappings is a fairly confusing concept that I think is less useful than simply allowing you to alias package names in the dependencies hash. i.e.

dependencies: {
"narwhal": {
"name": "narwhal-shim",
"version": "0.2.3"
}
}

Would allow you to require the package 'narwhal-shim' as 'narwhal' in your code. This is pretty straightforward.

I feel like the mappings hash is much more confusing. It is hard to predict just by looking at code what is going to happen thanks to the possibility of applying arbitrary remaps.

-C

Kris Zyp

unread,
May 10, 2010, 7:17:46 PM5/10/10
to comm...@googlegroups.com


On 5/10/2010 4:47 PM, Charles Jolley wrote:
> To me mappings is a fairly confusing concept that I think is less useful than simply allowing you to alias package names in the dependencies hash. i.e.
>
> dependencies: {
> "narwhal": {
> "name": "narwhal-shim",
> "version": "0.2.3"
> }
> }
>
> Would allow you to require the package 'narwhal-shim' as 'narwhal' in your code. This is pretty straightforward.
>

And where do you get narwhal-shim? This only works if your package is
lucky enough to be running on an environment that has the package
installed or has a package manager that is aware of that package. The
vast majority of modules are not registered in every package manager out
there (or even any), so this is a fail in the real world of
cross-platform applications with a diverse set of package managers (or
none at all on most systems). And this fails to address the need for
referencing folders in package (having to use names twice for packages
that are self-namespaced with nested folders is miserably error prone),
and single module referencing.
> I feel like the mappings hash is much more confusing. It is hard to predict just by looking at code what is going to happen thanks to the possibility of applying arbitrary remaps.
>
Mappings can be used as a single object hash. Everything I have seen
with more in depth dependencies involves complicated nested objects.
Mappings elegantly gives developers more power and portability with less
complexity.

Kris Zyp

unread,
May 10, 2010, 7:49:02 PM5/10/10
to comm...@googlegroups.com
I updated the package mapping proposal based on the discussions and suggestions for simplifying mappings. One does not need to suffix mappings with a slash for path-style mappings anymore. I defined the algorithm such that it is possible to explicitly choose path-based-only mappings and module-only mappings (or either). Also, there were two votes for using slashes as mapping delimiter character, and none otherwise, so I went with slashes. You can see the diff here:
http://wiki.commonjs.org/index.php?title=Packages/Mappings&diff=2768&oldid=2641

And the full proposal here:
http://wiki.commonjs.org/wiki/Packages/Mappings

Let me know if you think I got it wrong.

Thanks,
Kris

Christoph Dorn

unread,
May 11, 2010, 12:25:47 PM5/11/10
to comm...@googlegroups.com

Mappings are very different from dependencies and each solves a separate
concern as Kris alluded to below. I think we need both and they should
not be combined.

Christoph


Kris Zyp wrote:
>
> On 5/10/2010 4:29 PM, Irakli Gozalishvili wrote:
>> Maybe I miss something here, but can we:
>>
>> merge dependencies and mappings ?
>
> Maybe, although I don't really know how. They are operated on completely
> different concepts right now. "dependencies" works on named packages vs
> and mappings works with URIs. Mappings don't really indicate any
> package-wide dependency. If a mapping is not used (no require is made
> that matches a particular mapping), there is no reason that the target
> of the mapping needs to be available. Mappings explicitly specify how
> the target can be located and how it can be required. Mappings don't
> necessarily correspond to any package (could be to just a random URL on
> the web, a nested folder in a package, etc). I'm open to ideas how they
> could be merged, but I wouldn't want to complicate mappings too much.
>
>> can we have relative mappings for nested packages ?
>
> Following the standard rules for relative URIs (mapping targets can be
> relative URIs and module ids follow the relative paths part), that
> should be easy to do right now.
>

--

Christoph Dorn

unread,
May 11, 2010, 12:39:14 PM5/11/10
to comm...@googlegroups.com

I am pretty much happy with the proposal except for two items:

1) The "overlay" property should be in a separate proposal and the
mappings proposal should make reference to it. The overlay proposal
should begin to collect a list of "engines" that can be used as keys.
This is useful not only for the mappings proposal.

2) What happened to the mapping being restricted to one term only? IIRC
it was argued it would make it much more transparent as to what module
would be included. I believe this would address some of Charles'
transparency concerns.

What is the exact purpose of a multi-term mapping? Apart from providing
a nested namespace for mappings is the purpose to replace certain
modules far down the path tree? How does this work if the modules you
want to replace are relative-required from other modules which is likely
the typical case?

Christoph



Kris Zyp wrote:
> I updated the package mapping proposal based on the discussions and
> suggestions for simplifying mappings. One does not need to suffix
> mappings with a slash for path-style mappings anymore. I defined the
> algorithm such that it is possible to explicitly choose path-based-only
> mappings and module-only mappings (or either). Also, there were two
> votes for using slashes as mapping delimiter character, and none
> otherwise, so I went with slashes. You can see the diff here:
> http://wiki.commonjs.org/index.php?title=Packages/Mappings&diff=2768&oldid=2641
> <http://wiki.commonjs.org/index.php?title=Packages/Mappings&diff=2768&oldid=2641>
>
> And the full proposal here:
> http://wiki.commonjs.org/wiki/Packages/Mappings
>
> Let me know if you think I got it wrong.
>
> Thanks,
> Kris
> <http://wiki.commonjs.org/index.php?title=Packages/Mappings&diff=2768&oldid=2641>

Kris Zyp

unread,
May 11, 2010, 12:42:07 PM5/11/10
to comm...@googlegroups.com


On 5/11/2010 10:39 AM, Christoph Dorn wrote:
>
> I am pretty much happy with the proposal except for two items:
>
> 1) The "overlay" property should be in a separate proposal and the
> mappings proposal should make reference to it. The overlay proposal
> should begin to collect a list of "engines" that can be used as keys.
> This is useful not only for the mappings proposal.

I thought this was covered in Packages 1.1. Do we need a more extensive
overlay specification?

>
> 2) What happened to the mapping being restricted to one term only?
> IIRC it was argued it would make it much more transparent as to what
> module would be included. I believe this would address some of
> Charles' transparency concerns.

That's fine with me, I'm happy to add that restriction.

--
Thanks,
Kris

Christoph Dorn

unread,
May 11, 2010, 11:30:11 PM5/11/10
to comm...@googlegroups.com
Kris Zyp wrote:
> On 5/11/2010 10:39 AM, Christoph Dorn wrote:
>> I am pretty much happy with the proposal except for two items:
>>
>> 1) The "overlay" property should be in a separate proposal and the
>> mappings proposal should make reference to it. The overlay proposal
>> should begin to collect a list of "engines" that can be used as keys.
>> This is useful not only for the mappings proposal.
>
> I thought this was covered in Packages 1.1. Do we need a more extensive
> overlay specification?

It is indeed! I should have checked first. Sorry.

Christoph

George Moschovitis

unread,
May 12, 2010, 2:46:03 AM5/12/10
to CommonJS
> > require("json"); // getshttp://example.com/json-package/lib/index.js
> > <http://example.com/json-package/lib.js>
>
> I don't have any objections to the use of index.js as the default module
> for a folder.

-1

a default module is a bad idea, not really needed. and index.js is an
especially bad name.
a 'default' module is no index whatsoever.

-g.

jhuni

unread,
May 12, 2010, 5:42:28 AM5/12/10
to CommonJS
> all the existing suggestions are more or less compatible with mapping

The term "mappings" could mean any number of things...

Instead I advocate a system of "authorities" where as an authority
refers to a place in which a module can be obtained. (This is from the
Perl6 spec)

> why not put the mappings right in the dependency member

Indeed you should be able to do something like this:

{
"lang": "js",
"name": "foo",
"version": "0.0031",
"authority": "http://somewhere.com/somefile.js"
}

Or just:

"js-foo-0.0031-http://somewhere.com/somefile.js"

Also you should just be able to use "jsan" as an authority and it will
try to get the module from openjsan.org

> What would you prefer for a mapping delimiter character:

What's wrong with using dots? (Am I missing something here??)

Foo.Bar -> Foo/Bar.js
JooseX.Namespace.Dependended -> JooseX/Namespace/Dependended.js

Charles Jolley

unread,
May 12, 2010, 9:48:35 AM5/12/10
to comm...@googlegroups.com, CommonJS
Default modules are useful for packages. If just require('package-name') what module should I load?


Sent from my iPad

Tim Schaub

unread,
May 12, 2010, 3:50:32 PM5/12/10
to comm...@googlegroups.com
Charles Jolley wrote:
> Default modules are useful for packages. If just require('package-name') what module should I load?
>

package.json ~

{
"main": "__init__.js"
}

or "index.js" or whatever - left up to the package author.

Kris Kowal

unread,
May 12, 2010, 4:30:09 PM5/12/10
to comm...@googlegroups.com
On Wed, May 12, 2010 at 12:50 PM, Tim Schaub <tsc...@opengeo.org> wrote:
> Charles Jolley wrote:
>>
>> Default modules are useful for packages.  If just require('package-name')
>> what module should I load?
>
> package.json ~
>
> {
>    "main": "__init__.js"
> }
>
> or "index.js" or whatever - left up to the package author.

This is tentatively fine by me. Be clear about what path that's
resolved relative to. I favor resolving the path relative to
package.json.

Kris Kowal

Charles Jolley

unread,
May 12, 2010, 6:48:32 PM5/12/10
to comm...@googlegroups.com, comm...@googlegroups.com
Seems fine to me. If a developer does not specify a main property we should have a default.

The two most common right now are index.js and package-name.js Both seem about equal to me except that index.js does not require changing the filename if you alter the package name. I suppose you could also use main.js though that is a bit confusing if you are loading the package as a library and not an "executable".

Charles

Sent from my iPad

Christoph Dorn

unread,
May 13, 2010, 11:54:34 AM5/13/10
to comm...@googlegroups.com
Charles Jolley wrote:
> Seems fine to me. If a developer does not specify a main property we should have a default.
>
> The two most common right now are index.js and package-name.js Both seem about equal to me except that index.js does not require changing the filename if you alter the package name. I suppose you could also use main.js though that is a bit confusing if you are loading the package as a library and not an "executable".

I don't think there should be a default. It should always look for the
"main" property in package.json and if it does not exist throw an error.


>> On Wed, May 12, 2010 at 12:50 PM, Tim Schaub <tsc...@opengeo.org> wrote:
>>> Charles Jolley wrote:
>>>> Default modules are useful for packages. If just require('package-name')
>>>> what module should I load?
>>> package.json ~
>>>
>>> {
>>> "main": "__init__.js"
>>> }
>>>
>>> or "index.js" or whatever - left up to the package author.
>> This is tentatively fine by me. Be clear about what path that's
>> resolved relative to. I favor resolving the path relative to
>> package.json.

+1

Christoph

Christoph Dorn

unread,
May 13, 2010, 12:02:50 PM5/13/10
to comm...@googlegroups.com
jhuni wrote:
>> all the existing suggestions are more or less compatible with mapping
>
> The term "mappings" could mean any number of things...
>
> Instead I advocate a system of "authorities" where as an authority
> refers to a place in which a module can be obtained. (This is from the
> Perl6 spec)
>
>> why not put the mappings right in the dependency member

Mappings and dependencies are two different animals. This has been
explained further up in this thread.

Mappings map a one-term alias to a complete arbitrary URL that may
contain version segments as part of its path. It is not concerned with
packages but only linked modules. This can be used to quickly bootstrap
a bunch of components without a package manager (all you need is a
loader that supports dynamic loading of modules via mappings).

Dependencies are package declarations that need to be resolved/installed
by a package manager pulling the packages from catalogs for example. I
am advocating including URL locations of the catalog or package in the
descriptors while the current default is just a name.

Christoph

Zachary Carter

unread,
May 13, 2010, 12:03:20 PM5/13/10
to comm...@googlegroups.com
+1 for this algorithm:
1. check package.json for main property
2. check for index.js
3. check for package-name.js
4. throw error
--
Zach Carter

Kris Zyp

unread,
May 13, 2010, 12:37:37 PM5/13/10
to comm...@googlegroups.com


On 5/13/2010 10:03 AM, Zachary Carter wrote:
+1 for this algorithm:
1. check package.json for main property
2. check for index.js
3. check for package-name.js
4. throw error

The package mappings has been carefully designed to *never* introduce multiple lookups. This greatly simplifies implementation and improves performance. Introducing multiple looks can easily permutate to a numerous lookups with big performance penalties, and is particularly unacceptable in the browser. I would look to continue to avoid multiple lookups. Does the main module only apply to the package root, or does it apply to every folder in a package? With the latter, even with a known main module name, this seems to create situation where you have to go to the disk first to check to see if a directory exists, and then decide which module to look up from other (doubling every lookup).

Also, I would point out that the "main" module concept only really makes sense on file systems/directory structures. When a module id is mapped to a plain HTTP URL, there is no way to know if http://site.com/path/to/foo is directory and should be changed to http://site.com/path/to/foo/index.js instead of http://site.com/path/to/foo.js.

Personally this all seems more complicated than worth it for the value it adds (typing "package/index.js" isn't terribly hard), but I wouldn't stand in the way if everyone wants it added.

Kris
-- 
Thanks,
Kris

Zachary Carter

unread,
May 13, 2010, 1:07:04 PM5/13/10
to comm...@googlegroups.com
On Thu, May 13, 2010 at 12:37 PM, Kris Zyp <kri...@gmail.com> wrote:


On 5/13/2010 10:03 AM, Zachary Carter wrote:
+1 for this algorithm:
1. check package.json for main property
2. check for index.js
3. check for package-name.js
4. throw error

The package mappings has been carefully designed to *never* introduce multiple lookups. This greatly simplifies implementation and improves performance. Introducing multiple looks can easily permutate to a numerous lookups with big performance penalties, and is particularly unacceptable in the browser.
 
Great incentive for authors to always declare the default in package.json. The others checks are a compromise with other suggestions in this thread and interoperability with some existing idioms. I just wanted to throw it out there.
 
I would look to continue to avoid multiple lookups. Does the main module only apply to the package root, or does it apply to every folder in a package?
 
My comment was toward resolving the default package module.
 
With the latter, even with a known main module name, this seems to create situation where you have to go to the disk first to check to see if a directory exists, and then decide which module to look up from other (doubling every lookup).

Also, I would point out that the "main" module concept only really makes sense on file systems/directory structures. When a module id is mapped to a plain HTTP URL, there is no way to know if http://site.com/path/to/foo is directory and should be changed to http://site.com/path/to/foo/index.js instead of http://site.com/path/to/foo.js.

Personally this all seems more complicated than worth it for the value it adds (typing "package/index.js" isn't terribly hard), but I wouldn't stand in the way if everyone wants it added.

Agreed, I would happily abide as a package author. If others find value in a common default, however, I like the multistep approach.

Kris Kowal

unread,
May 13, 2010, 1:47:57 PM5/13/10
to comm...@googlegroups.com
I could buy "index.js" being the default value. If that's the case,
it might make more sense for the corresponding package.json property
to be "index". I could buy a proposal with an alternate name for
"index", but I also see how that would put some prototype packages out
of conformance out the starting gate with little reason to
substantiate the change. I could also buy a proposal that requires
explicit mention of an index if it exists; this has the advantage of
not mandating the existence of index.js. In fact, I prefer this
approach.

Having "index.js" fallback to (descriptor.name + ".js") does not jive
well with me, although my reasons are different than Kris Zyp's. It
seems to me that the purpose of a standard is to reduce the need for
complexity. The proposal of falling back is perhaps sound for
implementations attempting to assimilate non-standard packages, I
don't think it's sound for encouraging convergence on a single, simple
standard.

Kris Kowal

Irakli Gozalishvili

unread,
May 13, 2010, 3:52:53 PM5/13/10
to Kris Zyp, comm...@googlegroups.com

- I'm for having index property in package.json, which will default to index.js if not defined in package.json
- Since majority of votes are for the first option in "/" vs ":" battle, I'm buying it, but would've prefer 2nd option.

I don't think still don't agree that mappings and dependencies are different animals and if they are I'd like to have clear explanation (also in specs), since I think it can be a source for a lot of confusion.


--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands


On Thu, May 13, 2010 at 21:43, Irakli Gozalishvili <rfo...@gmail.com> wrote:
Sorry for delay on this one:


On Tue, May 11, 2010 at 00:43, Kris Zyp <kri...@gmail.com> wrote:


On 5/10/2010 4:29 PM, Irakli Gozalishvili wrote:
> Maybe I miss something here, but can we:
>
> merge dependencies and mappings ?

Maybe, although I don't really know how. They are operated on completely
different concepts right now. "dependencies" works on named packages vs
and mappings works with URIs. Mappings don't really indicate any
package-wide dependency. If a mapping is not used (no require is made
that matches a particular mapping), there is no reason that the target
of the mapping needs to be available.

I don't think they are that different, since in a current spec "A CommonJS package is a cohesive wrapping of a collection of modules, code and other assets into a single form", in my understanding aliased URI is the same, even though collection might contain just a single asset.

I also don't see any point in declaring mapping if declaring package is not using it (there for depends on it). I also was assuming that package managers will cache mapped url's locally so that it can be aliased by any package to any name.
 
Mappings explicitly specify how
the target can be located and how it can be required. Mappings don't
necessarily correspond to any package (could be to just a random URL on
the web, a nested folder in a package, etc).

I would interpret it a single module dependency, and it doesn't matters where is it coming from
 
I'm open to ideas how they
could be merged, but I wouldn't want to complicate mappings too much.


I think I posted example before and it looked like this

"dependencies": {
    "narwhal-lib": { // alias following same naming convention as module id's except `/`
        "uri": "http://github.com/kriskowal/narwhal-lib/",  // explicit mapping

        "version": "0.2"                                    // does verision makes sence with expliciting mapping ?
    },

    "foo": {    // alias
        "id": "foo-bar-pack",   // package installed in platform with the matching name
        "version": ["1.0", "2.0"] //(optional ?
platform preference) version
    }
}


Guess version property makes no sense in mapping which is first example. This also makes it easy to switch from mapped dependency used in dev process to a named package in production, even though I think that with a well designed package manager mapping can fully replace named packages, since name will can be url and it's guaranteed to be unique across all platforms, package managers, etc.
 
> can we have relative mappings for nested packages ?

Following the standard rules for relative URIs (mapping targets can be
relative URIs and module ids follow the relative paths part), that
should be easy to do right now.

--
Thanks,
Kris

Irakli Gozalishvili

unread,
May 13, 2010, 3:43:59 PM5/13/10
to Kris Zyp, comm...@googlegroups.com
Sorry for delay on this one:


On Tue, May 11, 2010 at 00:43, Kris Zyp <kri...@gmail.com> wrote:


On 5/10/2010 4:29 PM, Irakli Gozalishvili wrote:
> Maybe I miss something here, but can we:
>
> merge dependencies and mappings ?

Maybe, although I don't really know how. They are operated on completely
different concepts right now. "dependencies" works on named packages vs
and mappings works with URIs. Mappings don't really indicate any
package-wide dependency. If a mapping is not used (no require is made
that matches a particular mapping), there is no reason that the target
of the mapping needs to be available.
I don't think they are that different, since in a current spec "A CommonJS package is a cohesive wrapping of a collection of modules, code and other assets into a single form", in my understanding aliased URI is the same, even though collection might contain just a single asset.

I also don't see any point in declaring mapping if declaring package is not using it (there for depends on it). I also was assuming that package managers will cache mapped url's locally so that it can be aliased by any package to any name.
 
Mappings explicitly specify how

the target can be located and how it can be required. Mappings don't
necessarily correspond to any package (could be to just a random URL on
the web, a nested folder in a package, etc).
I would interpret it a single module dependency, and it doesn't matters where is it coming from
 
I'm open to ideas how they

could be merged, but I wouldn't want to complicate mappings too much.


I think I posted example before and it looked like this

"dependencies": {
    "narwhal-lib": { // alias following same naming convention as module id's except `/`
        "uri": "http://github.com/kriskowal/narwhal-lib/",  // explicit mapping
        "version": "0.2"                                    // does verision makes sence with expliciting mapping ?
    },

    "foo": {    // alias
        "id": "foo-bar-pack",   // package installed in platform with the matching name
        "version": ["1.0", "2.0"] //(optional ?
platform preference) version
    }
}


Guess version property makes no sense in mapping which is first example. This also makes it easy to switch from mapped dependency used in dev process to a named package in production, even though I think that with a well designed package manager mapping can fully replace named packages, since name will can be url and it's guaranteed to be unique across all platforms, package managers, etc.
 
> can we have relative mappings for nested packages ?

Following the standard rules for relative URIs (mapping targets can be
relative URIs and module ids follow the relative paths part), that
should be easy to do right now.

--
Thanks,
Kris

Irakli Gozalishvili

unread,
May 17, 2010, 7:35:28 PM5/17/10