[CommonJS] packages

[Irakli Gozalishvili]

Hi

I thought I'll try to push this topic once again, since we are pretty close to an agreement (I think), since all the existing suggestions are more or less compatible with mapping proposal from KrisZ.

- Is there any point of keeping package dependency listings in package.json? Mapping expresses dependencies explicitly I believe.

- Can mapping be used for providing an overlays as well for example
mappings: {'foo/bar': 'http://myhost.com/lib/bar.js'}   // where require('foo/bar') requires http://myhost.com/lib/bar.js

- @Charles can we make seedjs compatible with Kris's proposal ? Cause one issue I think of is illustrated below
mappings: {'foo:': 'http://myhost.com/lib/'}  // to require http://myhost.com/lib/bar.js
seed will expect require('foo:bar) while other platforms will expect require('foo:/bar)

Is there any other open questions ?

KrisZ Mind writing proposal since you're an author (Sorry if you already did so, can't find it on a wiki) ?
--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

--
You received this message because you are subscribed to the Google Groups "CommonJS" group.
To post to this group, send email to comm...@googlegroups.com.
To unsubscribe from this group, send email to commonjs+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/commonjs?hl=en.

[Charles Jolley]

- @Charles can we make seedjs compatible with Kris's proposal ? Cause one issue I think of is illustrated below
mappings: {'foo:': 'http://myhost.com/lib/'}  // to require http://myhost.com/lib/bar.js
seed will expect require('foo:bar) while other platforms will expect require('foo:/bar)

I don't mind making seed compatible, but I have some concern still about using / as the package separator.  The problem is:

require('foo/bar/baz') 

Is ambiguous.  This could be:

'foo/bar/baz' module from the current package

'baz' module from the 'foo/bar' package

'bar/baz' from the 'foo' package

'foo/bar/baz' from the underlying system (i.e. to remain compatible with node for example, you have to fallback to looking for a simple module outside of packages)

Even the mappings as proposed by Kris doesn't solve this.  What if I have defined:

mappings: {

  "foo":  "...",

  "foo/bar": "..."

}

We basically end up back where we started - looking at a require() statement it is not clear what should be loaded (and any rules we come up with in the spec are going to be complex enough that implementors could easily screw them up).

On the other hand:

require('foo:bar/baz');

require('foo/bar:baz');

require('foo/bar/baz');

is far less ambiguous.  The last one is technically still ambiguous but less likely to be so since package cases are removed.

How would you address this ambiguous scenario with mappings?

-C

Is there any other open questions ?

KrisZ Mind writing proposal since you're an author (Sorry if you already did so, can't find it on a wiki) ?
--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

--
You received this message because you are subscribed to the Google Groups "CommonJS" group.
To post to this group, send email to comm...@googlegroups.com.
To unsubscribe from this group, send email to commonjs+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/commonjs?hl=en.

[Kris Zyp]

On 5/3/2010 4:54 PM, Irakli Gozalishvili wrote:

Hi

I thought I'll try to push this topic once again, since we are pretty close to an agreement (I think), since all the existing suggestions are more or less compatible with mapping proposal from KrisZ.

- Is there any point of keeping package dependency listings in package.json? Mapping expresses dependencies explicitly I believe.

- Can mapping be used for providing an overlays as well for example
mappings: {'foo/bar': 'http://myhost.com/lib/bar.js'}   // where require('foo/bar') requires http://myhost.com/lib/bar.js

Yes, and there is an example of using overlays in the proposal:
http://wiki.commonjs.org/wiki/Packages/Mappings

- @Charles can we make seedjs compatible with Kris's proposal ? Cause one issue I think of is illustrated below
mappings: {'foo:': 'http://myhost.com/lib/'}  // to require http://myhost.com/lib/bar.js
seed will expect require('foo:bar) while other platforms will expect require('foo:/bar)

This should already be compatible. With the mapping you defined, require('foo:bar')  should map to http://myhost.com/lib/bar.js. Doing require('foo:/bar') would map to http://myhost.com/lib//bar.js.

Is there any other open questions ?

I guess Charles is asking them right now...

KrisZ Mind writing proposal since you're an author (Sorry if you already did so, can't find it on a wiki) ?

http://wiki.commonjs.org/wiki/Packages/Mappings

--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands
--
You received this message because you are subscribed to the Google Groups "CommonJS" group.
To post to this group, send email to comm...@googlegroups.com.
To unsubscribe from this group, send email to commonjs+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/commonjs?hl=en.

-- 
Thanks,
Kris

[Tim Schaub]

Charles Jolley wrote:
>> - @Charles can we make seedjs compatible with Kris's proposal ? Cause one issue I think of is illustrated below
>> mappings: {'foo:': 'http://myhost.com/lib/'} // to require http://myhost.com/lib/bar.js
>> seed will expect require('foo:bar) while other platforms will expect require('foo:/bar)
>
> I don't mind making seed compatible, but I have some concern still about using / as the package separator. The problem is:
>
> require('foo/bar/baz')
>
> Is ambiguous. This could be:
>
> 'foo/bar/baz' module from the current package
> 'baz' module from the 'foo/bar' package
> 'bar/baz' from the 'foo' package
> 'foo/bar/baz' from the underlying system (i.e. to remain compatible with node for example, you have to fallback to looking for a simple module outside of packages)
>
> Even the mappings as proposed by Kris doesn't solve this. What if I have defined:
>
> mappings: {
> "foo": "...",
> "foo/bar": "..."
> }
>

How about specifying that aliases are a single term (only word chars)?

And (with apologies if this has been covered), why not put the mappings
right in the dependency member (assuming the order is not significant)?

"dependencies": {
"package-alias": "package-identifier"
}

Or, to allow more detail to be conveyed:

"dependencies": {
"narwhal-lib": {
"id": "http://github.com/kriskowal/narwhal-lib",
"version": "0.2"
}
}


Tim

[Kris Kowal]

On Mon, May 3, 2010 at 4:03 PM, Charles Jolley <cha...@sproutit.com> wrote:
> How would you address this ambiguous scenario with mappings?

I would describe this problem not as "ambiguous" but as "flexible", as
long as the behavior is well defined by the specification. My
understanding is that the mappings would be tried first, in order, for
the container package, falling back to the unspecified native module
search. It might be good to change the mappings to an array for that
reason, although object literals can be depended upon in JavaScript to
be enumerated in definition order — this would not be the case for
alternate language tooling with the same JSON.

Kris Kowal

[Kris Zyp]

On 5/3/2010 5:03 PM, Charles Jolley wrote:

- @Charles can we make seedjs compatible with Kris's proposal ? Cause one issue I think of is illustrated below
mappings: {'foo:': 'http://myhost.com/lib/'}  // to require http://myhost.com/lib/bar.js
seed will expect require('foo:bar) while other platforms will expect require('foo:/bar)

I don't mind making seed compatible, but I have some concern still about using / as the package separator.  The problem is:

require('foo/bar/baz') 

Is ambiguous.  This could be:

'foo/bar/baz' module from the current package

'baz' module from the 'foo/bar' package

'bar/baz' from the 'foo' package

'foo/bar/baz' from the underlying system (i.e. to remain compatible with node for example, you have to fallback to looking for a simple module outside of packages)

Even the mappings as proposed by Kris doesn't solve this.  What if I have defined:

mappings: {

  "foo":  "...",

  "foo/bar": "..."

}

We basically end up back where we started - looking at a require() statement it is not clear what should be loaded (and any rules we come up with in the spec are going to be complex enough that implementors could easily screw them up).

Actually, this is explicitly defined in the spec:
"The mapping with the longest property in the mappings object that matches should be applied, and subsequent mapping objects may then be ignored."
There is no ambiguity in how mappings are to be calculated. Of course you can argue that one can create confusing mappings ;).

On the other hand:

require('foo:bar/baz');

require('foo/bar:baz');

require('foo/bar/baz');

is far less ambiguous.  The last one is technically still ambiguous but less likely to be so since package cases are removed.

How would you address this ambiguous scenario with mappings?

As I have noted before, package mappings are completely with the seedjs convention. It is perfectly legal to do:
mappings:{
  "foo:": "...",
  "foo/bar:": "...",
 ..
The use of slashes was just what I used in the example because I thought it was the most likely convention. I'd be happy to change the examples if we want a different convention. I've noted my concern with the use of colon as a package delimiter being confusing with URI protocol syntax, but this is technically overcome-able, and I could live with colons if need be.

However, I think what you are really getting at is that users shouldn't be expected to include the package delimiter in the mapping key, and that the package delimiter should always be inferred. This is a reasonable objection. But, I believe this greatly limits the flexibility of simple string substitution or regex based model. I think the current design is elegant in that it can be used for package mapping, overlay mapping, mapping to nested folders within packages, and even mapping single path/identifers to nested paths. All of these various forms of mappings I am currently enjoying being able to use with the mappings impl I wrote [1]. If all we cared about was just mapping to the top level of packages, inferring package delimiters would be fine, but there is so much more we can do with mappings.

[1] http://github.com/kriszyp/nodules

-- 
Thanks,
Kris

[Kris Zyp]

And to address another open question that I had asked earlier on...
Having implemented the package mappings proposal, I have come to the
conclusion that treating the mappings as regex replacements (with
inferred ^ start string anchor) rather than plain string substitutions
is more flexible, and actually slightly easier to implement. In
particular this makes it safer to write mappings for single identifier
module ids:
mappings:{
"model$": "..." # only matches "model", not "model/something",
"model/": "..." # matches anything that starts with "model/",

Kris

[Kris Kowal]

On Mon, May 3, 2010 at 4:34 PM, Kris Zyp <kri...@gmail.com> wrote:
> And to address another open question that I had asked earlier on...
> Having implemented the package mappings proposal, I have come to the
> conclusion that treating the mappings as regex replacements (with
> inferred ^ start string anchor) rather than plain string substitutions
> is more flexible, and actually slightly easier to implement. In
> particular this makes it safer to write mappings for single identifier
> module ids:

This however mixes poorly with the "longest expression first" rule.

Kris Kowal

[Kris Zyp]

On 5/3/2010 5:36 PM, Kris Kowal wrote:
> On Mon, May 3, 2010 at 4:34 PM, Kris Zyp <kri...@gmail.com> wrote:
>
>> And to address another open question that I had asked earlier on...
>> Having implemented the package mappings proposal, I have come to the
>> conclusion that treating the mappings as regex replacements (with
>> inferred ^ start string anchor) rather than plain string substitutions
>> is more flexible, and actually slightly easier to implement. In
>> particular this makes it safer to write mappings for single identifier
>> module ids:
>>
> This however mixes poorly with the "longest expression first" rule.
>

You are probably right, and I can think of some contrived cases where
longer regex strings would be less specific, but do you have an example
of how this would be really problematic/confusing in useful mappings?

--
Thanks,
Kris

[James Burke]

On Mon, May 3, 2010 at 4:12 PM, Kris Zyp <kri...@gmail.com> wrote:
> http://wiki.commonjs.org/wiki/Packages/Mappings

Kris Zyp, how would names be matched in this scenario?

mappings: {
"foo": "some/path/to/foo"
}

then I do a require("foo") and a require("foobar")? Does the mapping
above apply to both?

I am hoping it only applies to require("foo") and not
require("foobar"). I have found it useful to have a mapping like so:

mappings: {
"foo": "some/path/v2/foo"
}

where I want just require("foo") to be resolved to
"some/path/v2/foo.js". I would not want require("foobar") mapping to
"some/path/v2/foobar.js"

I also like being able to map all modules "under" foo to a directory:

require("foo/baz") -> "some/path/v2/foo/baz.js"
require("foobar") -> "possible/other/path/foobar.js"

I appreciate I could do the last one with mappings for a "foo/", but I
liked in general that a value without a slash indicated a first level
"segment" in a module ID. It is also not clear how I would do the case
where I want "foo" to be mapped to "foo.js" and "foo/bar" mapped to
"foo/bar.js" with just one mapping entry. My first impression of
needing a separate rule that uses a regexp for a "$" on the end
strikes me as odd, but then I am coming from the way we did mappings
in Dojo too.

My general question could be restated as, why not use the mapping
algorithm that Dojo uses (except use "/" instead of ".")?

James

[Kris Kowal]

Perhaps we should go back to the simplest form of this proposal and
figure out what we buy by generalizing, and see whether the
generalization is worth paying for.

Suppose that mappings map ID "directories" to package lib directories
and nothing more. Thus "foo/bar/baz" could be affected by either a
"foo" or a "foo/bar" mapping, but no other. Mappings would not employ
regular expressions. We can use the longest-pattern-applies-first
algorithm without any uncertainty or hazards. Subtrees can be
subverted transparently, if needed.

The weakness of this proposal (and even more flexible proposals) is
that the package prefix does not map to any particular module. NPM
resolves this by associating the package prefix with an "index.js" in
the root of the package. For example, if "foo/bar" is mapped to the X
package, X/index.js would be returned by require("foo/bar"). I am
indifferent to this approach.

If we use regexen, we open the possibility of using any delimiter at
all to distinguish self-packaged modules from sub-packaged modules. I
don't see an advantage of introducing arbitrary delimiters,
particularly since this might reduce our options for standardizing
extensions to the module identifier name space.

I'm, in any case, in favor of keeping things simple. I could vote
"yes" on mapping dirs to dirs with an implied "/" delimiter between
right now. I would take some convincing for regexen or alternate
delimiters.

Kris Kowal

[Charles Jolley]

Based on my experience writing the SproutCore build tools three times now (including the dependency system), I have found that dependencies are an endless source of grief for most developers. Therefore it is important to keep the rules very simple and easy to understand.

For example:

require('foo/bar/baz') will get the bar/baz module from whatever package is mapped to foo unless you have listed foo/bar in your mappings, in which case it will get the 'baz' module from whatever packages is mapped to foo/bar. It will load the latest version of the package mapped to foo or foo/bar in the mappings hash as listed in the dependencies hash.

Even the simplest proposal for mapping is pretty complicated!

Maybe the proposal that we extend the dependencies hash (as proposed by Tim Schaub) and limit package names to the first term is the best? i.e.

require('foo/bar/baz') will get the bar/baz module from the packaged mapped to 'foo' in the dependencies hash. You can name a package, location, and version in the dependencies hash.

Seems pretty simple. Does this still meet the requirements for mappings?

-C

[Kris Kowal]

On Mon, May 3, 2010 at 9:20 PM, Charles Jolley <cha...@sproutit.com> wrote:

> Maybe the proposal that we extend the dependencies hash (as proposed
> by Tim Schaub) and limit package names to the first term is the
> best? i.e.
>
> require('foo/bar/baz') will get the bar/baz module from the packaged
> mapped to 'foo' in the dependencies hash.  You can name a package,
> location, and version in the dependencies hash.
>
> Seems pretty simple.  Does this still meet the requirements for
> mappings?

I think that it would be a good start. It's certainly the option we
would be least likely to regret. We could surely extend the system in
a future version as we collectively gain more experience.

[Irakli Gozalishvili]

Please let's exclude fallback scenarios since it's browser platform unfriendly. I do understand that this is flexibility but it means either you use tooling or server support for browser development or you cannot rely on commonjs packages. Think that tooling enforcement is not something that commonjs should do. So disambiguity would be nice.

I also would like to see merge of mappings with dependencies

--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

[Irakli Gozalishvili]

Cool I also like @Tim's proposal a lot!! It also addresses all the concerns I had with other proposals.

I think proposal can be extended further by making it like this, where either id or uri should be presented.
 
"dependencies": {
    "narwhal-lib": { // alias following same naming convention as module id's except `/`
        "uri": "http://github.com/kriskowal/narwhal-lib/",  // explicit mapping
        "version": "0.2"                                    // does verision makes sence with expliciting mapping ?
    },
    "foo": {    // alias
        "id": "foo-bar-pack",   // package installed in platform with the matching name
        "version": ["1.0", "2.0"] //(optional ? platform preference) version
    }
}

In this case we'll end up with a simple extension to a current packages

--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

[Kris Zyp]

On 5/3/2010 6:32 PM, James Burke wrote:
> On Mon, May 3, 2010 at 4:12 PM, Kris Zyp <kri...@gmail.com> wrote:
>
>> http://wiki.commonjs.org/wiki/Packages/Mappings
>>
> Kris Zyp, how would names be matched in this scenario?
>
> mappings: {
> "foo": "some/path/to/foo"
> }
>
> then I do a require("foo") and a require("foobar")? Does the mapping
> above apply to both?
>

Yes, with the current specified algorithm it would apply to both.

> I am hoping it only applies to require("foo") and not
> require("foobar"). I have found it useful to have a mapping like so:
>
> mappings: {
> "foo": "some/path/v2/foo"
> }
>
> where I want just require("foo") to be resolved to
> "some/path/v2/foo.js". I would not want require("foobar") mapping to
> "some/path/v2/foobar.js"
>
> I also like being able to map all modules "under" foo to a directory:
>
> require("foo/baz") -> "some/path/v2/foo/baz.js"
> require("foobar") -> "possible/other/path/foobar.js"
>
> I appreciate I could do the last one with mappings for a "foo/", but I
> liked in general that a value without a slash indicated a first level
> "segment" in a module ID. It is also not clear how I would do the case
> where I want "foo" to be mapped to "foo.js" and "foo/bar" mapped to
> "foo/bar.js" with just one mapping entry. My first impression of
> needing a separate rule that uses a regexp for a "$" on the end
> strikes me as odd, but then I am coming from the way we did mappings
> in Dojo too.
>
> My general question could be restated as, why not use the mapping
> algorithm that Dojo uses (except use "/" instead of ".")?
>

Hmm, the Dojo algorithm, that sounds like something I should know about
:). Could you share it?

Anyway, I totally agree that we need to meet the use cases you shared.
So far I haven't heard any alternates to the search & replace (plain
string or regex) algorithm that would meet them (Tim's proposal can't
map to individual modules or won't work with ad-hoc folder structures,
so it won't do). However, I certainly don't have my heart set on the
current algorithm, so I would be eager to hear other ideas.

--
Thanks,
Kris

[Christoph Dorn]

Suppose we limit the package aliases to the first term:

"require('foo/bar/baz') will get the bar/baz module from the package

mapped to 'foo'"

We can map packages with:

"mappings": {
"foo": { // alias
"uri": "http://github.com/foo/...",
"version": "0.2"
}
}

To map to a specific module you could now do:

"mappings": {
"foo": { // alias
"uri": "http://github.com/foo/...",
"version": "0.2",
"module": "specific-foo"
}
}

require("foo")

This could be used to overwrite system modules as well.

I too have found the need to map to a module in package.json and think
it is a necessary feature.

I am adamant about needing a clear separation between the package
component and module component in a module ID. To achieve this for
one-arg require the package must *always* be prefixed even for system
modules.

require("system/system")
require("commonjs/system")
require("foo/bar")
require("foo") -> special case (mapped to module as described above)

This would necessitate the platform providing some default packages such
as "system" or "commonjs" or the package.json needs to map *all* packages.

The reason I like the two-arg require is that it provides in my mind
more flexibility.

require("system") -> the "system" module from the system namespace
require("system", "commonjs") -> the "system" module from the "commonjs"
package

*Any* package can install modules into the system namespace by declaring
them in the "dependencies" package.json property while "packages" get
defined in the "packages" property.

Mappings in this case would be used to map modules in the system
namespace. This can be used to organize modules for transport to the
browser with a browser loader that re-maps them to the two-arg require
when they arrive at the client. I have not had a chance to play with
that but believe it is possible and fairly trivial.

I see the module system at several levels:

Level 0: bootstrapping
Level 1: 1 arg require global namespace module system
Level 2: 2 arg require or other aliased package-based module systems
Level 3: package spaces with dependencies mapped in package.json

Level 0 and 1 would be platform specific while level 2 and 3 would be
interoperable across all platforms.

I presume we all have the goal of writing programs split into packages
that run on a variety of platforms. I am also observing that
implementors have their own ideas of how they believe packages should
operate on their platforms.

Maybe our focus should remain at level 1 with the exception of
specifying a common package.json format that works for all package
managers and loaders. I think we are close in that regard. In my mind
this should suffice to achieve interoperability because when a package
loads it can load a module loader to load its own dependencies from
whatever package ecosystem it desires. I see this as no different to a
website running prototype, jquery and dojo in parallel each with their
own plugin/module systems.

Over time I see certain package loaders succeeding likely primarily
based on the strength of the community surrounding them, not based on
the specifics of how they operate or their require syntax.

Christoph

Irakli Gozalishvili wrote:
> Cool I also like @Tim's proposal a lot!! It also addresses all the
> concerns I had with other proposals.
>
> I think proposal can be extended further by making it like this, where
> either id or uri should be presented.
>

> *"dependencies": {

> "narwhal-lib": { // alias following same naming convention as module
> id's except `/`
> "uri": "http://github.com/kriskowal/narwhal-lib/", // explicit
> mapping
> "version": "0.2" // does
> verision makes sence with expliciting mapping ?
> },
> "foo": { // alias
> "id": "foo-bar-pack", // package installed in platform with
> the matching name

> "version": ["1.0", "2.0"] //(optional ? **platform preference**)
> version
> }
> }
> *

> <mailto:comm...@googlegroups.com>.

> To unsubscribe from this group, send email to
> commonjs+u...@googlegroups.com

> <mailto:commonjs%2Bunsu...@googlegroups.com>.

[Kris Zyp]

On 5/4/2010 10:25 AM, Christoph Dorn wrote:

Suppose we limit the package aliases to the first term:

"require('foo/bar/baz') will get the bar/baz module from the package mapped to 'foo'"

We can map packages with:

"mappings": {
    "foo": { // alias
        "uri": "http://github.com/foo/...",
        "version": "0.2"
    }
}

To map to a specific module you could now do:

"mappings": {
    "foo": { // alias
        "uri": "http://github.com/foo/...",
        "version": "0.2",
        "module": "specific-foo"
    }
}

require("foo")

This could be used to overwrite system modules as well.

I too have found the need to map to a module in package.json and think it is a necessary feature.

This strategy sounds good, although I think we can do all of this without adding new properties:
"mapping":{
  "base64": "http://github.com/kriskowal/narwhal-lib/raw/narwhal-lib/lib/base64.js",
  "jack": http://github.com/tlrobinson/jack/raw/0.3/lib/jack/",
 ...

We can determine that the first mapping defines a module, so we can do require("base64"). The second one ends with a slash, so we could determine that it is mapping to a folder, so we could infer the package delimiter in the mapping, to do require("jack/querystring") (or require("jack:querysting") to follow Charles preferred package delimiter). A couple other important things to notice about the second mapping. First, Jack has all of its modules in side it's lib/jack/ directory. If we can only map to packages themselves, we would have to write jack twice for each require (like require("jack/jack/querystring")) in every module which is far more error prone than even the most complicated rule set for a single package.json mapping. It is critical that we are mapping to URIs is the foundational concept, packages work within that scheme. Also, URIs easily allow us to indicate versioning, specifically in a way that indicates how to locate the specified version. In the example, I am linking to version 0.3 (branch).

I am adamant about needing a clear separation between the package component and module component in a module ID. To achieve this for one-arg require the package must *always* be prefixed even for system modules.

require("system/system")
require("commonjs/system")
require("foo/bar")
require("foo") -> special case (mapped to module as described above)

This would necessitate the platform providing some default packages such as "system" or "commonjs" or the package.json needs to map *all* packages.

The reason I like the two-arg require is that it provides in my mind more flexibility.

require("system") -> the "system" module from the system namespace
require("system", "commonjs") -> the "system" module from the "commonjs" package

*Any* package can install modules into the system namespace by declaring them in the "dependencies" package.json property while "packages" get defined in the "packages" property.

Mappings in this case would be used to map modules in the system namespace. This can be used to organize modules for transport to the browser with a browser loader that re-maps them to the two-arg require when they arrive at the client. I have not had a chance to play with that but believe it is possible and fairly trivial.

I see the module system at several levels:

Level 0: bootstrapping
Level 1: 1 arg require global namespace module system
Level 2: 2 arg require or other aliased package-based module systems
Level 3: package spaces with dependencies mapped in package.json

Level 0 and 1 would be platform specific while level 2 and 3 would be interoperable across all platforms.

I presume we all have the goal of writing programs split into packages that run on a variety of platforms. I am also observing that implementors have their own ideas of how they believe packages should operate on their platforms.

Maybe our focus should remain at level 1 with the exception of specifying a common package.json format that works for all package managers and loaders. I think we are close in that regard. In my mind this should suffice to achieve interoperability because when a package loads it can load a module loader to load its own dependencies from whatever package ecosystem it desires. I see this as no different to a website running prototype, jquery and dojo in parallel each with their own plugin/module systems.

I agree, separating a package.json-aware module loader from the bootstrap module system is very reasonable.

-- 
Thanks,
Kris

[Tim Schaub]

Right. My thinking previously (that didn't go over well) was
require("js/file") etc.

The advantage of this is that a package could specify what flavor of js
they depend on.

If going with require("package-alias/module-id") is too verbose, I
assume a module loader could allow a default package to be specified
(e.g. narwhal for narwhal). Then require("file") would load the file
module from the default package if a package with the alias "file" is
not specified in the current package dependencies.

The default package (or root or base or whatever) could also be
specified in package.json.

Tim

[Christoph Dorn]

> <http://github.com/tlrobinson/jack/raw/master/lib/jack/>

> ...
>
> We can determine that the first mapping defines a module, so we can do
> require("base64"). The second one ends with a slash, so we could
> determine that it is mapping to a folder, so we could infer the package
> delimiter in the mapping, to do require("jack/querystring") (or
> require("jack:querysting") to follow Charles preferred package
> delimiter). A couple other important things to notice about the second
> mapping. First, Jack has all of its modules in side it's lib/jack/
> directory. If we can only map to packages themselves, we would have to
> write jack twice for each require (like
> require("jack/jack/querystring")) in every module which is far more
> error prone than even the most complicated rule set for a single
> package.json mapping. It is critical that we are mapping to URIs is the
> foundational concept, packages work within that scheme. Also, URIs
> easily allow us to indicate versioning, specifically in a way that
> indicates how to locate the specified version. In the example, I am
> linking to version 0.3 (branch).

Ok, I think I finally got the idea behind the "mapping" property. Keep
it simple and for pointing to URIs only and use the expanded package
locators for the "dependencies" and "packages" properties with
significant more flexibility so you can do for example:

{
"packages": {
"jack": { // a package locator object
"locator": "http://github.com/tlrobinson/jack/zipball/master/",
"prefix": "jack"
}
}
}

Which assumes jack's package.json specifies modules are in the "lib"
directory and "prefix" is added before module ids are resolved as one
example so you can do:

require("jack:querystring");

instead of

require("jack:jack/querystring");

> I agree, separating a package.json-aware module loader from the
> bootstrap module system is very reasonable.

So now the question is what properties the package locators should
support to allow for interoperability between package managers.

Christoph

[James Burke]

On Tue, May 4, 2010 at 6:41 AM, Kris Zyp <kri...@gmail.com> wrote:
> Hmm, the Dojo algorithm, that sounds like something I should know about
> :). Could you share it?

Dojo, and RequireJS, use a "longest match wins" but only allow
segmented paths (in CommonJS-land, the paths are segmented by "/"), no
regexps, and the mappings start from the beginning of the module IDs:

mappings: {
"foo/bar": "path/to/foo",
"foo/bar/baz": "different/path/to/baz",
"bam": "path/to/bam"
}

require("foo/bar") -> path/to/foo/bar.js
require("foo") -> path/to/foo.js
require("foo/bar/baz") -> different/path/to/baz.js
require("foo/bar/baz/far") -> different/path/to/baz/far.js
require("foo/bar/bam") -> path/to/foo/bar/bam.js
require("bam/blur") -> /path/to/bam/blur.js

From what I understand of the conversation in this thread, it sounds
like the desire to only map one segment, the first one, is being
advocated. That would work with the approach described above, just
specify the first segment only.

I find the above approach fairly flexible and allows mapping a single
module ID to a single file, which has been useful in practice -- it
allows an easy swap of a module without modifying code that depends on
it. That may be out of context for the package mapping discussion
here, so feel free to ignore that.

James

[Kris Zyp]

Assuming the first mapping was supposed to be "foo": ..., this makes
total sense, and I think we are generally in agreement that this is what
we want. I would propose a slight alteration. I think that if target
path ends with a slash, we should only match prefixes, and not exact
matches. Ending with a slash should be a way to provide explicit path
only conversion. For example:

mappings:{
"json": "http://example.com/json-package/lib/"
}
Then:
// this should not go to http://example.com/json-package/lib.js, it
should go directly to the default lookup path
require("json");
// this should go to http://example.com/json-package/lib/schema.js
require("json/schema");

But I think the primary question that is still in contention is what the
mapping delimiter character should be. In these examples we are using
slashes. Charles has suggested colons (for less confusion with paths),
so with the previous example:
require("json:schema");

Maybe we should just take this to a vote? What would you prefer for a
mapping delimiter character:
a. '/'
b. ':'
c. Something else, '!' or ';' anyone?

--
Thanks,
Kris

[Dean Landolt]

Dean Landolt: +1 (until someone can make a stronger case for confusion)

 

b. ':'
c. Something else, '!' or ';' anyone?

--
Thanks,
Kris

--
You received this message because you are subscribed to the Google Groups "CommonJS" group.
To post to this group, send email to comm...@googlegroups.com.
To unsubscribe from this group, send email to commonjs+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/commonjs?hl=en.

[Kris Kowal]

On Thu, May 6, 2010 at 11:59 AM, Kris Zyp <kri...@gmail.com> wrote:
> Maybe we should just take this to a vote? What would you prefer for a
> mapping delimiter character:
> a. '/'
> b. ':'
> c. Something else, '!' or ';' anyone?

a. "/"

Kris Kowal

[Irakli Gozalishvili]

I'm fine with both a) and b) even though I see advantages of having different delimiter. I can also suggest
@ delimiter

--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

[Irakli Gozalishvili]

KrisZ I believe seedjs has one more thing which I personally won't miss much, but some might. From your mapping example seedjs would:

require("json"); // gets http://example.com/json-package/lib/index.js

--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

[Kris Zyp]

On 5/10/2010 4:07 PM, Irakli Gozalishvili wrote:

KrisZ I believe seedjs has one more thing which I personally won't miss much, but some might. From your mapping example seedjs would:

require("json"); // gets http://example.com/json-package/lib/index.js

I don't have any objections to the use of index.js as the default module for a folder, but shouldn't that be handled by the web server, just like index.html? Your browser doesn't request http://somesite.com/index.html for you when you type in http://somesite.com/.

-- 
Thanks,
Kris

[Charles Jolley]

It is handled by the module loader.  So when you require('json') it finds json/index (and if not that, looks for json/json).

This fits with the current node.js behavior.

-C

[Irakli Gozalishvili]

I would love to eliminate fallback, since it will just complicate things on browser platform (which does not relies on build / backend).

--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

[Kris Zyp]

On 5/10/2010 4:21 PM, Charles Jolley wrote:
> It is handled by the module loader. So when you require('json') it
> finds json/index (and if not that, looks for json/json).
>
> This fits with the current node.js behavior.

I'm assuming by "module loader" in this context you are referring to the
lookup on the file system, which is fine. A URI on the otherhand (which
is what the package mapping deals with), shouldn't to know what you are
using for your default module name, and how that maps to the system. The
web server or file loader implementation can certainly choose to handle
folder URIs with index.js, but I think that should be beyond the scope
of the concerns of package mapping specification.

[Irakli Gozalishvili]

Maybe I miss something here, but can we:

merge dependencies and mappings ?
can we have relative mappings for nested packages ?

--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

[Kris Zyp]

On 5/10/2010 4:27 PM, Irakli Gozalishvili wrote:
> I would love to eliminate fallback, since it will just complicate
> things on browser platform (which does not relies on build / backend).

+1. Multiple lookups can easily move into exponential permutations and
terrible performance. The package mapping specification has been very
carefully crafted to unambiguously produce only a single URI for any
given module id.

[Kris Zyp]

On 5/10/2010 4:29 PM, Irakli Gozalishvili wrote:
> Maybe I miss something here, but can we:
>
> merge dependencies and mappings ?

Maybe, although I don't really know how. They are operated on completely
different concepts right now. "dependencies" works on named packages vs
and mappings works with URIs. Mappings don't really indicate any
package-wide dependency. If a mapping is not used (no require is made
that matches a particular mapping), there is no reason that the target
of the mapping needs to be available. Mappings explicitly specify how
the target can be located and how it can be required. Mappings don't
necessarily correspond to any package (could be to just a random URL on
the web, a nested folder in a package, etc). I'm open to ideas how they
could be merged, but I wouldn't want to complicate mappings too much.

> can we have relative mappings for nested packages ?

Following the standard rules for relative URIs (mapping targets can be
relative URIs and module ids follow the relative paths part), that
should be easy to do right now.

[Charles Jolley]

To me mappings is a fairly confusing concept that I think is less useful than simply allowing you to alias package names in the dependencies hash. i.e.

dependencies: {
"narwhal": {
"name": "narwhal-shim",
"version": "0.2.3"
}
}

Would allow you to require the package 'narwhal-shim' as 'narwhal' in your code. This is pretty straightforward.

I feel like the mappings hash is much more confusing. It is hard to predict just by looking at code what is going to happen thanks to the possibility of applying arbitrary remaps.

-C

[Kris Zyp]

On 5/10/2010 4:47 PM, Charles Jolley wrote:
> To me mappings is a fairly confusing concept that I think is less useful than simply allowing you to alias package names in the dependencies hash. i.e.
>
> dependencies: {
> "narwhal": {
> "name": "narwhal-shim",
> "version": "0.2.3"
> }
> }
>
> Would allow you to require the package 'narwhal-shim' as 'narwhal' in your code. This is pretty straightforward.
>

And where do you get narwhal-shim? This only works if your package is
lucky enough to be running on an environment that has the package
installed or has a package manager that is aware of that package. The
vast majority of modules are not registered in every package manager out
there (or even any), so this is a fail in the real world of
cross-platform applications with a diverse set of package managers (or
none at all on most systems). And this fails to address the need for
referencing folders in package (having to use names twice for packages
that are self-namespaced with nested folders is miserably error prone),
and single module referencing.

> I feel like the mappings hash is much more confusing. It is hard to predict just by looking at code what is going to happen thanks to the possibility of applying arbitrary remaps.
>

Mappings can be used as a single object hash. Everything I have seen
with more in depth dependencies involves complicated nested objects.
Mappings elegantly gives developers more power and portability with less
complexity.

[Kris Zyp]

I updated the package mapping proposal based on the discussions and suggestions for simplifying mappings. One does not need to suffix mappings with a slash for path-style mappings anymore. I defined the algorithm such that it is possible to explicitly choose path-based-only mappings and module-only mappings (or either). Also, there were two votes for using slashes as mapping delimiter character, and none otherwise, so I went with slashes. You can see the diff here:
http://wiki.commonjs.org/index.php?title=Packages/Mappings&diff=2768&oldid=2641

And the full proposal here:
http://wiki.commonjs.org/wiki/Packages/Mappings

Let me know if you think I got it wrong.

Thanks,
Kris

[Christoph Dorn]

Mappings are very different from dependencies and each solves a separate
concern as Kris alluded to below. I think we need both and they should
not be combined.

Christoph

Kris Zyp wrote:
>
> On 5/10/2010 4:29 PM, Irakli Gozalishvili wrote:
>> Maybe I miss something here, but can we:
>>
>> merge dependencies and mappings ?
>
> Maybe, although I don't really know how. They are operated on completely
> different concepts right now. "dependencies" works on named packages vs
> and mappings works with URIs. Mappings don't really indicate any
> package-wide dependency. If a mapping is not used (no require is made
> that matches a particular mapping), there is no reason that the target
> of the mapping needs to be available. Mappings explicitly specify how
> the target can be located and how it can be required. Mappings don't
> necessarily correspond to any package (could be to just a random URL on
> the web, a nested folder in a package, etc). I'm open to ideas how they
> could be merged, but I wouldn't want to complicate mappings too much.
>
>> can we have relative mappings for nested packages ?
>
> Following the standard rules for relative URIs (mapping targets can be
> relative URIs and module ids follow the relative paths part), that
> should be easy to do right now.
>

--

[Christoph Dorn]

I am pretty much happy with the proposal except for two items:

1) The "overlay" property should be in a separate proposal and the
mappings proposal should make reference to it. The overlay proposal
should begin to collect a list of "engines" that can be used as keys.
This is useful not only for the mappings proposal.

2) What happened to the mapping being restricted to one term only? IIRC
it was argued it would make it much more transparent as to what module
would be included. I believe this would address some of Charles'
transparency concerns.

What is the exact purpose of a multi-term mapping? Apart from providing
a nested namespace for mappings is the purpose to replace certain
modules far down the path tree? How does this work if the modules you
want to replace are relative-required from other modules which is likely
the typical case?

Christoph

Kris Zyp wrote:
> I updated the package mapping proposal based on the discussions and
> suggestions for simplifying mappings. One does not need to suffix
> mappings with a slash for path-style mappings anymore. I defined the
> algorithm such that it is possible to explicitly choose path-based-only
> mappings and module-only mappings (or either). Also, there were two
> votes for using slashes as mapping delimiter character, and none
> otherwise, so I went with slashes. You can see the diff here:
> http://wiki.commonjs.org/index.php?title=Packages/Mappings&diff=2768&oldid=2641
> <http://wiki.commonjs.org/index.php?title=Packages/Mappings&diff=2768&oldid=2641>
>
> And the full proposal here:
> http://wiki.commonjs.org/wiki/Packages/Mappings
>
> Let me know if you think I got it wrong.
>
> Thanks,
> Kris

> <http://wiki.commonjs.org/index.php?title=Packages/Mappings&diff=2768&oldid=2641>

[Kris Zyp]

On 5/11/2010 10:39 AM, Christoph Dorn wrote:
>
> I am pretty much happy with the proposal except for two items:
>
> 1) The "overlay" property should be in a separate proposal and the
> mappings proposal should make reference to it. The overlay proposal
> should begin to collect a list of "engines" that can be used as keys.
> This is useful not only for the mappings proposal.

I thought this was covered in Packages 1.1. Do we need a more extensive
overlay specification?

>
> 2) What happened to the mapping being restricted to one term only?
> IIRC it was argued it would make it much more transparent as to what
> module would be included. I believe this would address some of
> Charles' transparency concerns.

That's fine with me, I'm happy to add that restriction.

--
Thanks,
Kris

[Christoph Dorn]

Kris Zyp wrote:
> On 5/11/2010 10:39 AM, Christoph Dorn wrote:
>> I am pretty much happy with the proposal except for two items:
>>
>> 1) The "overlay" property should be in a separate proposal and the
>> mappings proposal should make reference to it. The overlay proposal
>> should begin to collect a list of "engines" that can be used as keys.
>> This is useful not only for the mappings proposal.
>
> I thought this was covered in Packages 1.1. Do we need a more extensive
> overlay specification?

It is indeed! I should have checked first. Sorry.

Christoph

[George Moschovitis]

> > require("json"); // getshttp://example.com/json-package/lib/index.js
> > <http://example.com/json-package/lib.js>

>
> I don't have any objections to the use of index.js as the default module

> for a folder.

-1

a default module is a bad idea, not really needed. and index.js is an
especially bad name.
a 'default' module is no index whatsoever.

-g.

[jhuni]

> all the existing suggestions are more or less compatible with mapping

The term "mappings" could mean any number of things...

Instead I advocate a system of "authorities" where as an authority
refers to a place in which a module can be obtained. (This is from the
Perl6 spec)

> why not put the mappings right in the dependency member

Indeed you should be able to do something like this:

{
"lang": "js",
"name": "foo",
"version": "0.0031",
"authority": "http://somewhere.com/somefile.js"
}

Or just:

"js-foo-0.0031-http://somewhere.com/somefile.js"

Also you should just be able to use "jsan" as an authority and it will
try to get the module from openjsan.org

> What would you prefer for a mapping delimiter character:

What's wrong with using dots? (Am I missing something here??)

Foo.Bar -> Foo/Bar.js
JooseX.Namespace.Dependended -> JooseX/Namespace/Dependended.js

[Charles Jolley]

Default modules are useful for packages. If just require('package-name') what module should I load?


Sent from my iPad

[Tim Schaub]

Charles Jolley wrote:
> Default modules are useful for packages. If just require('package-name') what module should I load?
>

package.json ~

{
"main": "__init__.js"
}

or "index.js" or whatever - left up to the package author.

[Kris Kowal]

On Wed, May 12, 2010 at 12:50 PM, Tim Schaub <tsc...@opengeo.org> wrote:
> Charles Jolley wrote:
>>
>> Default modules are useful for packages.  If just require('package-name')
>> what module should I load?
>
> package.json ~
>
> {
>    "main": "__init__.js"
> }
>
> or "index.js" or whatever - left up to the package author.

This is tentatively fine by me. Be clear about what path that's
resolved relative to. I favor resolving the path relative to
package.json.

Kris Kowal

[Charles Jolley]

Seems fine to me. If a developer does not specify a main property we should have a default.

The two most common right now are index.js and package-name.js Both seem about equal to me except that index.js does not require changing the filename if you alter the package name. I suppose you could also use main.js though that is a bit confusing if you are loading the package as a library and not an "executable".

Charles

Sent from my iPad

[Christoph Dorn]

Charles Jolley wrote:
> Seems fine to me. If a developer does not specify a main property we should have a default.
>
> The two most common right now are index.js and package-name.js Both seem about equal to me except that index.js does not require changing the filename if you alter the package name. I suppose you could also use main.js though that is a bit confusing if you are loading the package as a library and not an "executable".

I don't think there should be a default. It should always look for the
"main" property in package.json and if it does not exist throw an error.

>> On Wed, May 12, 2010 at 12:50 PM, Tim Schaub <tsc...@opengeo.org> wrote:
>>> Charles Jolley wrote:
>>>> Default modules are useful for packages. If just require('package-name')
>>>> what module should I load?
>>> package.json ~
>>>
>>> {
>>> "main": "__init__.js"
>>> }
>>>
>>> or "index.js" or whatever - left up to the package author.
>> This is tentatively fine by me. Be clear about what path that's
>> resolved relative to. I favor resolving the path relative to
>> package.json.

+1

Christoph

[Christoph Dorn]

jhuni wrote:
>> all the existing suggestions are more or less compatible with mapping
>
> The term "mappings" could mean any number of things...
>
> Instead I advocate a system of "authorities" where as an authority
> refers to a place in which a module can be obtained. (This is from the
> Perl6 spec)
>
>> why not put the mappings right in the dependency member

Mappings and dependencies are two different animals. This has been
explained further up in this thread.

Mappings map a one-term alias to a complete arbitrary URL that may
contain version segments as part of its path. It is not concerned with
packages but only linked modules. This can be used to quickly bootstrap
a bunch of components without a package manager (all you need is a
loader that supports dynamic loading of modules via mappings).

Dependencies are package declarations that need to be resolved/installed
by a package manager pulling the packages from catalogs for example. I
am advocating including URL locations of the catalog or package in the
descriptors while the current default is just a name.

Christoph

[Zachary Carter]

+1 for this algorithm:

1. check package.json for main property

2. check for index.js

3. check for package-name.js

4. throw error

--
Zach Carter

[Kris Zyp]

On 5/13/2010 10:03 AM, Zachary Carter wrote:

+1 for this algorithm:

1. check package.json for main property

2. check for index.js

3. check for package-name.js

4. throw error

The package mappings has been carefully designed to *never* introduce multiple lookups. This greatly simplifies implementation and improves performance. Introducing multiple looks can easily permutate to a numerous lookups with big performance penalties, and is particularly unacceptable in the browser. I would look to continue to avoid multiple lookups. Does the main module only apply to the package root, or does it apply to every folder in a package? With the latter, even with a known main module name, this seems to create situation where you have to go to the disk first to check to see if a directory exists, and then decide which module to look up from other (doubling every lookup).

Also, I would point out that the "main" module concept only really makes sense on file systems/directory structures. When a module id is mapped to a plain HTTP URL, there is no way to know if http://site.com/path/to/foo is directory and should be changed to http://site.com/path/to/foo/index.js instead of http://site.com/path/to/foo.js.

Personally this all seems more complicated than worth it for the value it adds (typing "package/index.js" isn't terribly hard), but I wouldn't stand in the way if everyone wants it added.

Kris

-- 
Thanks,
Kris

[Zachary Carter]

On Thu, May 13, 2010 at 12:37 PM, Kris Zyp <kri...@gmail.com> wrote:

On 5/13/2010 10:03 AM, Zachary Carter wrote:

+1 for this algorithm:

1. check package.json for main property

2. check for index.js

3. check for package-name.js

4. throw error

The package mappings has been carefully designed to *never* introduce multiple lookups. This greatly simplifies implementation and improves performance. Introducing multiple looks can easily permutate to a numerous lookups with big performance penalties, and is particularly unacceptable in the browser.

 

Great incentive for authors to always declare the default in package.json. The others checks are a compromise with other suggestions in this thread and interoperability with some existing idioms. I just wanted to throw it out there.

 

I would look to continue to avoid multiple lookups. Does the main module only apply to the package root, or does it apply to every folder in a package?

 

My comment was toward resolving the default package module.

 

With the latter, even with a known main module name, this seems to create situation where you have to go to the disk first to check to see if a directory exists, and then decide which module to look up from other (doubling every lookup).

Also, I would point out that the "main" module concept only really makes sense on file systems/directory structures. When a module id is mapped to a plain HTTP URL, there is no way to know if http://site.com/path/to/foo is directory and should be changed to http://site.com/path/to/foo/index.js instead of http://site.com/path/to/foo.js.

Personally this all seems more complicated than worth it for the value it adds (typing "package/index.js" isn't terribly hard), but I wouldn't stand in the way if everyone wants it added.

Agreed, I would happily abide as a package author. If others find value in a common default, however, I like the multistep approach.

[Kris Kowal]

I could buy "index.js" being the default value. If that's the case,
it might make more sense for the corresponding package.json property
to be "index". I could buy a proposal with an alternate name for
"index", but I also see how that would put some prototype packages out
of conformance out the starting gate with little reason to
substantiate the change. I could also buy a proposal that requires
explicit mention of an index if it exists; this has the advantage of
not mandating the existence of index.js. In fact, I prefer this
approach.

Having "index.js" fallback to (descriptor.name + ".js") does not jive
well with me, although my reasons are different than Kris Zyp's. It
seems to me that the purpose of a standard is to reduce the need for
complexity. The proposal of falling back is perhaps sound for
implementations attempting to assimilate non-standard packages, I
don't think it's sound for encouraging convergence on a single, simple
standard.

Kris Kowal

[Irakli Gozalishvili]

- I'm for having index property in package.json, which will default to index.js if not defined in package.json
- Since majority of votes are for the first option in "/" vs ":" battle, I'm buying it, but would've prefer 2nd option.

I don't think still don't agree that mappings and dependencies are different animals and if they are I'd like to have clear explanation (also in specs), since I think it can be a source for a lot of confusion.

--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

On Thu, May 13, 2010 at 21:43, Irakli Gozalishvili <rfo...@gmail.com> wrote:

Sorry for delay on this one:

On Tue, May 11, 2010 at 00:43, Kris Zyp <kri...@gmail.com> wrote:

On 5/10/2010 4:29 PM, Irakli Gozalishvili wrote:
> Maybe I miss something here, but can we:
>
> merge dependencies and mappings ?

Maybe, although I don't really know how. They are operated on completely
different concepts right now. "dependencies" works on named packages vs
and mappings works with URIs. Mappings don't really indicate any
package-wide dependency. If a mapping is not used (no require is made
that matches a particular mapping), there is no reason that the target
of the mapping needs to be available.

I don't think they are that different, since in a current spec "A CommonJS package is a cohesive wrapping of a collection of modules, code and other assets into a single form", in my understanding aliased URI is the same, even though collection might contain just a single asset.

I also don't see any point in declaring mapping if declaring package is not using it (there for depends on it). I also was assuming that package managers will cache mapped url's locally so that it can be aliased by any package to any name.
 

Mappings explicitly specify how
the target can be located and how it can be required. Mappings don't
necessarily correspond to any package (could be to just a random URL on
the web, a nested folder in a package, etc).

I would interpret it a single module dependency, and it doesn't matters where is it coming from
 

I'm open to ideas how they
could be merged, but I wouldn't want to complicate mappings too much.

I think I posted example before and it looked like this

"dependencies": {
    "narwhal-lib": { // alias following same naming convention as module id's except `/`

        "uri": "http://github.com/kriskowal/narwhal-lib/",  // explicit mapping

        "version": "0.2"                                    // does verision makes sence with expliciting mapping ?
    },

    "foo": {    // alias
        "id": "foo-bar-pack",   // package installed in platform with the matching name
        "version": ["1.0", "2.0"] //(optional ? platform preference) version
    }
}

Guess version property makes no sense in mapping which is first example. This also makes it easy to switch from mapped dependency used in dev process to a named package in production, even though I think that with a well designed package manager mapping can fully replace named packages, since name will can be url and it's guaranteed to be unique across all platforms, package managers, etc.
 

> can we have relative mappings for nested packages ?

Following the standard rules for relative URIs (mapping targets can be
relative URIs and module ids follow the relative paths part), that
should be easy to do right now.

--
Thanks,
Kris

[Irakli Gozalishvili]

Sorry for delay on this one:

On Tue, May 11, 2010 at 00:43, Kris Zyp <kri...@gmail.com> wrote:

On 5/10/2010 4:29 PM, Irakli Gozalishvili wrote:
> Maybe I miss something here, but can we:
>
> merge dependencies and mappings ?

Maybe, although I don't really know how. They are operated on completely
different concepts right now. "dependencies" works on named packages vs
and mappings works with URIs. Mappings don't really indicate any
package-wide dependency. If a mapping is not used (no require is made
that matches a particular mapping), there is no reason that the target
of the mapping needs to be available.

I don't think they are that different, since in a current spec "A CommonJS package is a cohesive wrapping of a collection of modules, code and other assets into a single form", in my understanding aliased URI is the same, even though collection might contain just a single asset.

I also don't see any point in declaring mapping if declaring package is not using it (there for depends on it). I also was assuming that package managers will cache mapped url's locally so that it can be aliased by any package to any name.
 

Mappings explicitly specify how

the target can be located and how it can be required. Mappings don't
necessarily correspond to any package (could be to just a random URL on
the web, a nested folder in a package, etc).

I would interpret it a single module dependency, and it doesn't matters where is it coming from
 

I'm open to ideas how they

could be merged, but I wouldn't want to complicate mappings too much.

I think I posted example before and it looked like this

"dependencies": {
    "narwhal-lib": { // alias following same naming convention as module id's except `/`
        "uri": "http://github.com/kriskowal/narwhal-lib/",  // explicit mapping
        "version": "0.2"                                    // does verision makes sence with expliciting mapping ?
    },

    "foo": {    // alias
        "id": "foo-bar-pack",   // package installed in platform with the matching name
        "version": ["1.0", "2.0"] //(optional ? platform preference) version
    }
}

Guess version property makes no sense in mapping which is first example. This also makes it easy to switch from mapped dependency used in dev process to a named package in production, even though I think that with a well designed package manager mapping can fully replace named packages, since name will can be url and it's guaranteed to be unique across all platforms, package managers, etc.
 

> can we have relative mappings for nested packages ?

Following the standard rules for relative URIs (mapping targets can be
relative URIs and module ids follow the relative paths part), that
should be easy to do right now.

--
Thanks,
Kris

[Irakli Gozalishvili]

Hi everyone,

Just realized that I have no idea what are the plans for dependencies of the mapped packages. Do we assume that if package maps http://myfoo.com/foo/ -> 'foo' then http://myfoo.com/foo/package.json should be read and it dependencies should me mapped as well??

--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

[Kris Zyp]

On 5/17/2010 5:35 PM, Irakli Gozalishvili wrote:
> Hi everyone,
>
> Just realized that I have no idea what are the plans for dependencies
> of the mapped packages. Do we assume that if package maps
> http://myfoo.com/foo/ -> 'foo' then http://myfoo.com/foo/package.json
> should be read and it dependencies should me mapped as well??

Any package mappings for http://myfoo.com/foo/ should be applied to the
modules in that package. The package.json file indicates the expected
mappings for the modules in that package, and don't need to be applied
outside that package. The package.json is expected to be in the root of
a package, but determining the root of the package from a URI can be a
bit more involved. The mapping specification provides guidelines for
determining the package root (and thus the location of the package.json).

[Irakli Gozalishvili]

@KrisZ can you also comment on my example of merging dependencies and mappings I'm not married to syntax but, I think idea of treating mappings as dependencies is good (in some cases mappings might be used for dev and replaced by dependencies in production)

--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

[Irakli Gozalishvili]

I think we are more or less aligned on this one now can we move to voting part??

If I got it correctly we have to show hands on:

1. mapping is restricted on one term only
2. package delimiter is:
   a) /
   b) :
   c) something else
3. index module name:
   a. index.js
   b. package-name.js
   c. specified as `index` property in package.json
   d. (c) and defaults (a)
   e. (c) and defaults (b)
   f. specified as `main` property in package.json
   g. (f) and defaults to (a)
   h. (f) and defaults to (b)
   
   

- Is there something I miss and we still need to agree on ?
- Will syntax of requiring dependencies be the same? Meaning that if my package depends on foo module bar from it can be loaded by require('foo/bar')

 
--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

[Kris Kowal]

On Tue, May 25, 2010 at 7:00 AM, Irakli Gozalishvili <rfo...@gmail.com> wrote:
> If I got it correctly we have to show hands on:
>
> mapping is restricted on one term only

+1

> package delimiter is:
> a) /
> b) :
> c) something else

I think we have a log of hand showing for this somewhere,
but I can't find it. In any case:

A +1

> index module name:
> a. index.js
> b. package-name.js
> c. specified as `index` property in package.json
> d. (c) and defaults (a)
> e. (c) and defaults (b)
> f. specified as `main` property in package.json
> g. (f) and defaults to (a)
> h. (f) and defaults to (b)

C. If there's no `index` property in `package.json`, there
is no `index.js`.

> - Is there something I miss and we still need to agree on
> ?

The "jar" URL's remain contentious. I favor plain URL's
to archives, presently in zip format, with the possibility
of supporting tar and tgz eventually.

If we need deep archive paths, I favor using the URL's
anchor/search/hash.

> - Will syntax of requiring dependencies be the same?
> Meaning that if my package depends on foo module bar from
> it can be loaded by require('foo/bar')

It's my understanding that, provided that there is a "foo"
mapping, "foo" would grab the "index.js" of the referenced
package, "foo/bar" would grab the "lib/bar.js" of the
package. In the absence of a "foo" mapping, "foo" would
reference "lib/foo.js" and "foo/bar" would grab
"lib/foo/bar.js". Anything else would be beyond
specification.

Obviously, some implementations, like Narwhal, will still
have a module search path, which will be useful for overlays
(particularly for engine-specific overlays) and the like,
but I don't think we need to agree on that. We might as
well remove and reserve the `dependencies` property of the
`package.json` since it is a vestige of the older,
overlay-based model of package management. I don't object
to repurposing it for `mappings` as you suggested, since it
is likely to be the dominant form of dependency management.

Tom and some Narwhal users might disagree with me about the
above paragraph.

Kris Kowal

[Kris Zyp]

On 5/25/2010 3:41 PM, Kris Kowal wrote:
> [snip]

>> - Is there something I miss and we still need to agree on
>> ?
>>
> The "jar" URL's remain contentious. I favor plain URL's
> to archives, presently in zip format, with the possibility
> of supporting tar and tgz eventually.
>
> If we need deep archive paths, I favor using the URL's
> anchor/search/hash.
>

There are some key advantages to using archives, obviously that you can
download packages much faster than individually downloading each module.
Archives are also a great indicator of the root of a package as well
(module loaders face more effort with mappings to plain URLs like
"foo":"http://site.com/some/where/in/here/is/the/root/of/my/package/and/the/rest/is/a/path/to/my/mapping/target").
We could force the use of "mirrors" for indicating the archive that
corresponds to "plain" URLs, but this adds complexity, and as I noted in
the spec, this can lead to some troubling security problems with
substituting alternate versions of code that later gets cached by the
URL rather than the archive source.

Of course I prefer tar/gzip to zip format as well, but is there any URI
scheme for tar/gzip? I didn't really want to get into the business of
inventing URI schemes as well. I am certainly open to other ideas, but
the vast majority use case of creating mapping that indicates a target
folder from an downloadable package archive is a drop dead simple one
line mapping right now, and I would hope that we don't make things more
complicated.

(cc'ed Isaac, since I think this is partly based off a conversation from
#node.js about npm)

--
Thanks,
Kris

[James Burke]

I care primarily about browser implementations, so weight the
following feedback accordingly:

On Tue, May 25, 2010 at 7:00 AM, Irakli Gozalishvili <rfo...@gmail.com> wrote:

> index module name:
> a. index.js
> b. package-name.js
> c. specified as `index` property in package.json
> d. (c) and defaults (a)
> e. (c) and defaults (b)
> f. specified as `main` property in package.json
> g. (f) and defaults to (a)
> h. (f) and defaults to (b)

I prefer a. index.js, only to make configuration in the browser
easier. It would be nice if the config in the browser for the packages
it used could be done via a simple collection of mappings (this is
just an example, not sure how it would really look):

{
"packages": {
"foo": "scripts/packages/foo/v1.2",
...
}
}

and the code could assume if there was a require('foo') that it
translated directly to 'scripts/packages/foo/v1.2/index.js',
require('foo/bar') to be 'scripts/packages/foo/v1.2/lib/bar.js'.
Otherwise, it seems like more copy/paste of code from each
package.json into the top level HTML page to get all the paths to work
out:

{
"packages": {
"foo": {
"mapping": "scripts/packages/foo/v1.2",
"index": "random.js"
},
...
}

In summary, I prefer stronger constraints in the package layout to
make it easier to configure a web page to use packages. I believe it
will also help make packages more uniform and therefore easier to
understand and visually inspect. Another example, I would prefer if
there was not "directories" option in the Packages spec either, but
just mandate the names of the directories. I suppose just fixing 'lib'
to hold other .js files is probably enough for avoiding the extra
config for the browser.

Feel free to disregard this feedback particularly if this has already
been beaten to death. I am only now starting to look at supporting
CommonJS-type of packages in a browser.

James

[Christoph Dorn]

I would not mind optimizing the defaults for package.json to be as
browser friendly as possible.

James Burke wrote:
> I care primarily about browser implementations, so weight the
> following feedback accordingly:
>
> On Tue, May 25, 2010 at 7:00 AM, Irakli Gozalishvili <rfo...@gmail.com> wrote:
>> index module name:
>> a. index.js
>> b. package-name.js
>> c. specified as `index` property in package.json
>> d. (c) and defaults (a)
>> e. (c) and defaults (b)
>> f. specified as `main` property in package.json
>> g. (f) and defaults to (a)
>> h. (f) and defaults to (b)
>
> I prefer a. index.js, only to make configuration in the browser
> easier. It would be nice if the config in the browser for the packages
> it used could be done via a simple collection of mappings (this is
> just an example, not sure how it would really look):
>
> {
> "packages": {
> "foo": "scripts/packages/foo/v1.2",
> ...
> }
> }

I still think mappings is the best name for the property.

> and the code could assume if there was a require('foo') that it
> translated directly to 'scripts/packages/foo/v1.2/index.js',
> require('foo/bar') to be 'scripts/packages/foo/v1.2/lib/bar.js'.
> Otherwise, it seems like more copy/paste of code from each
> package.json into the top level HTML page to get all the paths to work
> out:
>
> {
> "packages": {
> "foo": {
> "mapping": "scripts/packages/foo/v1.2",
> "index": "random.js"
> },
> ...
> }

I was under the impression the index.js file is a module in the "lib"
directory.

So given:

package.json ~ {
"mappings": {
"foo": "scripts/packages/foo/v1.2"
}
}

You could do:

require('foo') and get scripts/packages/foo/v1.2/lib/index.js
require('foo/bar') and get scripts/packages/foo/v1.2/lib/bar.js

> In summary, I prefer stronger constraints in the package layout to
> make it easier to configure a web page to use packages. I believe it
> will also help make packages more uniform and therefore easier to
> understand and visually inspect. Another example, I would prefer if
> there was not "directories" option in the Packages spec either, but
> just mandate the names of the directories. I suppose just fixing 'lib'
> to hold other .js files is probably enough for avoiding the extra
> config for the browser.

The "directories" property is optional and defaults to "lib" for modules.

Christoph

[Christoph Dorn]

Kris Kowal wrote:
> On Tue, May 25, 2010 at 7:00 AM, Irakli Gozalishvili <rfo...@gmail.com> wrote:
>> If I got it correctly we have to show hands on:
>>
>> mapping is restricted on one term only

+1

>> package delimiter is:
>> a) /
>> b) :
>> c) something else

a) +1

>> index module name:
>> a. index.js
>> b. package-name.js
>> c. specified as `index` property in package.json
>> d. (c) and defaults (a)
>> e. (c) and defaults (b)
>> f. specified as `main` property in package.json
>> g. (f) and defaults to (a)
>> h. (f) and defaults to (b)

c) +1
d) +1

Christoph

[Dean Landolt]

My votes are exactly the same as Christoph's.

[James Burke]

On Wed, May 26, 2010 at 10:15 AM, Christoph Dorn
<christ...@christophdorn.com> wrote:
>> {
>>  "packages": {
>>    "foo": "scripts/packages/foo/v1.2",
>>    ...
>>  }
>> }
>
> I still think mappings is the best name for the property.

"mappings" as the config property name works for me, I was just trying
to demonstrate what some imaginary config object in the HTML page
might look like.

> I was under the impression the index.js file is a module in the "lib"
> directory.
>
> So given:
>
> package.json ~ {
>  "mappings": {
>    "foo": "scripts/packages/foo/v1.2"
>  }
> }
>
> You could do:
>
> require('foo') and get scripts/packages/foo/v1.2/lib/index.js
> require('foo/bar') and get scripts/packages/foo/v1.2/lib/bar.js

Great, if they are all in lib, that makes things simpler.

> The "directories" property is optional and defaults to "lib" for modules.

Thanks for the clarification. I must have glossed over that part in the spec.

James

[Zachary Carter]

+1

--
Zach Carter

[Kris Zyp]

On 5/25/2010 8:00 AM, Irakli Gozalishvili wrote:

I think we are more or less aligned on this one now can we move to voting part??

If I got it correctly we have to show hands on:

1. mapping is restricted on one term only

-0.5

1. package delimiter is:
   a) /
   b) :
   c) something else

0/neutral

1. index module name:
   a. index.js
   b. package-name.js
   c. specified as `index` property in package.json
   d. (c) and defaults (a)
   e. (c) and defaults (b)
   f. specified as `main` property in package.json
   g. (f) and defaults to (a)
   h. (f) and defaults to (b)
   
   

I am still not sure I understand how and where this applies. I am -1 on special index modules applying to mappings. I am +1 to an index module in situations where a module loader provides a way of "executing" a package in some sense. However, I wouldn't choose any of the options above, I would vote for lib/index.js (I'd prefer to keep my modules in the lib directory).

If we do have the index module applied to mappings, I think I would need a better explanation of the algorithm for how we determine if/when to add the index module name to arbitrary URL in order to add it to the spec (or someone else would need to add it). It is not clear to me at all how this would work.

-- 
Thanks,
Kris

[Irakli Gozalishvili]

I think lib/index.js is exactly what (a) stands for. My interpretation is that in case require was called with an argument which is a mapping term, require should return exports from the lib/index.js of that mapping. I do think it's good idea if someone will update specs indeed. English is not native to me and I unnecessarily complicate things sometimes, so I would be happy if someone else will do that, in worst case I can pick that up.
 

-- 
Thanks,
Kris

[Irakli Gozalishvili]

--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

On Tue, May 25, 2010 at 16:00, Irakli Gozalishvili <rfo...@gmail.com> wrote:

I think we are more or less aligned on this one now can we move to voting part??

If I got it correctly we have to show hands on:

1. mapping is restricted on one term only

+1

1. package delimiter is:
   a) /
   b) :
   c) something else

prefer b) or c) but I don't mind a) as well and it looks it's preferred by most of a people here so...

± 0

1. index module name:
   a. index.js
   b. package-name.js
   c. specified as `index` property in package.json
   d. (c) and defaults (a)
   e. (c) and defaults (b)
   f. specified as `main` property in package.json
   g. (f) and defaults to (a)
   h. (f) and defaults to (b)
   
   

I also do think that commonjs should be first class citizen on Browser and simplicity is beneficial so I'm for

+1 a) 
-1 with all the combinations of b) since it will only complicate nested packages. 

[Irakli Gozalishvili]

I think there was two important things mentioned that should be decided

1. Should dependencies and mappings get merged ?

2. What should be the name of mappings / dependencies

a) mappings 

b) dependencies

c) packages

 
--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

[Charles Jolley]

So do I understand correctly that we are now considering dropping the dependencies property in favor of mappings? If not, what is the purpose of having both? It seems that if you name packages mapped to a URL explicitly you just don't even need package versions anymore.

Also if you map a package to a URL how would you handle cases where I want to keep a copy of all my packages in an alternate location - say on a CDN or using an internal server. It would be a maintenance nightmare if I had to go in and modify every package.json that I depend upon to do so.

C

[Kris Zyp]

So if I do
mappings:{
  foo:"http://site.com/might/be/a/package/root/and/module/path/somewhere/in/here"
}

and

require("foo");

How do I determine if I should append "/index.js" to the URL? The URL may not be the root (or lib directory in the root) of the package. My best guess is that it would involve using the algorithm from the spec for finding the package root, and then doing some string matching to see if we are pointing to the "lib" directory inside the package root, or whatever path is indicated for the index module.

-- 
Thanks,
Kris

[Irakli Gozalishvili]

On Thu, May 27, 2010 at 16:59, Irakli Gozalishvili <rfo...@gmail.com> wrote:

I think there was two important things mentioned that should be decided

1. Should dependencies and mappings get merged ?

+1 merged

 

2. What should be the name of mappings / dependencies

a) mappings 

b) dependencies

c) packages

0/neutral

[Kris Zyp]

On 5/27/2010 9:02 AM, Charles Jolley wrote:
> So do I understand correctly that we are now considering dropping the dependencies property in favor of mappings? If not, what is the purpose of having both? It seems that if you name packages mapped to a URL explicitly you just don't even need package versions anymore.
>
> Also if you map a package to a URL how would you handle cases where I want to keep a copy of all my packages in an alternate location - say on a CDN or using an internal server. It would be a maintenance nightmare if I had to go in and modify every package.json that I depend upon to do so.
>

I think the packages specification includes a "repositories" property
for situations where the target package wants to specify the CDN, and
the mappings doesn't include a "mirrors" property, but I assume that is
what you are trying to avoid using. I don't think there is anything
wrong with other configuration techniques to remap URLs to CDN URLs. But
that would be beyond the scope of the package mappings mechanisms, which
primarily is intended to indicate the canonical URL.

--
Thanks,
Kris

[Irakli Gozalishvili]

I thought we accept mappings with url's which either ends by / or file extension. In first case it's package / sub-package mapping in second it's a module. if it's a sub-package I believe it should also contain lib and index.js but endeed it's getting messy. On the other hand if you make nested mapping most likely you dont want to require modules by this mapping so extra lib folder is not so bad.

I think we should choose from:

1) removing directories as was proposed and always default to `lib` and have index.js

2) keep directories and add index property to package json

In any case having same index for package and nested package doesn't makes much sense in my opinion 

 
--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

[Kris Zyp]

On 5/27/2010 8:59 AM, Irakli Gozalishvili wrote:
> I think there was two important things mentioned that should be decided
>
> 1. Should dependencies and mappings get merged ?

What specifically would this mean for mappings? Hopefully it wouldn't
make mappings more complex.

> 2. What should be the name of mappings / dependencies
>
> a) mappings
> b) dependencies
> c) packages
>

Obviously I prefer "mappings". "packages" doesn't make sense because
targets don't have to be packages. "dependencies" is closer, but we are
indicating more than a dependency, we are also indicating how module ids
are mapped to the dependencies.

--
Thanks,
Kris

[Kris Zyp]

On 5/27/2010 9:15 AM, Irakli Gozalishvili wrote:
> I thought we accept mappings with url's which either ends by / or file
> extension. In first case it's package / sub-package mapping in second
> it's a module. if it's a sub-package I believe it should also contain
> lib and index.js but endeed it's getting messy. On the other hand if
> you make nested mapping most likely you dont want to require modules
> by this mapping so extra lib folder is not so bad.

Currently the spec says that if the mapping target ends with a slash,
require'd module ids directly match the mapping source, they can only
match when used in a path. If we did change to matching, what if the
path ends with a slash and is not package?

>
> I think we should choose from:
>
> 1) removing directories as was proposed and always default to `lib`
> and have index.js
> 2) keep directories and add index property to package json
>
> In any case having same index for package and nested package doesn't
> makes much sense in my opinion

I am not sure I understand these options.

This seems like it is getting very complicated. The original spec was
simple string substitution. I like simple.

--
Thanks,
Kris

[Irakli Gozalishvili]

On Thu, May 27, 2010 at 17:16, Kris Zyp <kri...@gmail.com> wrote:

On 5/27/2010 8:59 AM, Irakli Gozalishvili wrote:
> I think there was two important things mentioned that should be decided
>
> 1. Should dependencies and mappings get merged ?

What specifically would this mean for mappings? Hopefully it wouldn't
make mappings more complex.

I asked you to look at the example I posted couple of times already, you can find it in one of my previous messages I believe.

By merging I mean that dependencies will be mappings either to uri's or to system wide packages, wehere probably versions are also necessary.  

 

> 2. What should be the name of mappings / dependencies
>
> a) mappings
> b) dependencies
> c) packages
>

Obviously I prefer "mappings". "packages" doesn't make sense because
targets don't have to be packages. "dependencies" is closer, but we are
indicating more than a dependency, we are also indicating how module ids
are mapped to the dependencies.

I do agree but prefer dependencies, from my point of you it's just a symlink to your dependency

[Irakli Gozalishvili]

I think what I'm describing is simpler and I believe it was preference of others here, but I might be wrong. I'll try to illustrate it in example:

mappings:{

'foo': 'http://foo.com/path/foo.js'

'bar': 'http://foo.com/path/bar/'

'baz': 'http://foo.com/path/bar/subpath/baz/'

}

require('foo')              // -> http://foo.com/path/foo.js

require('foo/whatever')     // -> will throw

require('bar')              // -> http://foo.com/path/bar/lib/index.js

require('bar/foo')          // -> http://foo.com/path/bar/lib/foo.js

require('baz')              // -> http://foo.com/path/bar/subpath/baz/lib/index.js

require('baz/foo')          // -> http://foo.com/path/bar/subpath/baz/lib/index.js

require('baz/foo')          // -> http://foo.com/path/bar/subpath/baz/lib/foo.js

--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

[Kris Zyp]

On 5/27/2010 9:36 AM, Irakli Gozalishvili wrote:

I think what I'm describing is simpler and I believe it was preference of others here, but I might be wrong. I'll try to illustrate it in example:

mappings:{

'foo': 'http://foo.com/path/foo.js'

'bar': 'http://foo.com/path/bar/'

'baz': 'http://foo.com/path/bar/subpath/baz/'

}

require('foo')              // -> http://foo.com/path/foo.js

require('foo/whatever')     // -> will throw

require('bar')              // -> http://foo.com/path/bar/lib/index.js

require('bar/foo')          // -> http://foo.com/path/bar/lib/foo.js

require('baz')              // -> http://foo.com/path/bar/subpath/baz/lib/index.js

require('baz/foo')          // -> http://foo.com/path/bar/subpath/baz/lib/index.js

require('baz/foo')          // -> http://foo.com/path/bar/subpath/baz/lib/foo.js

How do I make a mapping for "bar" such that:

require("bar/foo") -> http://foo.com/path/bar/foo.js

And how do I make a mapping for "bar" such it only matches "bar/" used in a path, and does not match require("bar")?

It seems like the more magic we add, the less flexible this gets.

-- 
Thanks,
Kris

[Charles Jolley]

Hi I don't think this will work in practice for two reasons:

1. Many times packages will be hosted as a single file.  It may be a zipped archive, or in the browser case, you may have a single js file which actually contains one or more packages and all of their modules to minimize round trips to the server.  Simple string substitution won't work in either of these cases.

2. In general you can't download just one module if it belongs to another package because you need the package.json for the module's package to know how to interpret requires inside the other module.

 I think the simplest implementation would be to state that the first term maps to a URL which represents the package of the other module.  The CommonJS runtime would then look up the module within that package.

In other words, you might map foo to /pkgs/foo/1.2.3. In this case for require('foo/bar') I would expect to find a package.json at /pkgs/foo/1.2.3/package.json which will in turn tell me how to get to the bar module within the package.

-Charles 

Sent from my iPad

[ihab...@gmail.com]

On Thu, May 27, 2010 at 8:54 AM, Charles Jolley <cha...@sproutit.com> wrote:
>  I think the simplest implementation would be to state that the first term
> maps to a URL which represents the package of the other module.  The
> CommonJS runtime would then look up the module within that package.

+1. This is indeed a far simpler and more predictable state of
affairs. Simple + predictable = less likely to trip up developers. The
extra flexibility of alternate approaches may save a few characters of
typing here or there, but in the end, for programming in the large
(which is what packages attempt to do, after all), clarity and
predictability beat all, I think.

Ihab

--
Ihab A.B. Awad, Palo Alto, CA

[Kris Zyp]

I am not sure what this objection is in response to, but the mappings
specification does indicate how to find the package.json file and
supports archives.

On 5/27/2010 9:54 AM, Charles Jolley wrote:
> Hi I don't think this will work in practice for two reasons:
>
> 1. Many times packages will be hosted as a single file. It may be a
> zipped archive, or in the browser case, you may have a single js file
> which actually contains one or more packages and all of their modules
> to minimize round trips to the server. Simple string substitution
> won't work in either of these cases.
>
> 2. In general you can't download just one module if it belongs to
> another package because you need the package.json for the module's
> package to know how to interpret requires inside the other module.
>
> I think the simplest implementation would be to state that the first
> term maps to a URL which represents the package of the other module.
> The CommonJS runtime would then look up the module within that package.

This doesn't work because not every target is a package.

--
Thanks,
Kris

[Irakli Gozalishvili]

What I was trying to suggest is exactly what @charles and @ihab just mentioned.  

--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

Thanks,
Kris

[Irakli Gozalishvili]

Forwarding message which by mistace I've send to @KrisZ only

--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

---------- Forwarded message ----------
From: Irakli Gozalishvili <rfo...@gmail.com>
Date: Thu, May 27, 2010 at 18:01
Subject: Re: [CommonJS] packages
To: Kris Zyp <kri...@gmail.com>

--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

No it's simple you just don't. Either you map package or module as simple as it sounds. If you will tell me the case wehere mapping directory is actually required / is very beneficial you might convince me that it worse it.

Actually I don't mind throwing index all together, but it looks like a lot of people use it, on the other hand we can omit directory mappings and everyone is happy. In any case if we'll have directory mappings it's not clear where to look for package.json either   

-- 
Thanks,
Kris

[Kris Zyp]

On 5/27/2010 10:01 AM, Irakli Gozalishvili wrote:

--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

On Thu, May 27, 2010 at 17:42, Kris Zyp <kri...@gmail.com> wrote:

No it's simple you just don't. Either you map package or module as simple as it sounds. If you will tell me the case wehere mapping directory is actually required / is very beneficial you might convince me that it worse it.

Actually I don't mind throwing index all together, but it looks like a lot of people use it, on the other hand we can omit directory mappings and everyone is happy. In any case if we'll have directory mappings it's not clear where to look for package.json either  

One place I have been using it is for Jack:
"mappings":{
  "jack": "http://github.com/kriszyp/jack/raw/master/lib/jack/"
}

If I can only map to the package root, than I have to do
require("jack/jack/querystring");

Which is obviously pretty redundant/error prone.

I suppose mapping to package roots would be more acceptable if we could also optionally specify a sub-path in the package for the mapping:
"mappings":{
  "jack": {
    "to": "http://github.com/kriszyp/jack/raw/master/",
    "path": "lib/jack"
  }
}

(where the path obviously would default to "lib")

However, I still feel that moving to a mapping system that requires a more complicated machinery to achieve the same level of functionality doesn't seem very compelling to me.

-- 
Thanks,
Kris

[Irakli Gozalishvili]

Me and I do believe @charles as well meant that you map package root and instead. So it will be


"mappings":{
  "jack": "http://github.com/kriszyp/jack/raw/master/"
}

require("jack/querystring");

we can even say that you directly specify module or package descriptor in mapping, it can help us avoiding confusions.
 

[Kris Zyp]

That doesn't work, there is no querystring.js module in the lib directory (see http://github.com/kriszyp/jack/tree/master/lib), it is in the jack subfolder.

-- 
Thanks,
Kris

[Christoph Dorn]

On 10-05-27 9:16 AM, Kris Zyp wrote:
> One place I have been using it is for Jack:
> "mappings":{
> "jack": "http://github.com/kriszyp/jack/raw/master/lib/jack/"
> }
>
> If I can only map to the package root, than I have to do
> require("jack/jack/querystring");
>
> Which is obviously pretty redundant/error prone.
>
> I suppose mapping to package roots would be more acceptable if we could
> also optionally specify a sub-path in the package for the mapping:
> "mappings":{
> "jack": {
> "to": "http://github.com/kriszyp/jack/raw/master/",
> "path": "lib/jack"
> }
> }
>
> (where the path obviously would default to "lib")

This is getting close to the using packages implementation I am working on:

"packages": { // I think "packages" is better than "using"
"jack": {
"location": "http://github.com/kriszyp/jack/zipball/master/",
"descriptor": {
"directories": {
"lib": "lib/jack"
}
}
}
}

You can override the package.json of the referenced package for the
mapping. IMO this is by far the simplest and cleanest solution.

> However, I still feel that moving to a mapping system that requires a
> more complicated machinery to achieve the same level of functionality
> doesn't seem very compelling to me.

I think we are trying to combine mappings to *modules* with mappings to
*packages* again.

In my view "mappings" deals with mapping a term to a URL where suffixes
are simply appended. I made a mistake in my other email stating that
package.json would be consulted and the directories.lib property spliced
into the URL.

I think "mappings" should strictly deal with referencing modules
*without* consulting package.json. i.e. via a simple path concatenation
In this case you are mapping all modules onto one system namespace. This
is what KrisZ is advocating I believe.

The "packages" property on the other hand can deal with mapping to
packages with the ability to override package.json, more dynamic
versioning etc...

I think this distinction needs to be taken to a vote as it keeps
tripping us up and complicating things. If we can agree on the split
then we can ratify "mappings" and start work on "packages".

Christoph

[Kris Zyp]

Its not really my primary intent to avoid consulting package.json, and
while you can load modules without package.json, to execute a module,
you certainly need to consult package.json (to know how the target
modules require'd module ids are mapped). While I am not sure what the
merging of "dependencies" and "mappings" would mean, I do agree with
Irakli that we don't want two mechanisms of specifying how module ids
are mapped to dependencies (we wouldn't to have the "package" way and
the URL way). I actually like your format for packages if we are going
to get into dependent packages specifying information about the
dependency packages.

IMO, the real key premises of mappings are:
* A URL-based location of dependents is the included such that modules
can be downloaded from anywhere and we are not dependent on Isaac's,
Charles', Tom's, TJ's or anybody else's catalog or repository.
* Target modules can be specified even if they do not conform to a
CommonJS package structure (with package.json and a module lib directory).
* Keep it simple to use in the common cases
* Packages own their own namespace of module ids, and are allowed to
dictate how they expect their require statements to be mapped to
dependencies.

If we want to move to a more package-oriented approach, I wouldn't mind
definitions like your example above (and maybe this is kind of Irakli
meant by merging dependencies with mappings) in mappings, but it should
be something that can be applied locally (different packages could
potentially map "jack" to different targets). Version ranges might also
be a good addition with this approach.

--
Thanks,
Kris

[Kris Kowal]

On Thu, May 27, 2010 at 7:26 AM, Kris Zyp <kri...@gmail.com> wrote:
>
> index module name:
> a. index.js
> b. package-name.js
> c. specified as `index` property in package.json
> d. (c) and defaults (a)
> e. (c) and defaults (b)
> f. specified as `main` property in package.json
> g. (f) and defaults to (a)
> h. (f) and defaults to (b)
>
> I am still not sure I understand how and where this applies. I am -1 on
> special index modules applying to mappings.

It is my understanding that this election is about having a
mapping for "foo" to some package and what module
require("foo") returns.

> I am +1 to an index module in situations where a module
> loader provides a way of "executing" a package in some
> sense.

I'm also +1, although I do not think that is the intent of
this election. However, I do think an executable package
feature could piggy-back on the index module.

> However, I wouldn't choose any of the options above, I
> would vote for lib/index.js (I'd prefer to keep my modules
> in the lib directory).

I've convinced myself to be indifferent about this. It is
my personal preference to not introduce a situation where
"foo" and "foo/index" refer to the same module.

Kris Kowal

[Kris Kowal]

On Thu, May 27, 2010 at 7:48 AM, Irakli Gozalishvili <rfo...@gmail.com> wrote:
>
>> index module name:
>> a. index.js
>

> I think lib/index.js is exactly what (a) stands for. My interpretation is
> that in case require was called with an argument which is a mapping term,
> require should return exports from the lib/index.js of that mapping. I do
> think it's good idea if someone will update specs indeed. English is not
> native to me and I unnecessarily complicate things sometimes, so I would be
> happy if someone else will do that, in worst case I can pick that up.

My interpretation was that, if package A has a mapping from "foo" to B,
require("foo") in package A would:

I.) map to a file in package B, relative to the package
root, in which case the index property would be a file
name.
II.) map to a module in package B, in which case the index
property would be a module top-level identifier.

Which is orthogonal to the concerns above of how to balance
convention and configuration:

A.) convention (always foo/index.js)
B.) configuration (always refers to foo/package.json's
"index" property, in which case it is possible to *not*
map require("foo") to any module.)
C.) configuration before convention (if a property exists in
package.json, use it, otherwise default to the
conventional module name or package path location.

Which is orthogonal to the concern of what the name the
property and the default file name or module name. It seems
clear that "index" is the general preference, either way.
The choices were:

X.) index
Y.) main
Z.) same name as package's self-described name in its
package.json (although there may have been some
confusion that it would be the name of the mapping)

Kris Kowal

[Kris Kowal]

On Thu, May 27, 2010 at 7:59 AM, Irakli Gozalishvili <rfo...@gmail.com> wrote:
> I think there was two important things mentioned that should be decided
> 1. Should dependencies and mappings get merged ?
> 2. What should be the name of mappings / dependencies
> a) mappings
> b) dependencies
> c) packages

Let's table this. There are proposals that use these names
for clearly different purposes, so let's just reserve them
for the purposes defined in their original proposals and
vote on the proposals rather than the names. That will at
least keep us less confused in discussing the proposals.

* "mappings" is used in Kris Zyp's proposal for mapping
short names to packages
* "dependencies" is used by precedent in Narwhal and NPM for
different purposes.
* In Narhwal/Tusk, it is used to
determine what packages are installed globally, and in
what order their lib directories will appear.
* In NPM it is used to determine top-level short names for
each package in a single lib directory, into which the
packages' lib directories will be linked.
* "packages" is used differently by Christoph Dorn and
Charles Jolley:
* Charles uses a packages mapping to map a short name in
the current package's module name space to a location
and additional metadata about what path to traverse into
for its library path.
* Christoph Dorn uses a packages mapping to map a short
name in the current package's module name space to a
location and to an object that overrides the contents of
that package's package.json "descriptor" object, such as
the library path.

Kris Kowal

[Kris Kowal]

On Thu, May 27, 2010 at 8:02 AM, Charles Jolley <cha...@sproutit.com> wrote:
> So do I understand correctly that we are now considering dropping
> the dependencies property in favor of mappings?  If not, what is the
> purpose of having both?  It seems that if you name packages mapped
> to a URL explicitly you just don't even need package versions
> anymore.

Agreed, for the purposes of the mappings or packages proposal, the
dependencies property is not useful and should be removed from the
specification. I do not think we should rename "mappings" or
"packages" to make use of the "dependencies" property name.

Narwhal will continue to support "dependencies" as it has before,
since it is so useful for having engine packages mix modules into the
top level name space and other uses, but I plan to migrate it into a
narwhal.dependencies property in future versions. I am in favor of
dropping the "dependencies" property from the specification and
reserving it for a future specification if it proves possible to mix
the strategies in a way that is beneficial outside the scope of
Narwhal.

> Also if you map a package to a URL how would you handle cases where
> I want to keep a copy of all my packages in an alternate location -
> say on a CDN or using an internal server.  It would be a maintenance
> nightmare if I had to go in and modify every package.json that I
> depend upon to do so.

I actually think that it's reasonable to solve the web depolyment
problem with tooling; it's necessary to build "lean" versions of
packages for web deployment anyway. I think that using packages on
the web is relevant in so far as that it should be possible create a
loader that can operate on source packages if they're unarchived at a
particular URL. For production deployment, it is still necessary to
compile packages into bundles or transports, at the very least.

Kris Kowal

[Kris Kowal]

On Thu, May 27, 2010 at 8:04 AM, Kris Zyp <kri...@gmail.com> wrote:
> So if I do
> mappings:{
> foo:"http://site.com/might/be/a/package/root/and/module/path/somewhere/in/here"
> }
>
> and
>
> require("foo");
>
> How do I determine if I should append "/index.js" to the URL? The
> URL may not be the root (or lib directory in the root) of the
> package. My best guess is that it would involve using the algorithm
> from the spec for finding the package root, and then doing some
> string matching to see if we are pointing to the "lib" directory
> inside the package root, or whatever path is indicated for the index
> module.

It's my opinion that a mapping should always be from a short name to
the root of a package, albeit its archive. That seems simplest to me.
If you need to delve into a package, I think it should be through some
other mechanism as Charles and Christoph have proposed.

I think some of your concerns are in addressing existing Narwhal
packages, which are designed for a different packaging scheme, where
their lib directories are simply overlayed in a global, system
installation of packages. You cite the example of Jack. I think the
right way to grandfather these packages in is as Christoph proposes,
to override the "index" and "directories.lib" properties of the
package.

{
"packages": {
"jack": {
"location": "...zip",
"descriptor": {
"index": "lib/jack.js",
"directories": {
"lib": "lib/jack"
}
}
}
}
}

I think it would be lovely if we could set up a migration path so
packages can advertise package.json metadata for both the old and
busted Narwhal approach and the "mappings" new hawtness, but it might
be better to just use the above technique to assist new packages in
using old packages and just hope that the old style disappears in
time.

Regardless, it will always be useful for the consumer of a package to
be able to override its package.json descriptor in cases where a
package was not designed for use in the JavaScript package ecosystem,
like the Google App Engine standard library archive. In Narwhal, we
provide a descriptor for that package in the catalog and write it out
to gae/package.json after installing it.

Kris Kowal

[Christoph Dorn]

It looks like there is a lot of overlap between the "mappings", "system"
and "using packages" approaches now.

Maybe we can combine them as follows:

Drop "mappings", "using" and "packages" in favor of "dependencies":

package.json ~ {
"dependencies": {
"foo": { // mapping alias
"as": "mapping", // map onto system namespace
"location": "http://domain.com/packages/package1/"
}
}
}
require("foo/<module>")
-> http://domain.com/packages/package1/[lib]/<module>

package.json ~ {
"dependencies": {
"foo": { // informational only in this case
"as": "system", // dump onto system namespace
"location": "http://domain.com/packages/package1/"
}
}
}
require("<module>")
-> http://domain.com/packages/package1/[lib]/<module>

package.json ~ {
"dependencies": {
"foo": { // package alias
"as": "local", // local to package
"location": "http://domain.com/packages/package1/"
}
}
}
require("<module>", "foo")
-> http://domain.com/packages/package1/[lib]/<module>

The package locator can support the following optional properties among
others:

"foo": {
"as": "mapping",
"location": "http://domain.com/packages/package1/",
"path": "<pathToSubPackage>"
"module": "<module>"
}

require("foo")
-> http://domain.com/packages/package1/<pathToSubPackage>/[lib]/<module>

Possible target specifiers:

// zip archive or base URL
"foo": {
"location": "http://domain.com/packages/package1/"
}
// explicit archive types
"foo": {
"location": "http://domain.com/packages/package1.<ext>"
}
"foo": {
"location": "<type>:http://domain.com/packages/package1/"
}
// catalog based
"foo": {
"catalog": "http://domain.com/packages/catalog.json",
"name": "package1",
"version": <version selector>
}

Overwriting package.json for target package local to the dependency:

package.json ~ {
"dependencies": {
"foo": {
"descriptor": {
"directories": {
"lib": "lib/bar"
}
}
}
}
}

Thoughts?

Christoph

[Kris Kowal]

On Thu, May 27, 2010 at 3:04 PM, Christoph Dorn
<christ...@christophdorn.com> wrote:
> It looks like there is a lot of overlap between the
> "mappings", "system" and "using packages" approaches now.
>
> Maybe we can combine them as follows:
>
> Drop "mappings", "using" and "packages" in favor of "dependencies":
>

> [snip]
>
> Thoughts?

No, let's just focus on getting mappings right, drop
dependencies, and drop require2. Keep it simple.

Kris Kowal

[George Moschovitis]

> drop require2.  Keep it simple.

+1

-g.

[Christoph Dorn]

It is not so much about the naming of the property than the approaches
to requiring dependencies. Originally I thought mappings would only deal
with modules directly. Now they deal with packages as well.

Narwhal's "dependencies" property can be nicely grandfathered:

"dependencies": [
"jack"
]

can be equivalent to

"dependencies": {
"jack": ["1.0", "2.0"]
}

and

"dependencies": {
"group": {
"jack": ["1.0", "2.0"]
}
}

and

"dependencies": {
"jack": {
"as": "system",
"location": "http://..."
}
}

and

"dependencies": {
"jack": {
"as": "system",
"catalog": "http://...",
"name": "jack"
}
}

Now instead of dumping the dependency onto the system namespace you use
"as" to map it:

"dependencies": {
"jack": {
"as": "mapping",
"location": "http://...",
"descriptor": {
"directories": {
"lib": "lib/jack"
}
}
}
}

Having two properties, one for mappings and one for proper packages,
will be more confusing I think than having one property with the "as"
feature.

Christoph

[Christoph Dorn]

On 10-05-27 3:14 PM, George Moschovitis wrote:
>> drop require2. Keep it simple.
>
> +1

The official "as" values can be limited to:

- mapping
- system

I may be able to drop require2 for my purposes completely if "mappings"
is going to deal with packages and is limited to one term.

Christoph