Packages 0.1

19 views
Skip to first unread message

mob

unread,
Dec 16, 2009, 4:22:04 PM12/16/09
to CommonJS
I've mined the various information and pulled together a proposal for
Packages 0.1. This proposal defines the package.json file, the package
directory structure and package file format. It does not specify the
catalog file (yet).

I'd like to get feedback on this so we can iterate and then nail down
this part of the packaging system. Then we can all create packages
with confidence they will be usable in the various package managers,
tools and loaders.

I've tried to collect the various input from all, but I've also added
a couple of new fields. Please point out any fields that are missing
from the descriptor file.

I've kept the package.json file fairly flat to make it easier for
tools to inspect and modify fields.

Enjoy

--mob

mob

unread,
Dec 16, 2009, 4:22:46 PM12/16/09
to CommonJS
Ah -- you'll need the link: http://wiki.commonjs.org/wiki/Packages/0.1

--mob

Ash Berlin

unread,
Dec 16, 2009, 4:27:29 PM12/16/09
to comm...@googlegroups.com

Ash Berlin

unread,
Dec 16, 2009, 4:38:25 PM12/16/09
to comm...@googlegroups.com

On 16 Dec 2009, at 21:22, mob wrote:

What are the differences between categories and keywords? And do we really need either?

Author: being a single field - split it into three (or an array of 3 elements) rather than requiring a fixed format that people will inevitably get wrong (and then only the name of the 3 is required).

Dependencies: not sure the version handling is right. if we are using semantic versioning then isn't just [ "ejs", "1.0.0"] enough and implies the >= part and "< 2.0" is implicit from SemVer. (I also don't like '>=' as it implies '=' should work and I really dont want to get into the culture of letting people do that)

stability: again, doesn't SemVer cover this and make it redundant?

checksum: why [ "md5", "valuehere" ] and not just { "md5": "valuehere" } - its not like you can have more than one checksum of a given type, and this would make access easier.

builtin: is this needed?

directories: purpose? And if its a hash what are the keys?

scripts: Hmmm I thought this was going to be list of scripts to install or some such. Perhaps "commands" or "build_commands" is a better name?

Haven't yet had time to think though and see if there is anything missing.

-ash

Kris Kowal

unread,
Dec 16, 2009, 4:44:46 PM12/16/09
to comm...@googlegroups.com
I agree that semantic versioning should alleviate the need for complex
version predicates, even in the case where someone breaks their API
accidentally; they can ramp their patch version and revert to fix the
problem.

Last Friday I did some work on Tusk's "tusk update" which adds support
for multiple catalog consolidation, and subsumes the responsibility of
"tusk create-catalog". It also embodies my latest set of
recommendations for the catalog schema.

http://github.com/kriskowal/narwhal/blob/refactor/docs/catalog.md

Kris Kowal

Dean Landolt

unread,
Dec 16, 2009, 4:46:59 PM12/16/09
to comm...@googlegroups.com
On Wed, Dec 16, 2009 at 4:38 PM, Ash Berlin <ash_flu...@firemirror.com> wrote:

On 16 Dec 2009, at 21:22, mob wrote:

> I've mined the various information and pulled together a proposal for
> Packages 0.1. This proposal defines the package.json file, the package
> directory structure and package file format. It does not specify the
> catalog file (yet).
>
> I'd like to get feedback on this so we can iterate and then nail down
> this part of the packaging system. Then we can all create packages
> with confidence they will be usable in the various package managers,
> tools and loaders.
>
> I've tried to collect the various input from all, but I've also added
> a couple of new fields. Please point out any fields that are missing
> from the descriptor file.
>
> I've kept the package.json file fairly flat to make it easier for
> tools to inspect and modify fields.
>
> Enjoy
>
> --mob


What are the differences between categories and keywords? And do we really need either?

Author: being a single field - split it  into three (or an array of 3 elements) rather than requiring a fixed format that people will inevitably get wrong (and then only the name of the 3 is required).

+1

Dependencies: not sure the version handling is right. if we are using semantic versioning then isn't just [ "ejs", "1.0.0"] enough and implies the >= part and "< 2.0" is implicit from SemVer. (I also don't like '>=' as it implies '=' should work and I really dont want to get into the culture of letting people do that)

stability: again, doesn't SemVer cover this and make it redundant?

Does it? IIUC every release >= 1.0.0 is stable. That said I'm a little apprehensive to support adding a stability flag to the core packages standard as it would effectively augment the version number which could complicate things.


checksum: why [ "md5", "valuehere" ] and not just { "md5": "valuehere" } - its not like you can have more than one checksum of a given type, and this would make access easier.

I noticed quite a few things were tuples instead of hashes -- is there a good reason for this?

I'll review now and throw more comments out there, but thanks for getting the ball rolling and compiling this, mob.

mob

unread,
Dec 16, 2009, 5:08:06 PM12/16/09
to CommonJS
>
> What are the differences between categories and keywords? And do we really need either?

Keywords are free form. I need something to do with "md5" or "jsgi".

Categories are used when creating a web portal for the packages. When
we have 5,000 packages, a portal can present various categories such
as: "System Tools", "Cryptography", "Web Templating".

>
> Author: being a single field - split it  into three (or an array of 3 elements) rather than requiring a fixed format that people will inevitably get wrong (and then only the name of the 3 is required).

Okay. Sounds good.

>
> Dependencies: not sure the version handling is right. if we are using semantic versioning then isn't just [ "ejs", "1.0.0"] enough and implies the >= part and "< 2.0" is implicit from SemVer. (I also don't like '>=' as it implies '=' should work and I really dont want to get into the culture of letting people do that)

Yes, you can omit the upper bound version which implies 1.0.0 or
later. Sounds reasonable to omit the operator to mean ">=" for the
lower and "<=" for the upper. We could omit the operators and just
have 2 simple versions where the are:

LOWER <= VERSION <= UPPER


>
> stability: again, doesn't SemVer cover this and make it redundant?

Partially. There is quite a bit of legacy stuff not using SemVer. But
if folks are happy with SemVer, then it would be simpler to remove
"stability". I like simplicity.

>
> checksum: why [ "md5", "valuehere" ] and not just { "md5": "valuehere" } - its not like you can have more than one checksum of a given type, and this would make access easier.

Agree. Good pickup.

>
> builtin: is this needed?

It means you can't remove some packages. When a platform is installed,
it will typically contain many packages. These are intrinsic to the
platform. It is in the optional fields section. Most packages won't
use it.

>
> directories: purpose? And if its a hash what are the keys?

Ah, my last edit adding an example got deleted in the page rename.
I'll re-add that and it will be clearer.

>
> scripts: Hmmm I thought this was going to be list of scripts to install or some such. Perhaps "commands" or "build_commands" is a better name?

It is a list of scripts to run when installing, removing or building.

>
> Haven't yet had time to think though and see if there is anything missing.

Thanks

--mob
>
> -ash

mob

unread,
Dec 16, 2009, 5:10:16 PM12/16/09
to CommonJS
>
> Last Friday I did some work on Tusk's "tusk update" which adds support
> for multiple catalog consolidation, and subsumes the responsibility of
> "tusk create-catalog".  It also embodies my latest set of
> recommendations for the catalog schema.
>
> http://github.com/kriskowal/narwhal/blob/refactor/docs/catalog.md
>
> Kris Kowal

Can we do the Catalog after we settle on the package.json format?
Since the catalogue depends so heavily on it.

Kris: could you identify which of your package.json fields should be
in this spec. Some are custom fields for Narwhal, but I'd like to get
a complete set of fields here. Can you identify any candidates I've
missed?

--mob

mob

unread,
Dec 16, 2009, 5:13:06 PM12/16/09
to CommonJS

> Does it? IIUC every release >= 1.0.0 is *stable*. That said I'm a little
> apprehensive to support adding a stability flag to the core packages
> standard as it would effectively augment the version number which could
> complicate things.

It needs to be clean. But perhaps we can't achieve it cleanly. The
need is when I download a package into a production system, I'd like
to know if the developers think it is suitable for that. After a 1.0,
you could still have a period of rapid development where a package is
"evolving" and not suitable for production.

Versions are different, they are used for dependency resolution.
Stability is a guide to the user.


>
> > checksum: why [ "md5", "valuehere" ] and not just { "md5": "valuehere" } -
> > its not like you can have more than one checksum of a given type, and this
> > would make access easier.
>
> I noticed quite a few things were tuples instead of hashes -- is there a
> good reason for this?
>

Thanks. Ash picked this up too.

> I'll review now and throw more comments out there, but thanks for getting
> the ball rolling and compiling this, mob.

No problem, keep throwing them stones.

--mob

Kris Kowal

unread,
Dec 16, 2009, 5:34:30 PM12/16/09
to comm...@googlegroups.com
On Wed, Dec 16, 2009 at 1:38 PM, Ash Berlin
<ash_flu...@firemirror.com> wrote:

> What are the differences between categories and keywords? And do we really need either?

We should have keywords for "package-manager search keyword" commands,
like "pkgmgr search wiki markup". We might want to go so far as
having an evolving wiki page to recommend keywords for particular
types of services.

> Author: being a single field - split it  into three (or an array of 3 elements) rather than requiring a fixed format that people will inevitably get wrong (and then only the name of the 3 is required).

Tusk normalizes author, maintainer, and contributor fields to a {name,
url, email} hash. A string of comma separated terms might suffice
with a parser that can distinguish a web page from an email. The
author hash might be extended with some sort of unique identifier like
a Github user name for package managers to ensure consistency of
author fields across packages.

> Dependencies: not sure the version handling is right. if we are using semantic versioning then isn't just [ "ejs", "1.0.0"] enough and implies the >= part and "< 2.0" is implicit from SemVer. (I also don't like '>=' as it implies '=' should work and I really dont want to get into the culture of letting people do that)
> stability: again, doesn't SemVer cover this and make it redundant?

It's my hope that semantic versioning reduces the need for version
predicates as well.

> checksum: why [ "md5", "valuehere" ] and not just { "md5": "valuehere" } - its not like you can have more than one checksum of a given type, and this would make access easier.

+1. Might consider "hash" or "signature".

> builtin: is this needed?

I haven't found a need for this qualifier. Might be well enough to
the implementations.

> directories: purpose? And if its a hash what are the keys?

This is intended to replace the top-level "lib", "bin", "docs", &c
properties that Tusk uses. I think it's fine to make a special name
space for these properties; we're on the verge of name conflicts in
Tusk. It should be emphasized that these are intended to override
default locations by the same names, and that an array should be
provided if there are multiple directories that contain that kind of
file, like {"directories": {"bin": ["bin", "sbin"]}}.

> scripts: Hmmm I thought this was going to be list of scripts to install or some such. Perhaps "commands" or "build_commands" is a better name?

+1 to scripts to forestall bike shedding if possible.


On Wed, Dec 16, 2009 at 1:46 PM, Dean Landolt <de...@deanlandolt.com> wrote:
> On Wed, Dec 16, 2009 at 4:38 PM, Ash Berlin <ash_flu...@firemirror.com>
>> On 16 Dec 2009, at 21:22, mob wrote:

>> Author: being a single field - split it into three (or an array of 3
>> elements) rather than requiring a fixed format that people will inevitably
>> get wrong (and then only the name of the 3 is required).
>
> +1

+/- 0

>> stability: again, doesn't SemVer cover this and make it redundant?
> Does it? IIUC every release >= 1.0.0 is stable. That said I'm a little
> apprehensive to support adding a stability flag to the core packages
> standard as it would effectively augment the version number which could
> complicate things.

Let's leave "stability" off. It's purely informational. We can take
care of "stable", "unstable", and "test" by providing different
catalogs that offer different versions. If someone wants to volunteer
what catalog they would like to be included in, we might need to do
this differently. I had considered possibly advertising a "versions":
{} property and "trains": {stable, unstable, test, dev} with alternate
sources. My brain exploded and I had to scale back. Unless someone
can write a thesis on this issue right now, I think we should table
it.

> I noticed quite a few things were tuples instead of hashes -- is there a
> good reason for this?

I've regretted using tuples instead of hashes on several occasions. I
advise the use of Objects for any structural data instead of having
position imply the label.

> I'll review now and throw more comments out there, but thanks for getting
> the ball rolling and compiling this, mob.

Here here!

From http://wiki.commonjs.org/wiki/Packages/B

> It does not (yet) specify a package catalogue file or format.

Rule, Britania.

> * name - the name of the package. This must be a lowercase alpha-numeric name without spaces. It may include "." or "_" or "-" characters.

I recommend ditching "_" and ".".

> * description - a brief description of the package. By convention, the
first sentence should be usable as a package title in listings.

+1

> * version - a version string conforming to the Semantic Versioning
requirements (http://semver.org/)

+1

> * keywords - an Array of String keywords to assist users searching for the
package in catalogs.

+1

> * categories - an Array of String keywords to classify the package.

I think that we should wrap these into keywords and have catalog
maintainers encourgage consistency here.

> * author - Original or primary author of the package. Must be of the format:
> "Name (URL) <EMAIL>", where (URL) and <EMAIL> are optional but recommended. For
> example:
> "Bill Bloggs (http://www.bblogmedia.com) <bill...@bblogmedia.com)

I think we should allow this format, comma delimited, and an Object format,
all of which would get normalized to the Object.

> * contributors - an Array of strings each containing the details of a
> contributor. Format is the same as for author.

+1

> * bugs - URL for submitting bugs. Can be mailto or http.

I'm thinking we should so something similar to author for this one, where
"url" and "email" can be specified, comma delimited, or (urled) and <emailed>.

> * license - array of licenses under which the package is provided. Each
> license is a tuple where the first element is the kind of license and the
> second element is a URL to the license. For example
> licenses: [["GPL",
> "http://www.ejscript.org/products/ejs/doc/licenses/gpl.html"]]

I'm thinking we should have a standard set of license names, and then allow an
Object for custom variants with {"name", "url"} objects instead of tuples.

> * location - Array of repositories where the package can be located. Each repository is a tuple where the first element is the kind of repository and the second element is a URL path to clone/checkout the package. For example:
> location: [["mercurial", "http://hg.embedthis.com/ejs"]]

I think we ought to leave this out for now since it's highly coupled to the
catalog specification, I think. In my latest work on Tusk, the catalog
consolidation system automatically adds a "source" property to the package
descriptor which is the source descriptor that was used to get the rest of the
information. It has a "type" field and the rest of its properties depend on
the VCS or whether a VCS was used at all.

> * dependencies - Array of prerequisite packages on which this package depends in order to install and run. Each dependency is an array with one to three elements. The first element is the dependent package name and is mandatory. The second is an optional version expression defining the lowest qualifying version. The third element is a version expression defining the highest qualifying version. Version expressions are of the form: operator SPACE version. For example:
> dependencies: [ "ejs", ">= 1.0.0", "< 2.0"].

It does need to be an array. For the simple case, I think a mere package name
should be permitted, and the version "" implied, which should accept anything.
For the complex case, it should be an Object with something like {name,
version} or {name, min, max} if you think that's critical.

> This specifies a dependency on the "ejs" package with versions between 1.0 and 2.0. If the version are omitted, the latest available version of the dependency is acceptable.

Let's reserve "example" as a package name much like the reserved "example.com",
although I *do* think you deserve the free advertising :P

> * conforms - Array of relevant CommonJS specifications this package supports. TBD: define these specifiers. For example:
> conforms: [ "CommonJS-Modules-1.0", "CommonJS-JSGI-1.0"]

I recommend "implements" and that the value be a URI of the specification,
perhaps identical to its URL. Perhaps alternately its Wiki page name in case
we move.

> Optional Keywords
> * homepage - URL string for the package web site

+0. We can migrate from "url". Doesn't really matter as long as "url" isn't
reserved for something else.

> * os - Array of supported operating systems. If absent or set to the empty set, the package makes no platform assumptions. TBD: specify leading OS names.

+1

> * cpu - Array of supported CPU architectures. If absent or set to the empty set, the package makes no platform assumptions. TDB: specify leading CPU architectures.

+1

> * builtin- Boolean value indicating the package is built in as a standard component of the underlying platform

-1. It's fine for this to be implied by dependency, I think.

> * directories - Object hash of package directories. Typical directories include "lib", "src", "doc", "test" and "bin". Package manager tools SHOULD use these directory definitions to find various package components.

+1

> * scripts - Object hash of scripts used in managing the package. A package manager tool may use these scripts to install, build, test or uninstall the package. For example:
> scripts: {
> install: "install.js",
> uninstall: "uninstall.js",
> build: "build.js,
> }

+1

> Package managers and loaders should ignore unknown fields in the package descriptor file.

If we make this a normative clause, we should aggressively reserve any words we
will possibly need for the evolution of the specification. For example,
"schema", "extra", "package", "descriptor", "info", "docs", "documentation",
"reserved", "archive", "format", ...

> == Package File Format ==
>
> The package files shall be a simple archive of the package directory
> including the package.json file in a relative, ZIP archive format.

+1

> == Package Directory Layout ==
>
> A CommonJS package will observe the following:
> * A package.json file must be in the top level directory
> * Binary files SHOULD be in the "bin" directory,
> * Javascript code SHOULD be under the "lib" directory
> * Documentation SHOULD be under the "doc" directory
> * Unit tests SHOULD be under the "test" directory

s/doc/docs/ s/test/tests/ are the current practice in Narwhal.

The SHOULD needs to be qualified; if you use an alternate directory, it should
also be noted in the package.json directories property so that the package
manager and module loader can find these things.

There's also plenty of room to grow out the directory schema. I'm proposing a
"templates" and "media" down the line for web applications, with perhaps even a
structure under media. No need to specify that here, but it's something to
keep in mind.

> == Package Files ==
>
> To install and uninstall a CommonJS package some local installation steps may be required. A package MAY specify various scripts to run via the "scripts" package.json field.

+1



Kris Kowal

Kris Kowal

unread,
Dec 16, 2009, 5:41:24 PM12/16/09
to comm...@googlegroups.com
On Wed, Dec 16, 2009 at 2:10 PM, mob <m...@embedthis.com> wrote:
> Can we do the Catalog after we settle on the package.json format?
> Since the catalogue depends so heavily on it.

Yes, although a reference to the package descriptor schema should
suffice for that portion of the catalog format.

> Kris: could you identify which of your package.json fields should be
> in this spec. Some are custom fields for Narwhal, but I'd like to get
> a complete set of fields here. Can you identify any candidates I've
> missed?

Sure.

* maintainer
* directory.jars should be recommended even though it is not
generally applicable
* directory.engines should be in there for any system that has
engine-specific components for one or more engines, including general
engines like "default". We may or may not want to maintain a registry
for engine, os, cpu, and license names. We probably should, as a
freely-editable wiki page; "spidermonkey", "v8", "node", "rhino",
"mozilla", "jsc", "flusspferd", "narwhal", "gpsee". It's like a style
sheet.
* async needs to be there as a recommendation to package maintainers
or users who are trying to maintain the "no-sync" ideology.

Kris Kowal

mob

unread,
Dec 16, 2009, 5:52:32 PM12/16/09
to CommonJS
On Dec 16, 2:34 pm, Kris Kowal <cowbertvon...@gmail.com> wrote:

> We should have keywords for "package-manager search keyword" commands,
> like "pkgmgr search wiki markup".  We might want to go so far as
> having an evolving wiki page to recommend keywords for particular
> types of services.

Agree, that was my intent/understanding of the keywords field. Are you
advocating then to remove categories?


> > Author: being a single field - split it  into three (or an array of 3 elements) rather than requiring a fixed format that people will inevitably get wrong (and then only the name of the 3 is required).
>
> Tusk normalizes author, maintainer, and contributor fields to a {name,
> url, email} hash.  A string of comma separated terms might suffice
> with a parser that can distinguish a web page from an email.  The
> author hash might be extended with some sort of unique identifier like
> a Github user name for package managers to ensure consistency of
> author fields across packages.

Okay, based on feedback, lets make it a hash of name, email, web. Same
for contributors

>
> > Dependencies: not sure the version handling is right. if we are using semantic versioning then isn't just [ "ejs", "1.0.0"] enough and implies the >= part and "< 2.0" is implicit from SemVer. (I also don't like '>=' as it implies '=' should work and I really dont want to get into the culture of letting people do that)
> > stability: again, doesn't SemVer cover this and make it redundant?
>

Okay, lets remove version operators and take a stance with SemVer.


> > directories: purpose? And if its a hash what are the keys?
>
> This is intended to replace the top-level "lib", "bin", "docs", &c
> properties that Tusk uses.  I think it's fine to make a special name
> space for these properties; we're on the verge of name conflicts in
> Tusk.  It should be emphasized that these are intended to override
> default locations by the same names, and that an array should be
> provided if there are multiple directories that contain that kind of
> file, like {"directories": {"bin": ["bin", "sbin"]}}.

AGREE. This is optional and only required if you have a non-standard
dir layout.

>
> > scripts: Hmmm I thought this was going to be list of scripts to install or some such. Perhaps "commands" or "build_commands" is a better name?
>
> +1 to scripts to forestall bike shedding if possible.

Yippee.

> Let's leave "stability" off.  It's purely informational.  We can take
> care of "stable", "unstable", and "test" by providing different
> catalogs that offer different versions.  If someone wants to volunteer
> what catalog they would like to be included in, we might need to do
> this differently.  I had considered possibly advertising a "versions":
> {} property and "trains": {stable, unstable, test, dev} with alternate
> sources.  My brain exploded and I had to scale back.  Unless someone
> can write a thesis on this issue right now, I think we should table
> it.

Uncle.

> I've regretted using tuples instead of hashes on several occasions.  I
> advise the use of Objects for any structural data instead of having
> position imply the label.

Agree.

> > * name - the name of the package. This must be a lowercase alpha-numeric name without spaces. It may include "." or "_" or "-" characters.
>
> I recommend ditching "_" and ".".

We need "." in ejscript. Please... We could recommend that the names
be camelCase if multi-word instead of "_". But do we really care
here? They are opaque names.

> > * categories - an Array of String keywords to classify the package.
> I think that we should wrap these into keywords and have catalog
> maintainers encourgage consistency here.

Agreed above.

>
> > * author - Original or primary author of the package. Must be of the format:
> > "Name (URL) <EMAIL>", where (URL) and <EMAIL> are optional but recommended. For
> > example:
> >     "Bill Bloggs (http://www.bblogmedia.com) <billbl...@bblogmedia.com)
>
> I think we should allow this format, comma delimited, and an Object format,
> all of which would get normalized to the Object.

Given the feedback, I'd recommend we make this a hash and not allow
dual formats. {name,email,web}

> > * bugs - URL for submitting bugs. Can be mailto or http.
>
> I'm thinking we should so something similar to author for this one, where
> "url" and "email" can be specified, comma delimited, or (urled) and <emailed>.

Make sense. How about:

bugs: {
mail: "bugs@acme",
http: "....",
}

>
> > * license - array of licenses under which the package is provided. Each
> > license is a tuple where the first element is the kind of license and the
> > second element is a URL to the license. For example
> >     licenses: [["GPL",
> > "http://www.ejscript.org/products/ejs/doc/licenses/gpl.html"]]
>
> I'm thinking we should have a standard set of license names, and then allow an
> Object for custom variants with {"name", "url"} objects instead of tuples.

It is a bit of a zoo. I tried this, but there are so many different
licenses.

>
> > * location - Array of repositories where the package can be located. Each repository is a tuple where the first element is the kind of repository and the second element is a URL path to clone/checkout the package. For example:
> >     location: [["mercurial", "http://hg.embedthis.com/ejs"]]
>
> I think we ought to leave this out for now since it's highly coupled to the
> catalog specification, I think.  In my latest work on Tusk, the catalog
> consolidation system automatically adds a "source" property to the package
> descriptor which is the source descriptor that was used to get the rest of the
> information.  It has a "type" field and the rest of its properties depend on
> the VCS or whether a VCS was used at all.

I was thinking this is the authoritative source for the package not
necessarily where a catalogue may be proxying it. So I think we should
retain this here. Tusk could still add a source which is where this
particular download came from.

> > * dependencies - Array of prerequisite packages on which this package depends in order to install and run. Each dependency is an array with one to  three elements. The first element is the dependent package name and is mandatory. The second is an optional version expression defining the lowest qualifying version. The third element is a version expression defining the highest qualifying version. Version expressions are of the form: operator SPACE version. For example:
> >     dependencies: [ "ejs", ">= 1.0.0", "< 2.0"].
>
> It does need to be an array.  For the simple case, I think a mere package name
> should be permitted, and the version "" implied, which should accept anything.

Agreed.

> For the complex case, it should be an Object with something like {name,
> version} or {name, min, max} if you think that's critical.

I might wait for more input on whether this should be a tuple or hash.
I can see your point.


>
> > * conforms - Array of relevant CommonJS specifications this package supports. TBD: define these specifiers. For example:
> >    conforms: [ "CommonJS-Modules-1.0", "CommonJS-JSGI-1.0"]
>
> I recommend "implements" and that the value be a URI of the specification,
> perhaps identical to its URL.  Perhaps alternately its Wiki page name in case
> we move.

Better.

> > Package managers and loaders should ignore unknown fields in the package descriptor file.
>
> If we make this a normative clause, we should aggressively reserve any words we ...

Agree.

Thanks for the detailed feedback. Appreciated.

--mob


mob

unread,
Dec 16, 2009, 6:06:47 PM12/16/09
to CommonJS
> > Can you identify any candidates I've
> > missed?
>
>  * maintainer

Is this the point contributor to receive issue requests? Can you
elaborate on the purpose. I tried to cut down to author and
contributors. Can we refactor these to only have 2 fields for these.
ie. pick 2 from (author, contributors, maintainer). Or refactor in
some way to have two instead of three.

>  * directory.jars should be recommended even though it is not
> generally applicable

I think this is an example of a collection of things which are not
required, but if present should adopt this form. I'll create a section
for these.

>  * directory.engines should be in there for any system that has
> engine-specific components for one or more engines, including general
> engines like "default".  We may or may not want to maintain a registry
> for engine, os, cpu, and license names.  We probably should, as a
> freely-editable wiki page; "spidermonkey", "v8", "node", "rhino",
> "mozilla", "jsc", "flusspferd", "narwhal", "gpsee".  It's like a style
> sheet.

Yes, this is just the same as os and cpu. I'll create a list of
approved names to the proposal.

>  * async needs to be there as a recommendation to package maintainers
> or users who are trying to maintain the "no-sync" ideology.

This doesn't quite feel right. I know the need, but there will always
be a long list of things like this. Surely this is the same as
"stability" and the answers are the same. ie. we may need an async-
only catalog.

--mob

Dean Landolt

unread,
Dec 16, 2009, 6:21:44 PM12/16/09
to comm...@googlegroups.com
Async would make a great keyword :)

Ash Berlin

unread,
Dec 16, 2009, 6:28:12 PM12/16/09
to comm...@googlegroups.com
[Two replies merged into one here]
On 16 Dec 2009, at 23:06, mob wrote:

Can you identify any candidates I've missed?

 * maintainer

Is this the point contributor to receive issue requests?  Can you
elaborate on the purpose. I tried to cut down to author and
contributors. Can we refactor these to only have 2 fields for these.
ie. pick 2 from (author, contributors, maintainer). Or refactor in
some way to have two instead of three.

Is author relevant at the package.json level, or is that just something that is better in the package/module's documentation?


 * directory.jars should be recommended even though it is not
generally applicable

I think this is an example of a collection of things which are not
required, but if present should adopt this form. I'll create a section
for these.

 * directory.engines should be in there for any system that has
engine-specific components for one or more engines, including general
engines like "default".  We may or may not want to maintain a registry
for engine, os, cpu, and license names.  We probably should, as a
freely-editable wiki page; "spidermonkey", "v8", "node", "rhino",
"mozilla", "jsc", "flusspferd", "narwhal", "gpsee".  It's like a style
sheet.

Yes, this is just the same as os and cpu. I'll create a list of
approved names to the proposal.

As a corollary to this how do you signal that "this package only works on engine Y"?


 * async needs to be there as a recommendation to package maintainers
or users who are trying to maintain the "no-sync" ideology.

This doesn't quite feel right. I know the need, but there will always
be a long list of things like this. Surely this is the same as
"stability" and the answers are the same. ie. we may need an async-
only catalog.

Or this is a keyword/category? (as dean said while I was composing this)


On 16 Dec 2009, at 22:52, mob wrote:

On Dec 16, 2:34 pm, Kris Kowal <cowbertvon...@gmail.com> wrote:

We should have keywords for "package-manager search keyword" commands,
like "pkgmgr search wiki markup".  We might want to go so far as
having an evolving wiki page to recommend keywords for particular
types of services.

Agree, that was my intent/understanding of the keywords field. Are you
advocating then to remove categories?

Unless we enumerate the values for one of these they are both just free-form inputs. So I think we either need to enumerate one and have the other as freeform, or remove one of them and have the remaining as 'free form'.

Dependencies: not sure the version handling is right. if we are using semantic versioning then isn't just [ "ejs", "1.0.0"] enough and implies the >= part and "< 2.0" is implicit from SemVer. (I also don't like '>=' as it implies '=' should work and I really dont want to get into the culture of letting people do that)
stability: again, doesn't SemVer cover this and make it redundant?


Okay, lets remove version operators and take a stance with SemVer.

Going with just a single version (impliying v2 is not compatible with v1, correct?)


scripts: Hmmm I thought this was going to be list of scripts to install or some such. Perhaps "commands" or "build_commands" is a better name?

+1 to scripts to forestall bike shedding if possible.

Yippee.

Good point: name is fine as it is. Is there a known/define set of keys on scripts? Just the 4 listed?

* name - the name of the package. This must be a lowercase alpha-numeric name without spaces. It may include "." or "_" or "-" characters.

I recommend ditching "_" and ".".

We need "." in ejscript. Please...  We could recommend that the names
be camelCase if multi-word instead of "_". But do we really care
here?  They are opaque names.

Agreed - name should be as unrestricted in this spec as possible. Why arbitrarily limit it? (in particular why no uppercase chars?)

* bugs - URL for submitting bugs. Can be mailto or http.

I'm thinking we should so something similar to author for this one, where
"url" and "email" can be specified, comma delimited, or (urled) and <emailed>.

Make sense. How about:

bugs: {
   mail: "bugs@acme",
   http:  "....",
}

Sounds good. Optionally allow just a simple string where in it looks for "@" as an email, or else treats as a URL (this might be more complex than is needed)?

* license - array of licenses under which the package is provided. Each
license is a tuple where the first element is the kind of license and the
second element is a URL to the license. For example
    licenses: [["GPL",
"http://www.ejscript.org/products/ejs/doc/licenses/gpl.html"]]

I'm thinking we should have a standard set of license names, and then allow an
Object for custom variants with {"name", "url"} objects instead of tuples.

It is a bit of a zoo. I tried this, but there are so many different
licenses.

Licenses are a bit of a zoo, but there are some common ones: GPLv2, GPLv3, LGPL, MIT, BSD, Apache, Mozilla would be the most common, and anything not in that list needs {"name": "url"}

* location - Array of repositories where the package can be located. Each repository is a tuple where the first element is the kind of repository and the second element is a URL path to clone/checkout the package. For example:
    location: [["mercurial", "http://hg.embedthis.com/ejs"]]

I think we ought to leave this out for now since it's highly coupled to the
catalog specification, I think.  In my latest work on Tusk, the catalog
consolidation system automatically adds a "source" property to the package
descriptor which is the source descriptor that was used to get the rest of the
information.  It has a "type" field and the rest of its properties depend on
the VCS or whether a VCS was used at all.

I was thinking this is the authoritative source for the package not
necessarily where a catalogue may be proxying it. So I think we should
retain this here. Tusk could still add a source which is where this
particular download came from.

So the package.json should include VCS repository locations, but yeah, maybe not the location where you can get releases from.


* dependencies - Array of prerequisite packages on which this package depends in order to install and run. Each dependency is an array with one to  three elements. The first element is the dependent package name and is mandatory. The second is an optional version expression defining the lowest qualifying version. The third element is a version expression defining the highest qualifying version. Version expressions are of the form: operator SPACE version. For example:
    dependencies: [ "ejs", ">= 1.0.0", "< 2.0"].

It does need to be an array.  For the simple case, I think a mere package name
should be permitted, and the version "" implied, which should accept anything.

Agreed.

For the complex case, it should be an Object with something like {name,
version} or {name, min, max} if you think that's critical.

I might wait for more input on whether this should be a tuple or hash.
I can see your point.

I'd vote for not allowing complex cases :) This is what SemVer is supposed to handle (at least by my reading of it).

Wheeeee long!

-ash

Ash Berlin

unread,
Dec 16, 2009, 6:34:18 PM12/16/09
to comm...@googlegroups.com

On 16 Dec 2009, at 22:10, mob wrote:

> Can you identify any candidates I've missed?

So ideally I'd like some way of signalling a dependancy on 'external libraries or programs' (such as libmysql or a particular build system). The package management tools might not be able to provide/fulfil this dependancy in all cases (or even in most cases) but they could at least signal to the user or allow it to be displayed in a nice format.

-ash

Dean Landolt

unread,
Dec 16, 2009, 6:35:29 PM12/16/09
to comm...@googlegroups.com

Because of Windows-pain, of course ;)
 

* bugs - URL for submitting bugs. Can be mailto or http.

I'm thinking we should so something similar to author for this one, where
"url" and "email" can be specified, comma delimited, or (urled) and <emailed>.

Make sense. How about:

bugs: {
   mail: "bugs@acme",
   http:  "....",
}

Sounds good. Optionally allow just a simple string where in it looks for "@" as an email, or else treats as a URL (this might be more complex than is needed)?

* license - array of licenses under which the package is provided. Each
license is a tuple where the first element is the kind of license and the
second element is a URL to the license. For example
    licenses: [["GPL",
"http://www.ejscript.org/products/ejs/doc/licenses/gpl.html"]]

I'm thinking we should have a standard set of license names, and then allow an
Object for custom variants with {"name", "url"} objects instead of tuples.

It is a bit of a zoo. I tried this, but there are so many different
licenses.

Licenses are a bit of a zoo, but there are some common ones: GPLv2, GPLv3, LGPL, MIT, BSD, Apache, Mozilla would be the most common, and anything not in that list needs {"name": "url"}

And as Kris pointed out, we could keep an authoritative json map (perhaps built from some wiki page) to extend this. This is actually something the internets need in general and perhaps could eventually be administered by a group outside of CommonJS.
 

--

You received this message because you are subscribed to the Google Groups "CommonJS" group.
To post to this group, send email to comm...@googlegroups.com.
To unsubscribe from this group, send email to commonjs+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/commonjs?hl=en.

mob

unread,
Dec 16, 2009, 6:40:29 PM12/16/09
to CommonJS
> >>  * maintainer
> > Is this the point contributor to receive issue requests?  Can you
> > elaborate on the purpose. I tried to cut down to author and
> > contributors. Can we refactor these to only have 2 fields for these.
> > ie. pick 2 from (author, contributors, maintainer). Or refactor in
> > some way to have two instead of three.
>
> Is author relevant at the package.json level, or is that just something that is better in the package/module's documentation?

Could be right. The original author is not nearly as relevant as the
current maintainer. How about we use those 2: maintainer and
contributors. The author could be the first contributor by convention?


> As a corollary to this how do you signal that "this package only works on engine Y"?

Create a psudo package for the engine and depend on it.

For ejscript, we will have a package for each key component of the
platform.

> Or this is a keyword/category? (as dean said while I was composing this)

Agree.

> Unless we enumerate the values for one of these they are both just free-form inputs. So I think we either need to enumerate one and have the other as freeform, or remove one of them and have the remaining as 'free form'.

I'm now leaning this way. Suggest we remove categories and just have
keywords. Simpler and then the package tools can take over from there.

> >>> Dependencies: not sure the version handling is right. if we are using semantic versioning then isn't just [ "ejs", "1.0.0"] enough and implies the >= part and "< 2.0" is implicit from SemVer. (I also don't like '>=' as it implies '=' should work and I really dont want to get into the culture of letting people do that)
> >>> stability: again, doesn't SemVer cover this and make it redundant?
>
> > Okay, lets remove version operators and take a stance with SemVer.
>
> Going with just a single version (impliying v2 is not compatible with v1, correct?)

We have a choice with that. One version could mean:
- That version or later
- That major version only.

I'm leaning toward the former. If someone says "package", "1.2.3".
What they really mean is 1.2.3 or later.
If you want it to mean only version 1, you would say

"package", "1", "2"

> >>> scripts: Hmmm I thought this was going to be list of scripts to install or some such. Perhaps "commands" or "build_commands" is a better name?
> >> +1 to scripts to forestall bike shedding if possible.

> Good point: name is fine as it is. Is there a known/define set of keys on scripts? Just the 4 listed?

I think this will grow from experience. We use install, uninstall and
build. But other platforms may need others. Perhaps "test"?

> > Make sense. How about:
>
> > bugs: {
> >    mail: "bugs@acme",
> >    http:  "....",
> > }
>
> Sounds good. Optionally allow just a simple string where in it looks for "@" as an email, or else treats as a URL (this might be more complex than is needed)?

I think so, I'd avocate that we don't allow such simplifications. The
above is very clear and it just makes parsers and tools harder to
write and maintain.


> >>> * license - array of licenses under which the package is provided. Each
> >>> license is a tuple where the first element is the kind of license and the
> >>> second element is a URL to the license. For example
> >>>     licenses: [["GPL",
> >>> "http://www.ejscript.org/products/ejs/doc/licenses/gpl.html"]]
>
> >> I'm thinking we should have a standard set of license names, and then allow an
> >> Object for custom variants with {"name", "url"} objects instead of tuples.
>
> > It is a bit of a zoo. I tried this, but there are so many different
> > licenses.
>
> Licenses are a bit of a zoo, but there are some common ones: GPLv2, GPLv3, LGPL, MIT, BSD, Apache, Mozilla would be the most common, and anything not in that list needs {"name": "url"}

Okay how about:

license: {
kind: "GPLv2",
url: "http://www.example.org/licenses/gpl.html"
},

Where the standard kinds are what you listed++

Thanks ash.

mob

unread,
Dec 16, 2009, 6:51:27 PM12/16/09
to CommonJS
Posted an update with most of the feedback so far. Please yell out if
I've missed anything.

http://wiki.commonjs.org/wiki/Packages/B

--mob

Christoph Dorn

unread,
Dec 16, 2009, 6:53:57 PM12/16/09
to comm...@googlegroups.com
Ash Berlin wrote:
> [Two replies merged into one here]
> On 16 Dec 2009, at 23:06, mob wrote:
>>>> * location - Array of repositories where the package can be located.
>>>> Each repository is a tuple where the first element is the kind of
>>>> repository and the second element is a URL path to clone/checkout
>>>> the package. For example:
>>>> location: [["mercurial", "http://hg.embedthis.com/ejs"]]
>>
>> I was thinking this is the authoritative source for the package not
>> necessarily where a catalogue may be proxying it. So I think we should
>> retain this here. Tusk could still add a source which is where this
>> particular download came from.
>
> So the package.json should include VCS repository locations, but yeah,
> maybe not the location where you can get releases from.

+1 I think it is essential to be able to easily find the original source
of a package to encourage collaboration no matter how it was distributed.

If bug reports always target the original source it may make sense to
combine "bugs" and "location".


>> * version - a version string conforming to the Semantic Versioning
> requirements (http://semver.org/)

As for versioning and stability. Being able to declare a 2.X release as
alpha or beta is important I think until it becomes stable. Can we
extend the Semantic Versioning with this:

https://developer.mozilla.org/en/Toolkit_version_format


>> Package managers and loaders should ignore unknown fields in the
>> package descriptor file.
>

> "schema", "extra", "package", "descriptor", "info", "docs",
> "documentation", "reserved", "archive", "format", ...

Please add: "using", "build"

Christoph


Christoph Dorn

unread,
Dec 16, 2009, 6:56:29 PM12/16/09
to comm...@googlegroups.com
mob wrote:
>>>>> scripts: Hmmm I thought this was going to be list of scripts to install or some such. Perhaps "commands" or "build_commands" is a better name?
>>>> +1 to scripts to forestall bike shedding if possible.
>> Good point: name is fine as it is. Is there a known/define set of keys on scripts? Just the 4 listed?
>
> I think this will grow from experience. We use install, uninstall and
> build. But other platforms may need others. Perhaps "test"?

Yes! Add "test" please.

Christoph

Ash Berlin

unread,
Dec 16, 2009, 7:06:02 PM12/16/09
to comm...@googlegroups.com
On 16 Dec 2009, at 23:53, Christoph Dorn wrote:

> Ash Berlin wrote:
>>
>>
>> So the package.json should include VCS repository locations, but yeah,
>> maybe not the location where you can get releases from.
>
> +1 I think it is essential to be able to easily find the original source
> of a package to encourage collaboration no matter how it was distributed.
>
> If bug reports always target the original source it may make sense to
> combine "bugs" and "location".

Hmmm yes that could work.

>
>>> * version - a version string conforming to the Semantic Versioning
>> requirements (http://semver.org/)
>
> As for versioning and stability. Being able to declare a 2.X release as
> alpha or beta is important I think until it becomes stable. Can we
> extend the Semantic Versioning with this:
>
> https://developer.mozilla.org/en/Toolkit_version_format

Or there is the .deb form of 2.0~1 (which is < 2.0 iirc.) In general I think a way of putting developer/preview versions up for explicit download is a good thing (so long as they need to be explicitly required, and by default the package installer only looks for normal versions).

Christoph Dorn

unread,
Dec 16, 2009, 7:08:36 PM12/16/09
to comm...@googlegroups.com

signature: {
md5: "719ea61444a0bd34c9cf7454227a2e5c"
}

How is the signature generated? If it is for the archive file and the
archive file includes package.json is that not going to be a problem?

Christoph


Dean Landolt

unread,
Dec 16, 2009, 7:09:27 PM12/16/09
to comm...@googlegroups.com

SemVer is one day old and we're already talking about extending it...heh...

But this is relevant version info and I can't see how (after 1.0.0) it's possible to express this. Not being able to cut a beta release package would be pretty constraining, especially for slow-moving projects. One comparatively easy way to extend SemVer is to allow letters and specify the sort as letters < numbers. Then you could have 2.a.1, 2.a.2, 2.b.1, 2.0.0, etc... You could also have use this to create a train for minor releases (2.2.a, 2.2.b, 2.2.c... all sorting before 2.2.0). Dependencies would never resolve to an extended SemVar because they're technically invalid and thus unstable.

Kris Kowal

unread,
Dec 16, 2009, 7:11:39 PM12/16/09
to comm...@googlegroups.com
On Wed, Dec 16, 2009 at 3:40 PM, mob <m...@embedthis.com> wrote:
> I'm leaning toward the former. If someone says "package", "1.2.3".
> What they really mean is 1.2.3 or later.

Actually, if they say "1.2", they mean greater or equal to "1.2.0" but
less than "2". Major versions introduce changes that require code
changes. Minor versions introduce changes that are strictly backward
compatible. Patch versions are backward and forward compatible and
the most recent is always the best choice. I think all the major
cases are covered.

Kris Kowal

Kris Kowal

unread,
Dec 16, 2009, 7:13:19 PM12/16/09
to comm...@googlegroups.com
On Wed, Dec 16, 2009 at 3:40 PM, mob <m...@embedthis.com> wrote:
> Okay how about:
>
>    license: {
>        kind: "GPLv2",
>        url: "http://www.example.org/licenses/gpl.html"
>    },

s/kind/type/g since they're synonyms with the latter being the
"codish" term, or s/kind/name/ if we assure that there is only one
license for each name.

Kris Kowal

Ash Berlin

unread,
Dec 16, 2009, 7:16:40 PM12/16/09
to comm...@googlegroups.com

One of the things that the 'python want a CPAN' thread was that the module metadata (which in our case is package.json) should live outside of the package. or at least be available outside it. In which case the one inside it could just not have signatures section. This does slightly complicate the upload process abit.

-ash

Christoph Dorn

unread,
Dec 16, 2009, 7:17:40 PM12/16/09
to comm...@googlegroups.com
Dean Landolt wrote:
> On Wed, Dec 16, 2009 at 6:53 PM, Christoph Dorn

> SemVer is one day old and we're already talking about extending it...heh...
>
> But this /is/ relevant version info and I can't see how (after 1.0.0)

> it's possible to express this. Not being able to cut a beta release
> package would be pretty constraining, especially for slow-moving
> projects. One comparatively easy way to extend SemVer is to allow
> letters and specify the sort as letters < numbers. Then you could have
> 2.a.1, 2.a.2, 2.b.1, 2.0.0, etc... You could also have use this to
> create a train for minor releases (2.2.a, 2.2.b, 2.2.c... all sorting

> before 2.2.0). Dependencies would never resolve to an /extended/ SemVar

> because they're technically invalid and thus unstable.

If you can create a train with 2.2.a it is only really useful if you can
resolve dependencies to it. I think it should depend on how your
dependency is defined (i.e. does it include the letter or not)

Christoph


Kris Kowal

unread,
Dec 16, 2009, 7:20:14 PM12/16/09
to comm...@googlegroups.com

Ah, it is probably technically impossible for the signature to be
inside the package anyway. Let's leave this out as something that
gets added to a package descriptor in a catalog.

Of course, this means that we'll have to be careful about the order in
which we add packages to the catalog for the using-packages style,
since the signature of each dependency needs to be in the package.json
of its dependees. Also means there can't be cyclic dependencies among
packages.

Kris Kowal

Dean Landolt

unread,
Dec 16, 2009, 7:20:53 PM12/16/09
to comm...@googlegroups.com

I meant dependencies would never resolve automagically -- for instance, if the package manager had upgrade functionality. The specific package would have to be explicitly referenced in order to be used (depending on the release habits of the author an awful lot of projects point to betas).

ihab...@gmail.com

unread,
Dec 16, 2009, 7:26:14 PM12/16/09
to comm...@googlegroups.com
On Wed, Dec 16, 2009 at 4:20 PM, Kris Kowal <cowber...@gmail.com> wrote:
> ... Also means there can't be cyclic dependencies among packages.

That's true for checksums, but signatures? As in, "this package must
be signed by Kris Kowal"?

Ihab

--
Ihab A.B. Awad, Palo Alto, CA

Dean Landolt

unread,
Dec 16, 2009, 7:28:31 PM12/16/09
to comm...@googlegroups.com

Just to elaborate on this, I only propose allowing letters and specifying the sort as a means to give package maintainers more flexibility for cutting releases. I don't think we should really specify what "a" means or where it should be -- these are, after all, still illegal SemVer versions so they have no meaning to a package system -- they just give a means by which to cut unstable releases and control the sort order of those releases.

I was half joking when I first starting writing that proposal but the more I think about it the more I like it.

mob

unread,
Dec 16, 2009, 7:31:59 PM12/16/09
to CommonJS

> > So the package.json should include VCS repository locations, but yeah,
> > maybe not the location where you can get releases from.
>
> +1 I think it is essential to be able to easily find the original source
> of a package to encourage collaboration no matter how it was distributed.
>
> If bug reports always target the original source it may make sense to
> combine "bugs" and "location".

What do you mean combine? Can you please give an example?

>  >> * version - a version string conforming to the Semantic Versioning
>  > requirements (http://semver.org/)
>
> As for versioning and stability. Being able to declare a 2.X release as
> alpha or beta is important I think until it becomes stable. Can we
> extend the Semantic Versioning with this:
>
>    https://developer.mozilla.org/en/Toolkit_version_format

Specifically, do you mean append suffixes to the digit portions?
It is a shame to mess with SemVer.

>  >> Package managers and loaders should ignore unknown fields in the
>  >> package descriptor file.
>  >
>  > "schema", "extra", "package", "descriptor", "info", "docs",
>  > "documentation", "reserved", "archive", "format", ...
>
> Please add: "using", "build"

Sure.

--mob

mob

unread,
Dec 16, 2009, 7:34:31 PM12/16/09
to CommonJS

Good catch.

The package.json lives inside the package AND should be present
outside the package in a catalog. This should be specified as only
existing in free standing package.json files when they are part of a
catalog.

--mob

mob

unread,
Dec 16, 2009, 7:36:12 PM12/16/09
to CommonJS
> SemVer is one day old and we're already talking about extending it...heh...
>
> But this *is* relevant version info and I can't see how (after 1.0.0) it's

> possible to express this. Not being able to cut a beta release package would
> be pretty constraining, especially for slow-moving projects. One
> comparatively easy way to extend SemVer is to allow letters and specify the
> sort as letters < numbers. Then you could have 2.a.1, 2.a.2, 2.b.1, 2.0.0,
> etc... You could also have use this to create a train for minor releases
> (2.2.a, 2.2.b, 2.2.c... all sorting before 2.2.0). Dependencies would never
> resolve to an *extended* SemVar because they're technically invalid and thus
> unstable.

That is fairly neat. Another similar proposal is to allow any alpha
suffix. ie.

1.0.0beta4
1.0b2

Parsing is easy to just parse as a number, the suffix will be ignored.

--mob

mob

unread,
Dec 16, 2009, 7:40:16 PM12/16/09
to CommonJS

Agree that sometimes that is the case. The use case I was meaning if I
have a package that needs feature X in package y which was introduced
in version 1.3. ie. I depend on any version >= 1.3.

With the meaning you propose, my package expires as the dependent
package increases its version. I'd like to be able to say I depend on
any version after 1.3 and not have to say:

"y", "1.3", "99.99.999"

I think the meaning of SemVer which says my package may break if Y
changes to a new major release is a "may break" not a must break. So
package managers should load the latest version of a package that
qualifies. If a user wants to lock to a given major version, they
should say:

"Y", "1.3", "2.0"

That then locks it to 1.X.

--mob

Ash Berlin

unread,
Dec 16, 2009, 7:44:21 PM12/16/09
to comm...@googlegroups.com

Thats kinda what i suggested in irc, but with one alteration:

"If the version isn't a valid SemVer, then the this version is treated as a developer/beta release and will only ever by installed by sepcifically asking for this version".


mob

unread,
Dec 16, 2009, 7:45:10 PM12/16/09
to CommonJS
>
> >    license: {
> >        kind: "GPLv2",
> >        url: "http://www.example.org/licenses/gpl.html"
> >    },
>
> s/kind/type/g since they're synonyms with the latter being the
> "codish" term, or s/kind/name/ if we assure that there is only one
> license for each name.


Good

--mob

Kris Kowal

unread,
Dec 16, 2009, 7:45:39 PM12/16/09
to comm...@googlegroups.com
On Wed, Dec 16, 2009 at 4:26 PM, <ihab...@gmail.com> wrote:
> On Wed, Dec 16, 2009 at 4:20 PM, Kris Kowal <cowber...@gmail.com> wrote:
>> ... Also means there can't be cyclic dependencies among packages.
>
> That's true for checksums, but signatures? As in, "this package must
> be signed by Kris Kowal"?

Depends on whether we're talking about:

1. a declaration that insists that some other package be signed, or
2. a declaration that this package has been signed

I'm suggesting that 2 is paradoxical, for both the crypto hash and
crypto sign cases. The "signature" property in a "package.json" would
be the 2nd case. The "using" property of a package descriptor would
be the first.

M.O'B., that's another thing we need to add to the spec, and Christoph
can speak to the point: a schema for a "using" property for
using-style packages, which we haven't yet ratified, but baby steps.

Kris Kowal

mob

unread,
Dec 16, 2009, 7:46:33 PM12/16/09
to CommonJS
> If you can create a train with 2.2.a it is only really useful if you can
> resolve dependencies to it. I think it should depend on how your
> dependency is defined (i.e. does it include the letter or not)

That may get too complicated.

I'd say allow suffixes, but dependencies work on the pure numbers
only.

--mob

mob

unread,
Dec 16, 2009, 7:47:46 PM12/16/09
to CommonJS
> Of course, this means that we'll have to be careful about the order in
> which we add packages to the catalog for the using-packages style,
> since the signature of each dependency needs to be in the package.json
> of its dependees.  Also means there can't be cyclic dependencies among
> packages.

To keep it simple, let's only ever have signatures in catalogs. Never
in package.json.
Then it is the job of the package manager tool that has access to the
catalog to check on a download.

--mob

Ash Berlin

unread,
Dec 16, 2009, 7:50:31 PM12/16/09
to comm...@googlegroups.com

Another idea (of something I wish i could do in perl but can't):

Dependancy groups: "I need *one* of these set of modules, if any are installed great, else pick the first in the list and install that"

This has been a pain many times in my life as a perl developer.

-ash

ihab...@gmail.com

unread,
Dec 16, 2009, 7:53:40 PM12/16/09
to comm...@googlegroups.com
On Wed, Dec 16, 2009 at 4:45 PM, Kris Kowal <cowber...@gmail.com> wrote:
> Depends on whether we're talking about:
>
>  1. a declaration that insists that some other package be signed, or
>  2. a declaration that this package has been signed
>
> I'm suggesting that 2 is paradoxical, for both the crypto hash and
> crypto sign cases.

There are a number of paradoxes I see. Maybe it's just me. :)

For case 1, for *hashes*, if

package A says "import package B having hash X"
package B says "import package A having hash Y"

there is no way to simultaneously compute X and Y (that I know of...).

For case 2, the problem can be solved by signing *part* of the
package, not the package itself. For example, the signatures in Java
JAR files are for individual files, but not the whole thing. With that
in mind, one could adopt some similar packaging convention that
applies signatures to individual files -- including any ancillary
JSON.

Kris Kowal

unread,
Dec 16, 2009, 7:55:13 PM12/16/09
to comm...@googlegroups.com

This sounds more like a spec dependency than an implementation
dependency. That is, it's related more to our "implements" property
than our "dependencies" property. More than one package might conform
to a specification. We need a property to permit a package to state
that it depends on a particular specification that can be fulfilled by
multiple packages. How the installer handles that is a different
issue. I suggest that the user be presented with the choices through
a package search and be required to explicitly include one of them in
the next install line.

Any ideas for the property name?

Kris Kowal

ihab...@gmail.com

unread,
Dec 16, 2009, 7:57:26 PM12/16/09
to comm...@googlegroups.com
On Wed, Dec 16, 2009 at 4:55 PM, Kris Kowal <cowber...@gmail.com> wrote:
> We need a property to permit a package to state
> that it depends on a particular specification that can be fulfilled by
> multiple packages.  How the installer handles that is a different
> issue.  I suggest that the user be presented with the choices through
> a package search and be required to explicitly include one of them in
> the next install line.

My 2 cents' worth -- we should strive for zero-admin operation. If
there is a choice, maybe that can be added to the catalog format
somehow. In other words, "I rely on catalog X to choose for me a
conforming implementation of Y".

mob

unread,
Dec 16, 2009, 8:02:39 PM12/16/09
to CommonJS

On Dec 16, 4:57 pm, ihab.a...@gmail.com wrote:

I'd agree. This should be done through the catalog and browser tools
and outside of the package.json file.
A package should state what the "code" depends on. What modules must
be present.

One solution to this is to define meta packages. For example: an "ssl"
package could mean that some form of SSL is present (openssl, gnutls
etc). We want to keep the job for the consuming package simple. They
should depend on a package. That package can be a real package
providing code, or it can be a meta package representing a service or
platform.

--mob

Kris Kowal

unread,
Dec 16, 2009, 8:04:52 PM12/16/09
to comm...@googlegroups.com
On Wed, Dec 16, 2009 at 4:40 PM, mob <m...@embedthis.com> wrote:
> On Dec 16, 4:11 pm, Kris Kowal <cowbertvon...@gmail.com> wrote:
>> On Wed, Dec 16, 2009 at 3:40 PM, mob <m...@embedthis.com> wrote:
>> > I'm leaning toward the former. If someone says "package", "1.2.3".
>> > What they really mean is 1.2.3 or later.
>>
>> Actually, if they say "1.2", they mean greater or equal to "1.2.0" but
>> less than "2".  Major versions introduce changes that require code
>> changes.  Minor versions introduce changes that are strictly backward
>> compatible.  Patch versions are backward and forward compatible and
>> the most recent is always the best choice.  I think all the major
>> cases are covered.
>
> Agree that sometimes that is the case. The use case I was meaning if I
> have a package that needs feature X in package y which was introduced
> in version 1.3. ie. I depend on any version >= 1.3.

With semantic versioning, if a feature is introduced in version 1.3,
it is guaranteed to exist in any minor version or patch that is equal
to or greater than 1.3, but when the major version changes, API
compatibility is no longer guaranteed, thus the need for explicit
declaration of compatibility. Perhaps this is a weakness of SemVer;
perhaps there's a need to state the maximum major version that is
guaranteed to be compatible. That means that the minimum bound would
need to be either two digits or one digit and the maximum bound would
only need to be one digit.

{"name": "package-name", "min": "1.3", "max": "3"}

1.3 <= version < 3

If that's what you're recommending, I'm down. It would be unwise to
have a package state that it is guaranteed to work with future APIs
that have not yet been defined, but if we studiously update our
dependency lists, there won't be a problem. This enforces a good
practice.

Kris Kowal

Kris Kowal

unread,
Dec 16, 2009, 8:08:22 PM12/16/09
to comm...@googlegroups.com
On Wed, Dec 16, 2009 at 5:02 PM, mob <m...@embedthis.com> wrote:
> One solution to this is to define meta packages. For example: an "ssl"
> package could mean that some form of SSL is present (openssl, gnutls
> etc). We want to keep the job for the consuming package simple. They
> should depend on a package. That package can be a real package
> providing code, or it can be a meta package representing a service or
> platform.

Let's call these things for what they are; a meta-package is a
standard that multiple packages can conform to. I do not think we
should conflate the package name space with the standard name space;
it would make the system too "clever" and confusing like so many
package management system that have come before.

Maybe instead of "implements", we need a "standards" name space:

{"standards": {
"implements": []
"dependencies": []
}}

Kris Kowal

ihab...@gmail.com

unread,
Dec 16, 2009, 8:08:35 PM12/16/09
to comm...@googlegroups.com
And so, to clarify where I was going --

On Wed, Dec 16, 2009 at 4:53 PM, <ihab...@gmail.com> wrote:
> For case 2, the problem can be solved by signing *part* of the
> package, not the package itself. For example, the signatures in Java
> JAR files are for individual files, but not the whole thing. With that
> in mind, one could adopt some similar packaging convention that
> applies signatures to individual files -- including any ancillary
> JSON.

Requiring "package A signed by X" means "package A containing a
well-known signatures file which in turn contains a signature, by X,
for each file of package A that I end up using".

Ash Berlin

unread,
Dec 16, 2009, 8:09:05 PM12/16/09
to comm...@googlegroups.com


Hmmm yes and no. If the dep-group is something speced by CommonJS then wonderful. But the use case I'm thinking of is when you wish to wrap two or three (or more) underlying packages. And those packages are just random features: both work, and the consuming package knows how to feature test as needed (if they dont just expose the same API anyway). For example, MySuperWonderfulDocTool needs a PEG parser, so it says:

dependencies: [
{ group: [
[ "packrat" ],
[ "some-other-peg-parser" ]
]
}
]

And I'm with Ihab on the "zero-admin operation". The most annoying part about cpan (the binary) is when it asks you questions.


On 17 Dec 2009, at 01:02, mob wrote:

> One solution to this is to define meta packages. For example: an "ssl"
> package could mean that some form of SSL is present (openssl, gnutls
> etc). We want to keep the job for the consuming package simple. They
> should depend on a package. That package can be a real package
> providing code, or it can be a meta package representing a service or
> platform.

Hmmm, this doesn't feel like the right solution to me. Its been done for perl (there are various ::Any modules, such as Config::Any) but you still have the issue of when you are installing on a native system you need *one* of the modules, but if you have the second choice already installed you dont want to install the first choice.

-ashj

ihab...@gmail.com

unread,
Dec 16, 2009, 8:10:56 PM12/16/09
to comm...@googlegroups.com
On Wed, Dec 16, 2009 at 5:08 PM, Kris Kowal <cowber...@gmail.com> wrote:
> {"standards": {
>    "implements": []
>    "dependencies": []
> }}

This is asserted by a catalog, right? Not by any individual package?

Kris Kowal

unread,
Dec 16, 2009, 8:13:47 PM12/16/09
to comm...@googlegroups.com
On Wed, Dec 16, 2009 at 5:10 PM, <ihab...@gmail.com> wrote:
> On Wed, Dec 16, 2009 at 5:08 PM, Kris Kowal <cowber...@gmail.com> wrote:
>> {"standards": {
>>    "implements": []
>>    "dependencies": []
>> }}
>
> This is asserted by a catalog, right? Not by any individual package?

I was thinking of the package, bearing in mind that this is not a
feature that "using-packages" would use ever. When you "use" a
package, it is a direct reference.

If we want to avoid having any installer ever ask you which package
you want that conforms to a particular specification, we should avoid
both standard implementation declarations and standard dependency
declarations since they are only informative and introduce
non-determinism respectively.

Kris Kowal

mob

unread,
Dec 16, 2009, 8:14:16 PM12/16/09
to CommonJS

Kris, thanks for the use case. I understand the coercing packages to
do the right thing, but it has a downside.

What about this case:

I've got a package X and I use package Y 1.0 APIs. I create the
package and archive it never to return, but users continue to download
an useful (you see it was perfect the first time I created package X).
Package Y is under rapid development and goes 1.1, 1.2. .... 2.0
pretty quickly. All catalogs move quickly to host the new versions of
package Y. Users don't want to store every version of package Y, they
only really want the most recent package. Whereas they have the
perfect package X, no need to upgrade. Soon people stop hosting
package Y 1.0. And now no-one can use package X because it said:

"y", "1.0"

Whereas, package X should continue to be useful and should not die.
Package dependencies are not guarantees that something WILL work.
Rather, it is a statement of what is required to function.

So the choices are:

- Have an implicit upper version and have packages only run up to that
version. If the dependent package is upgraded, then the depending
package WILL eventually need to be upgraded

or

- Have no implicit upper version. Packages continue to work. They may
fail if the dependent package changes an API that is being used by the
depending package.

Both have holes. My preference is for the latter as it doesn't
forcibly break anything, but I understand your perspective. We need to
choose one.

---mob

mob

unread,
Dec 16, 2009, 8:17:10 PM12/16/09
to CommonJS
> Maybe instead of "implements", we need a "standards" name space:
>
> {"standards": {
>     "implements": []
>     "dependencies": []
>
> }}

What does the package manager do with dependencies?

I was assuming that implements was informative only and not enforced
by package managers.

ihab...@gmail.com

unread,
Dec 16, 2009, 8:24:14 PM12/16/09
to comm...@googlegroups.com
On Wed, Dec 16, 2009 at 5:13 PM, Kris Kowal <cowber...@gmail.com> wrote:
>> This is asserted by a catalog, right? Not by any individual package?
>
> I was thinking of the package, bearing in mind that this is not a
> feature that "using-packages" would use ever.  When you "use" a
> package, it is a direct reference.

Ok, in your use case where you might ask a user what to do, that is
fine. Package X can assert that "I implement Mail 2.0" and, as long as
you are auditing that yourself, you can presumably take precautions.

However, for the zero-admin case which I advocate, I wanted to clarify
the security scenario: I rely on catalog Y to tell me that package X
implements Mail 2.0, and therefore I delegate the burden of due
diligence to the author of catalog Y.

> If we want to avoid having any installer ever ask you which package
> you want that conforms to a particular specification, we should avoid
> both standard implementation declarations and standard dependency
> declarations since they are only informative and introduce
> non-determinism respectively.

Standard implementation declarations, if asserted by a catalog based
on knowledge (testing, ...) that the catalog author has done, can be
far stronger than informative.

Standard dependency declarations are non-deterministic only to the
extent that any dependency on a possibly varying object is
non-deterministic. We already have that if we rely on something via
its location and a signature, or range of versions, or ....

Kris Kowal

unread,
Dec 16, 2009, 8:26:21 PM12/16/09
to comm...@googlegroups.com
Hand show.

Zero: Packages can declare that they are dependent on version X.Y.Z
through W. All of the terms of X.Y.Z default to zero if they are not
provided and W defaults to (X + 1) if it is not provided.
Implications: if a package maintainer falls asleep AND multiple
versions are not supported by the catalog system AND if old versions
of packages are culled from the repository, it is possible that a
package will become unusable because it is dependent on a package that
no longer exists. This also means that, except for errors, semantic
versioning provides strong guarantees.

Infinity: Packages can declare that they are dependent on version
X.Y.Z through W. All of the terms of X.Y.Z default to zero if they
are not provided and W defaults to Infinity. Implications: packages
may go unmaintained indefinitely without causing their dependees to
ever become unusable. Incrementing the major version of your package
does not guarantee that package maintainers will be required to review
whether their packages still work and users will be forced to upgrade.

Kris Kowal

mob

unread,
Dec 16, 2009, 8:28:51 PM12/16/09
to CommonJS
Kris,

One question before hand show.

If we select the "Infinity model" you can implement the zero case by
specifying an upper version bound.

If we select the "zero model", how can you specify the infinity case?

--mob

ihab...@gmail.com

unread,
Dec 16, 2009, 8:29:12 PM12/16/09