Re: [node-dev] Better support for packages

22 views
Skip to first unread message

Isaac Schlueter

unread,
Feb 22, 2011, 1:17:19 PM2/22/11
to nodej...@googlegroups.com, nod...@googlegroups.com, np...@googlegroups.com
We decided that implementing support for these fields in node-core was
too much. Additionally, package.json files must be proper json now,
not just JavaScript.

A goal of npm development has always been to leverage what's in
node-core as much as possible, and I am working on removing features
that are unnecessary by virtue of being implemented in node-core.

The "directories" folder mapping and overlays are not essential
features. They're sugar, and having maintained and supported the
implementations of these features over some months now, I feel
qualified to say that they're more trouble than they're worth.

The module shimming that npm had to do to accomplish this was
annoying. Since shims were the only way to manage dependency paths,
it was a small-ish thing to also support dir mapping. However,
removing the module shims means:

1. require() will still make good on the module dependency guarantees
after the first tick.
2. the "main" field is required if you want to do require("foo")
3. If you want to dive into the contents of a module, you'll have to
specify the actual path to the submodule within the package. For
example, require("foo/lib/foo/bar") instead of require("foo/bar").

If you want those paths to be shorter, then put the modules in
shallower locations. This way is simpler.

I apologize for ever implementing this lib-dir mapping. It seemed
like a good idea at the time, especially since there was no other way
to get at sub-modules in a package. But it's much better this way.
Just specify the full path. It's not so bad, really.

The invalid-json parsing seemed like a good idea as well, and has
turned out to be much more of a headache than anticipated.

As for overlays, I don't believe that was ever necessary. If it is
actually required because your package.json *conflicts* with another
program, then implement overlays over there.

The change was announced on the nodejs@ and npm-@ lists quite some
time before it landed. However, usually no one updates their programs
until they break. (I am as guilty of this as anyone.) And, when your
program breaks because of someone else's code, it's frustrating and
causes some anxiety. I understand that, and I believe that it is
sadly unavoidable.

--i

On Tue, Feb 22, 2011 at 01:10, Irakli Gozalishvili <rfo...@gmail.com> wrote:
> Hi,
>
> It seems that support for both "overlay" and "direcotories" fields in
> package descriptors have been dropped in node >= 0.4.*. I would like to
> express my negative opinion on that, since it significantly complicates
> interoperability between nodejs and other js based projects I have been
> working on.
>
> 1. For a while I worked on a project called
> [teleport](http://jeditoolkit.com/teleport/#guide) that uses npm to present
> package registry for browser devs. Basically any package installed or linked
> is treated as a  separate project with only it's dependencies being
> available for browser based environment that can use either build-in module
> loader, requirejs or any other AMD compliant loader. Obviously many packages
> that want to be loaded on such a different platform need a glue layer to
> abstracting diffs and overlay's / dirs play a crucial role there which
> allowed packages with a modules abstracting diffs. There are alternative
> options but they are very unfriendly to browser env where keeping amount of
> modules to minimum is a good idea.
>
> 2. I'm also core team member of [Mozilla
> jetpack](https://jetpack.mozillalabs.com/) project. There we have been
> working on better integration with npm / node to allow simple package
> sharing across this projects, but again overlays play a very important role
> here.
>
> 3. Probably there are / will be other platforms that will try to reach some
> interoperability with node and better packages support will be an important.
>
> I really hope that this arguments are good enough to considering a better
> support for packages. Also if man power is a problem here I personally don't
> mind to work on this to get this inn.
>
> If there are any good reasons to don't support
> [packages](http://wiki.commonjs.org/wiki/Packages/1.1) at least a mentioned
> subset of the spec I really would like to know!
>
> Thanks!
>

gozala

unread,
Feb 22, 2011, 6:23:38 PM2/22/11
to nod...@googlegroups.com, nodej...@googlegroups.com, np...@googlegroups.com

We decided that implementing support for these fields in node-core was
too much.  Additionally, package.json files must be proper json now,
not just JavaScript.

I'm glad that only real json's are supported and it's a good move IMO, but I don't really agree that supporting those two fields is that hard or complicated it's about 10 lines diff for https://github.com/ry/node/blob/master/lib/module.js really.

A goal of npm development has always been to leverage what's in
node-core as much as possible, and I am working on removing features
that are unnecessary by virtue of being implemented in node-core.

The "directories" folder mapping and overlays are not essential
features.  They're sugar, and having maintained and supported the
implementations of these features over some months now, I feel
qualified to say that they're more trouble than they're worth.

Seriously, npm indeed had a lot of sugar that probably was not worth it, but this is just a trivial and essential for interoperability. Also I know interoperability was never a goal, but breaking already established one is just bad.

The module shimming that npm had to do to accomplish this was
annoying.  Since shims were the only way to manage dependency paths,
it was a small-ish thing to also support dir mapping.  However,
removing the module shims means:

1. require() will still make good on the module dependency guarantees
after the first tick.
2. the "main" field is required if you want to do require("foo")
3. If you want to dive into the contents of a module, you'll have to
specify the actual path to the submodule within the package.  For
example, require("foo/lib/foo/bar") instead of require("foo/bar").

If you want those paths to be shorter, then put the modules in
shallower locations.  This way is simpler.


Again I ensure you that support for both 'directories' and 'overlay' can be kept without compromising any of these.

I apologize for ever implementing this lib-dir mapping.  It seemed
like a good idea at the time, especially since there was no other way
to get at sub-modules in a package.  But it's much better this way.
Just specify the full path.  It's not so bad, really.

It may not be that bad, but there are at least few implementation out there (at least 3 that I've been involved with) that build on top http://wiki.commonjs.org/wiki/Packages/1.1 and packages were interchangeable between node, jetpack, chromeless, teleport. To get to the same level of interoperability they will have to change as well.

The invalid-json parsing seemed like a good idea as well, and has
turned out to be much more of a headache than anticipated.

I never liked it to be honest

As for overlays, I don't believe that was ever necessary.  If it is
actually required because your package.json *conflicts* with another
program, then implement overlays over there.

That's is indeed an option until there are more then one platforms ignoring overlays. Also while this can be worked around, think it's
easy trivial to keep.

The change was announced on the nodejs@ and npm-@ lists quite some
time before it landed.  However, usually no one updates their programs
until they break.  

My bad, but this mailing list is just too active to keep up with. Also to be fare I always update programs when new version lands, also that's how I found I noticed this problem. Also I expect that more people will notice brake in a future, since most of packages I've been using rely on 'directories'. 

(I am as guilty of this as anyone.)  And, when your
program breaks because of someone else's code, it's frustrating and
causes some anxiety.  I understand that, and I believe that it is
sadly unavoidable.

Braking code is fine, specially if there is a good reason to do so, but really, I don't think this is a case this time, since all the goals listed here be achieved without sacrificing compatibility with packages spec http://wiki.commonjs.org/wiki/Packages/1.1 that some implementations depend on. If I miss something or misunderstand, please let me know.
Reply all
Reply to author
Forward
0 new messages