ALERT: npm support for package.json "lib" directory and "modules" hash going away
Isaac Schlueter
If you currently do stuff like this:
{ "directories" : { "lib" : "./lib" }}
then that's fine. You can keep doing that to indicate *for strictly
informational purposes* that the bulk of your code is in that folder.
That's a Good Thing.
However, if you then do stuff like:
require("connect/middleware/gzip")
to pick up connect/lib/middleware/gzip.js, then that's going to fail
soon. And that's not so great. Instead, this will be The Way moving
forward:
require("connect/lib/middleware/gzip") (Note the "lib/" folder added
into the path explicitly.)
or just:
require("connect").getMiddlewareOrSomething("gzip")
IE, the "main" module will be exposed as it currently is. Not going away.
The "lib" folder will not be mounted onto the package name.
The package root *will* be mounted onto the package name. (Not yet
implemented, but will be in the next release.)
The "modules" hash is going away. Don't use it, don't rely on it.
(Using the word "mount" very loosely here. No actual mounting is done.)
The end result will be something much simpler and better. You'll like
it when it's done.
--i
Nickolay Platonov
Nickolay
Pau
Charlie Robbins
I´ve seen lots of libraries not having a index.js and its a PITA.Everyone is developing having npm in mind, but sometimes you need to vendorize (or use ndistro) and relying just on '/lib' sucks.
--
You received this message because you are subscribed to the Google Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com.
To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/nodejs?hl=en.
Isaac Schlueter
<charlie...@gmail.com> wrote:
> Is support for "main" going to stick around? That's my target of choice for
Yes. Sorry, I should have mentioned that. "main" will continue to be
supported. If you don't have a "main" module, and you don't have an
"index.js" in the root of your package, then require("package-name")
won't do anything.
>> I´ve seen lots of libraries not having a index.js and its a PITA.
>> Everyone is developing having npm in mind, but sometimes you need to
>> vendorize (or use ndistro) and relying just on '/lib' sucks.
All the more reason to not rely on the "modules" hash or
directories.lib. Specify a "main" module, and/or have an "index.js"
in the root of your package.
On Sat, Jan 29, 2011 at 01:01, Nickolay Platonov <nick...@gmail.com> wrote:
> What are the rationales behind this decision?
Great question.
There are a few factors that have led to this decision, mostly
involving a lot of discussion between Ryan and me about how node's
module-loading system and npm's module-creating system can be more
harmonious.
The plan at this point (still experimental, drawing-board-ish, 0.5
stuff) is that you'll be able to drop a package as-is directly into
the ./node_modules folder, and then require("foo") will Just Work,
even if npm is never around.
If you do `npm bundle install socket.io`, then npm will go get
socket.io, do the right thing as far as dependencies and such, and
then require("socket.io") will work in your program, regardless of how
your program ends up being loaded later.
The other side is the goal of simplifying npm and making it work at
all ever on windows without cygwin, where symbolic links are lacking
and/or weird.
Once upon a time, a design goal of npm was to work just like people
were manually installing node programs, but without having to manually
do everything. In fact, a package definition was once *only* a name,
version, and main module.
Somewhere in the last year, that got a bit lost in the shuffle of
things, and we find ourselves in place where properly installing
things *without* npm is far more tedious and annoying that it ought to
be. (In fact, often the package authors themselves don't even know
how to do it.) I want "wget" and "git submodule init --recursive" to
be valid and alternative ways to install dependencies, and I want it
to keep us out of dependency hell forever. npm has been a great
exploration, but it's time to rein in the complexity and learn from
the experience.
As one of the people most responsible for getting us to this mess, I
feel somewhat responsible to help get us out of it. I also don't want
to make everything suddenly not work. We're no longer just a dozen or
so hackers accustomed to their handful of programs breaking every
week. Node is a Real Thing, with Real People using it, so that means
that it's important to chart our course a bit more carefully. That
being said, putting it off unnecessarily will only increase the cost.
--i
James Burke
> Once upon a time, a design goal of npm was to work just like people
> were manually installing node programs, but without having to manually
> do everything. In fact, a package definition was once *only* a name,
> version, and main module.
found the packages config with a main and a lib to be too much
configuration and not enough convention. I think I'm at the point
where I want to actively encourage packages to just be a directory
name, and if the developer wants to use a package's main module they
ask for it specifically: require('packageName/index') or
require('packageName/main').
[aside: a strong convention should be chosen-- to me packageName/main-
>packageName/main.js makes more sense vs packageName/index-
>packageName/index.js since the main package module is not an index/
listing of files in that package, but that is a minor bikeshed. I do
not really care, I just want a strong convention]
In that model there is no configuration, just the pathing convention.
This is particularly useful in the browser where it is even more
onerous to keep track of how "packageName" maps to a file path (or
require a package manager to massage the contents).
So anyway, a +1 for keeping it simple, and even taking it further by
deprecating a "main" config. Since there is a community to support,
allow "main" to continue to work, but start encouraging using direct,
specific module names that map to paths that work in an unmodified
source pull, and without having tools to parse a package.json to find
an entry point.
It might be useful to keep the "main" in the package.json,
particularly if a strong "packageName/something" convention cannot be
reached, but it should just be used as a human convenience, so the
developer knows how to construct the require call, vs. informing some
sort of auto-configuration. npm or other package managers could inform
the developer of how the package should be called on package
retrieval, but no special config or directory or file massaging would
be needed.
Ideally though, a uniform convention starts to emerge where the main
in package.json no longer becomes necessary.
James
khs4473
advantage to not supporting it? Or are we saying that it's not a good/
valid way to structure a package?
khs
khs4473
loader is package-aware.
khs
On Jan 29, 2:16 pm, Isaac Schlueter <i...@izs.me> wrote:
> On Sat, Jan 29, 2011 at 10:44, Charlie Robbins
>
> > Is support for "main" going to stick around? That's my target of choice for
>
> Yes. Sorry, I should have mentioned that. "main" will continue to be
> supported. If you don't have a "main" module, and you don't have an
> "index.js" in the root of your package, then require("package-name")
> won't do anything.
>
> >> I´ve seen lots of libraries not having a index.js and its a PITA.
> >> Everyone is developing having npm in mind, but sometimes you need to
> >> vendorize (or use ndistro) and relying just on '/lib' sucks.
>
> All the more reason to not rely on the "modules" hash or
> directories.lib. Specify a "main" module, and/or have an "index.js"
> in the root of your package.
>
Isaac Schlueter
> Why not mount the "lib" directory directly? Is there a technical
> advantage to not supporting it? Or are we saying that it's not a good/
> valid way to structure a package?
>
> loader is package-aware.
The goal is to have a module loader that is package aware. Towards
that end, we need to make node packages a bit more reasonable.
The problem comes in when you have a setup like this:
require("./some/path/foo/bar/baz")
And then, at ./some/path/foo/package.json, you've got this:
{ "name" : "foo", "version" : "1.2.3", "directories":{"lib":"./lib/blerg"}}
And, at ./some/path/foo/lib/blerg/bar/baz.js, is the module you want to load.
That means that you have to inspect every portion of the require()
path argument to see if it's got a package.json, and then if it's got
a lib directory, and then inject the lib directory into the path.
Even if we went with convention over configuration, and say that the
lib directory is always "./lib", then you have to check every portion
of the path for a package.json and then inject ./lib into the path.
No. That is not so good.
If you want to dive into a package dir with require(), that's fine,
but you just have to specify the full path to the submodule. If you
want foo/lib/bar/baz.js, then do require("foo/lib/bar/baz.js").
Adding complexity and unnecessary fs overhead to the module loader in
order to do something with 4 fewer characters is not the right
approach.
The "modules" hash is an implementation detail of how npm mounts the
"lib" directory, and I don't think anyone actually uses it manually.
(Please correct me if this is wrong!) But that causes the same
problems.
Basically, if you have require("some/long/nested/path"), then we don't
want to have to check for a package.json anywhere except at
"some/long/nested/path/package.json".
On Sat, Jan 29, 2011 at 20:57, James Burke <jrb...@gmail.com> wrote:
> While trying to support packages in browsers via RequireJS, I have
> found the packages config with a main and a lib to be too much
> configuration and not enough convention. I think I'm at the point
> where I want to actively encourage packages to just be a directory
> name, and if the developer wants to use a package's main module they
> ask for it specifically: require('packageName/index') or
> require('packageName/main').
I think it's perfectly appropriate for RequireJS to have different
functionality than Node's module loader. You could quite easily say
that RequireJS is *not* "package aware", and make users specify the
file they want to load explicitly.
Moving forward, Node is not going to design its module loader based
(nor is npm going to build its packages) on the needs and capabilities
of other systems. That's how we got into this mess.
But, don't you run in environments with ample capacity to parse a json string?
> So anyway, a +1 for keeping it simple, and even taking it further by
> deprecating a "main" config.
That is not planned. "main" is very handy, easy to implement in
node's module loader, and almost every package uses it already.
> Ideally though, a uniform convention starts to emerge where the main
> in package.json no longer becomes necessary.
If such a convention emerges, we'll happily use it. I don't think
it's very likely.
--i
Isaac Schlueter
https://github.com/isaacs/npm/blob/master/doc/future-ideas/folders.md
Making this work requires moving away from a
require.paths/NODE_MODULES system path approach, and towards a more
deterministic system that is easier to reason about.
--i
khs4473
> want to have to check for a package.json anywhere except at
> "some/long/nested/path/package.json".
khs
On Jan 30, 1:23 pm, Isaac Schlueter <i...@izs.me> wrote:
James Burke
> Moving forward, Node is not going to design its module loader based
> (nor is npm going to build its packages) on the needs and capabilities
> of other systems. That's how we got into this mess.
>
> But, don't you run in environments with ample capacity to parse a json string?
than Node. It is best if there is only one path to find a module file.
Sending a package config for every package used in a web app is not
ideal, particularly since that configuration should be before any
actual module fetching. So, very strong conventions that do not
require configuration are best. Achieving a design with those
constraints may benefit packages running on Node too.
Or not. Just throwing out ideas since I have had to deal with similar
issues. It would be great if more modules/packages/conventions could
be shared between browsers and Node, but if it does not work out, so
be it.
James