Tell npm to have only one copy per package

[Leza Morais Lutonda]

Hi,

I want, when to install dependencies with npm (`npm install`) to have only one folder for node_modules in my project directory so that all my dependencies and dependencies of my dependencies (and dependencies of dependencies of dependencies ... of my dependencies) would be placed there.

Is it possible with npm?

Thanks.

--
Leza Morais Lutonda, Lemol-C
http://lemol.github.io

50 Aniversario de la Cujae. Inaugurada por Fidel el 2 de diciembre de 1964 http://cujae.edu.cu

[Aria Stewart]

On May 27, 02014, at 19:14, Leza Morais Lutonda <lez...@fecrd.cujae.edu.cu> wrote:

>
> Hi,
>
> I want, when to install dependencies with npm (`npm install`) to have only one folder for node_modules in my project directory so that all my dependencies and dependencies of my dependencies (and dependencies of dependencies of dependencies ... of my dependencies) would be placed there.
>
> Is it possible with npm?

No.

But npm dedup might get you close.

It’s only theoretically possible if your dependencies are transitively compatible — which is not something many people put effort into, since it’s not required except to move dependencies to the root.

What are you actually trying to accomplish?

Aria

[Francesco Mari]

There is something similar to what you want to do, and it is the dedupe command of npm (npm help dedupe). Completely flattening the node_modules folder could not be possible, because there could be transitive dependencies on the same module with different versions - which version to use in this case?. Anyway, dedupe is a best effort to optimize the dependencies of your project.

--
Job board: http://jobs.nodejs.org/
New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
To post to this group, send email to nod...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/53851C5A.30101%40fecrd.cujae.edu.cu.
For more options, visit https://groups.google.com/d/optout.

[Leza Morais Lutonda]

On 28/05/14 10:23, Aria Stewart wrote:
> No. But npm dedup might get you close. It’s only theoretically
> possible if your dependencies are transitively compatible — which is
> not something many people put effort into, since it’s not required
> except to move dependencies to the root. What are you actually trying
> to accomplish? Aria

Thanks for the answer.

I try `npm depupe`, but not very happy because the dependencies packages
have different versions of common dependencies.

I ever wished a npm structure like this:

node_modules/
| package1/
| | version1/
| | version2/
| | ....
| | versionN
| package2/
| | version1/
| | version2/
| | ....
| | versionN
....

And maybe node resolve the path for `require` on starting the
application reading de package.json.

Anybody else wished this?

Thanks.

[Leza Morais Lutonda]

On 28/05/14 10:45, Francesco Mari wrote:

There is something similar to what you want to do, and it is the dedupe command of npm (npm help dedupe). Completely flattening the node_modules folder could not be possible, because there could be transitive dependencies on the same module with different versions - which version to use in this case?. Anyway, dedupe is a best effort to optimize the dependencies of your project.

OK, I understand, thanks.

--
Leza Morais Lutonda, Lemol-C
http://lemol.github.io

[Ryan Schmidt]

On May 28, 2014, at 5:38 PM, Leza Morais Lutonda wrote:

> On 28/05/14 10:23, Aria Stewart wrote:

>> No. But npm dedup might get you close. It's only theoretically possible if your dependencies are transitively compatible -- which is not something many people put effort into, since it's not required except to move dependencies to the root. What are you actually trying to accomplish? Aria

>
> Thanks for the answer.
>
> I try `npm depupe`, but not very happy because the dependencies packages have different versions of common dependencies.
>
> I ever wished a npm structure like this:
>
> node_modules/
> | package1/
> | | version1/
> | | version2/
> | | ....
> | | versionN
> | package2/
> | | version1/
> | | version2/
> | | ....
> | | versionN
> ....

*Why* do you wish this?

npm is specifically designed to work the way it currently works, because doing so solves some problems. What problem are you trying to solve by circumventing this?

[Francesco Mari]

The dependency resolution mechanism in Node is well thought. npm works this way because of how Node resolves the dependencies, and the folder structure you propose is not compatible with the dependency resolution strategy.

Il giovedì 29 maggio 2014, Leza Morais Lutonda <lez...@fecrd.cujae.edu.cu> ha scritto:

On 28/05/14 10:23, Aria Stewart wrote:

No. But npm dedup might get you close. It's only theoretically possible if your dependencies are transitively compatible -- which is not something many people put effort into, since it's not required except to move dependencies to the root. What are you actually trying to accomplish? Aria

Thanks for the answer.

I try `npm depupe`, but not very happy because the dependencies packages have different versions of common dependencies.

I ever wished a npm structure like this:

node_modules/
   | package1/
   |   | version1/
   |   | version2/
   |   | ....
   |   | versionN
   | package2/
   |   | version1/
   |   | version2/
   |   | ....
   |   | versionN
   ....

And maybe node resolve the path for `require` on starting the application reading de package.json.

Anybody else wished this?

Thanks.



50 Aniversario de la Cujae. Inaugurada por Fidel el 2 de diciembre de 1964  http://cujae.edu.cu

--
Job board: http://jobs.nodejs.org/
New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
--- You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
To post to this group, send email to nod...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/5386657B.3000903%40fecrd.cujae.edu.cu.

[Aria Stewart]

On May 28, 02014, at 18:38, Leza Morais Lutonda <lez...@fecrd.cujae.edu.cu> wrote:

> On 28/05/14 10:23, Aria Stewart wrote:

>> No. But npm dedup might get you close. It's only theoretically possible if your dependencies are transitively compatible -- which is not something many people put effort into, since it's not required except to move dependencies to the root. What are you actually trying to accomplish? Aria

>
> Thanks for the answer.
>
> I try `npm depupe`, but not very happy because the dependencies packages have different versions of common dependencies.
>
> I ever wished a npm structure like this:
>
> node_modules/
> | package1/
> | | version1/
> | | version2/
> | | ....
> | | versionN
> | package2/
> | | version1/
> | | version2/
> | | ....
> | | versionN
> ....
>
> And maybe node resolve the path for `require` on starting the application reading de package.json.
>
> Anybody else wished this?

Sometimes, but what does this get you?

What are the trade-offs?

[Leza Morais Lutonda]

One problem with the actual structure is, for example, this [1] stack
overflow question (not just the copy-paste problem). When I use grunt,
doing `npm install` to install grunt's lib it take a long installing
common package.

Now I know that `node dedupe` can help a bit, but maybe a option for
`npm install` (eg. npm install -dedupe) would be useful too.


[1]
http://stackoverflow.com/questions/13318364/how-to-deploy-node-js-application-with-deep-node-modules-structure-on-windows

[Patrick Mueller]

One problem is that resolving a module at runtime (eg, `require()`) will require matching versions with what's on the file system vs what's in package.json.  That happens at `npm install` time today, not at require().  Unless you're talking about adding more metadata.  NOT MORE METADATA!!!

It would be great to see flattened trees like this, as a report - in fact I've seen tools do this today. Would like to see more!

--
Patrick Mueller
http://muellerware.org

[Aria Stewart]

On May 29, 02014, at 14:07, Leza Morais Lutonda <lez...@fecrd.cujae.edu.cu> wrote:
>
> One problem with the actual structure is, for example, this [1] stack overflow question (not just the copy-paste problem). When I use grunt, doing `npm install` to install grunt's lib it take a long installing common package.
>
> Now I know that `node dedupe` can help a bit, but maybe a option for `npm install` (eg. npm install -dedupe) would be useful too.
>
>
> [1] http://stackoverflow.com/questions/13318364/how-to-deploy-node-js-application-with-deep-node-modules-structure-on-windows

Ah, you’re running into Windows path limits? That’s a whole separate issue and kind of tricky to solve generally without reworking everything, things that are very much fixed at this point in node.js’s lifecycle.

Is that actually what you’re running into?

[CoolAJ86]

In my wildest dreams there is a universe in which a future version of npm versions packages in a global node_modules folder like this:

node_modules/thi...@1.6.7-rc1+aef123/package.json

But with the way many authors currently treat modules, there's still a problem there:
Many modules give you singletons by default and some have no way of creating instances, which means that the only true way to create an instance is to have a separate copy of the code.

[Forrest Norvell]

On Mon, Jun 2, 2014 at 10:48 AM, CoolAJ86 <cool...@gmail.com> wrote:

In my wildest dreams there is a universe in which a future version of npm versions packages in a global node_modules folder like this:

node_modules/thi...@1.6.7-rc1+aef123/package.json

But with the way many authors currently treat modules, there's still a problem there:
Many modules give you singletons by default and some have no way of creating instances, which means that the only true way to create an instance is to have a separate copy of the code.

This isn’t really npm’s call to make. The only way to make require("thingy") map to node_modules/thi...@1.6.7-rc1+aef123 would be to make fairly invasive changes to the Node module loader, which has to be one of the most most locked-down chunks of Node’s API. I definitely want to see npm dedupe work properly, but a flat module structure seems extremely unlikely in Node as long as it’s still called Node.

F

​

[greelgorke]

Am Montag, 2. Juni 2014 19:48:51 UTC+2 schrieb CoolAJ86:

In my wildest dreams there is a universe in which a future version of npm versions packages in a global node_modules folder like this:

this would be a nightmare

node_modules/thi...@1.6.7-rc1+aef123/package.json

But with the way many authors currently treat modules, there's still a problem there:
Many modules give you singletons by default and some have no way of creating instances, which means that the only true way to create an instance is to have a separate copy of the code.

which is ok. many modules don't have state (logic should be singleton) or encapsulate resources which you would wan't to be present once only in your app anyway, like db connections. However, if there is a module, that shouldn't be a singleton, then open an issue and make a pr :)

[Floby]

Also, it's not a future version that works like that, it's how it used (very briefly) to work.

This behaviour was removed very promptly as it was a very bad idea.

Local modules prevent version conflicts. Global modules save disk space

disk space is cheap, engineer time to solve conflicts is not.

[AJ ONeal]

Am Montag, 2. Juni 2014 19:48:51 UTC+2 schrieb CoolAJ86:

In my wildest dreams there is a universe in which a future version of npm versions packages in a global node_modules folder like this:

this would be a nightmare

False. It's a nightmare when it's done the old ruby way where versioning isn't considered.

 

node_modules/thi...@1.6.7-rc1+aef123/package.json

npm already does a boatload of dependency resolution, so duplicate packages bloat memory and disk space (which means load times - normally not that important, but sometimes is). Since each module would have a specific version it would essentially being doing the same work it already does, but not all over the place.

When I haven't rf -rf'd old projects for a while, my node_modules directories grow to several GBs.

The only nightmarish thing would be the singleton problem. Singletons are bad because strange strang things start happening when you run multiple apps in a single process (vhost-ing, for example)

That said having an npm rmrf-the-test-and-examples-and-hundreds-of-useless-megabytes-that-i-dont-use option might solve my gripe decently well.

But with the way many authors currently treat modules, there's still a problem there:
Many modules give you singletons by default and some have no way of creating instances, which means that the only true way to create an instance is to have a separate copy of the code.

which is ok. many modules don't have state (logic should be singleton) or encapsulate resources which you would wan't to be present once only in your app anyway, like db connections. However, if there is a module, that shouldn't be a singleton, then open an issue and make a pr :)

Word.

Anyway, overloading node's require isn't that difficult if I understand correctly. I've considered writing a wrapper around npm that does the global package management, but alas, it's on the back burner of good ideas that I'll probably never have time for.