Advice please .... risks with "no standard" manipulation of packages in node_modules

[Tom Fennelly]

Hi.

I'm just wondering what might be the risks associated with manipulating the contents of the node_modules folder outside the normal "npm install" etc commands. IOW ... are there reasons why the a raw "drop in" of packages into the node_modules is a bad idea Vs doing it through the normal "npm install" mechanisms?

I have a colleague proposing this basic idea as a solution to a problem we have (longer story that probably doesn't need airing here). For me, it just feels like the wrong thing to do (scares me), but my "gut feeling" is not a good enough reason for me to say that I think we should not consider doing it.

So, I'd love if someone could enlighten me to the potential dangers, or if there's no real danger and nothing to worry about.

Thanks guys.

Regards,

Tom.

[Zlatko]

On Wednesday, November 30, 2016 at 6:46:33 PM UTC+1, Tom Fennelly wrote:

Hi.

I'm just wondering what might be the risks associated with manipulating the contents of the node_modules folder outside the normal "npm install" etc commands. IOW ... are there reasons why the a raw "drop in" of packages into the node_modules is a bad idea Vs doing it through the normal "npm install" mechanisms?

TL;DR: it's not a problem if you replace npm-cli tool with another capable tool. 

 

It's basically about convenience. For example, if you've tried yarn (the Facebook's new thingy and npm-cli replacement), it messes up with your node_modules. If you try some npm modules themselves, they can do it too (for example: fix-dropbox-node_modules-symlinks).

Having your module version numbers in package.json, now that's the real deal. What does npm-cli offer:

- keeps track of versions and updates stuff for you,

- takes note of what's a prod dependency, what's a dev dependency, peer dep etc,

- allows for cross-platform stuff.

There's more stuff, but these are, I think, the most important points.

The way I see it, only the last thing might bite you hard, but let's go over those points.

1. Versions and updates

Like I've said, yarn does package version management. Bower does it too. Many tools do, even Sublime or WebStorm can do it for you. The thing is, pick a good thing that you can repeat in production environment and on every dev's workstation, and you're probably ok. If your tool can also update package.json, you're golden - even people who don't use your tooling can probably repeat your steps (or run your one-liner install/setup/config script).

As for updates - if you're "manually" (or by other tool than npm-cli) updating a package version, but still plan to ship without node_modules, you might lose the correct package version. If your other tool is automated, you can simply reuse package.json or something like that, but if you're getting a new module by actually downloading it and saving into node_modules/, you might forget to update your package management.

So, unless you ship node_modules/ directory with all your code, this might be a problem. Again, tools like yarn can update on their own, so no issue.

2. Dev, prod, test, peer etc dependencies

Ah, basically the same. If your replacement tool supports this, no risks. If it doesn't, the risk is again that prod doesn't get a critical module or that devs work with different versions of modules or simply that your git repository is huge and checkout and deployment takes forever.

3. Binary modules

Say you're installing redis and hiredis. hiredis is a binary package. If you drop in your own version manually, and you're actually shipping the node_modules/ folder, your colleague might be using a mac or windows (guess what I'm using :P), so it won't work for them.

But again, tool that takes care of it, takes the pain away.

[Ben Noordhuis]

In the end it's just files on disk. If your node_modules has the
expected layout, node doesn't care if it was you or npm that put the
files there.

[Muhammad Wasim]

You might not be able to install it later on different environments. If you have plans to use containers then you should not drop anything in node_modules. It would be better if you just publish it as an npm package and use it in your code by installing it through proper channel.

Regards,

Muhammad Wasim

--
Job board: http://jobs.nodejs.org/
New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+unsubscribe@googlegroups.com.
To post to this group, send email to nod...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/028d28c0-7566-49fb-ba5b-a57cf64178b4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Regards,

Muhammad Wasim | Lead Software Engineer | Zigron Inc.
E-mail: muhamma...@zigron.com
Website: www.zigron.com

[Tom Fennelly]

Thanks guys. As with many things, the devil is in the detail and the detail of this (in my case) is yet to come out. Thanks a lot for your insights.