Manually invoking junction.exe/mklink is not an option IMHO; I've hacked together some prototype of the above (as node script) and made it to node buster-dev-tools/update.js telling me "Finished!" :D But no further... Of course I'll make available what I have but prior to that I need some questions answered:
Oh, just learned that node's latest is now v0.6.12, npm (which is included with node for Win) also updated. Need to check whether that solves some of the problems.
Anyways, looking forward to your replies :)
___
My current setup is:
- WinXP Pro SP3 32bit, node 0.6.11 via .msi installer, npm is 1.1.1 (came with the .msi)
- buster 0.4.5 beta2 installed via npm with a few hacks by hand to get it going:
- alongside buster/: pre-built contextify (ATTENTION: needs some more, not to be used as-is)
- watch-tree installed via npm into buster/node_modules (used by buster-autotest, missing from deps) - yet this is *defunct on Win*
- simple browser tests (html-scaffold only) use the 0.4.5 buster-test.js and buster-test.css from busterjs.org
- mysysgit v1.7.6 and TortoiseGit v1.7.7.0-32bit
- buster-dev-tools cloned from gitorious, everything else is then pulled from github
- I found that recursively deleting what's in a node_modules folder before making the symlinks is a bad idea, at least on Win (functions.js does a rmdir /s /q on Win and rm -rf on Unixoids). Shouldn't just the link be removed, not the contents of what it points to? Are we expecting to ever have something else than symlinks in the various node_modules/ of our dev env?
- How are npm link and npm link <package-name> supposed to work in case there already exists something (link or real file/folder) where the symlink would be put? Overwrite, delete-then-overwrite or error? Couldn't find it npm's docs and well, cannot just try either...
- What about all the external dependencies (e.g. when, glob, ...), they're not handled automatically by update.js, right? Do I really need to npm-install them separately or what am I missing here?
- Over here, global npm installs happen in %APPDATA%\npm\node_modules and I also have a %NODE_PATH% pointing there. Can't remember if that was set automatically when I installed node or if it was me. Could anyone tell me? 8-}...
- should the initial clone of buster-dev-tools be from gitorious (as busterjs.org suggests, buster-dev-tools/README.txt points you there) or better from github? E.g. August's fix re. circular dependencies (thanks!) is on github but not yet on gitorious.
- I found that recursively deleting what's in a node_modules folder before making the symlinks is a bad idea, at least on Win (functions.js does a rmdir /s /q on Win and rm -rf on Unixoids). Shouldn't just the link be removed, not the contents of what it points to? Are we expecting to ever have something else than symlinks in the various node_modules/ of our dev env?
Yes. When you do a `npm link` (or `npm install`), all the dependencies listed in the project's package.json will be installed locally in node_modules. The dev tools make sure that any buster- dependencies are symlinked (as to only have one copy of each project at all times), and other dependencies are installed as normal. Recursively removing node_modules removes all dependencies. Whether or not it is necessary I'm not 100% sure. August can probabl chip in here.
I found that recursively deleting what's in a node_modules folder before making the symlinks is a bad idea, at least on Win (functions.js does a rmdir /s /q on Win and rm -rf on Unixoids). Shouldn't just the link be removed, not the contents of what it points to? Are we expecting to ever have something else than symlinks in the various node_modules/ of our dev env?
- How are npm link and npm link <package-name> supposed to work in case there already exists something (link or real file/folder) where the symlink would be put? Overwrite, delete-then-overwrite or error? Couldn't find it npm's docs and well, cannot just try either...
`npm link` skips already installed dependencies. So if you do `npm link` twice in a row, the second invocation will be quick as it doesn't install any dependencies.
It just occurred to me that we've overlooked the simplest solution available:
buster/
node_modules/
buster-assertions/
buster-capture-server/
...
Simple, one copy of each project, works everywhere. Non-buster dependencies can be installed locally in individual projects.
Christian
The installer should bundle node and npm. But I don't think the bundled node and npm commands should be exposed in PATH. They should be completely internal. So then the uninstaller doesn't need to ask the user if node should also be removed, as it's just our internal node "install" that gets wiped.
Not sure how the output of "npm root" relates to the use of NODE_PATH, but with the setup Chris suggested, we don't need to alter NODE_PATH anyway (just PATH).
Regarding your linking_for_Windows branch, it seems to mostly be about making "npm link" work on windows, but with the approach we're discussing now we aren't going to use npm link at all. Relying on a big nasty hairball graph of symlinks was kind of dirty anyway... I must say I really appreciate you contributing though, so it's a shame if we don't end up using it. :)
Contextify: Since it's an optionalDependency, it should work just fine on windows.
Then I think there's a difference between simply removing a line from package.json vs. making a module in fact NOT depend on another. We've been using the term "removing a dependency" a bit sloppy lately w.r.t. that, haven't we? Doing only the former while the module factually still does have the dependency - doesn't really solve anything, rather on the contrary!
I thought I also made jsdom an optionalDependency in those projects, but if it's still a devDependency, then that seems wrong. As far as I can tell, only buster-html-doc has the devDependency.
My theory is that the whole optionalDependency business only works if jsdom fails to install, not its dependencies. But I have not investigated.
So now I'm a bit puzzled what to do with those devDepencies on buster. In fact, I'd be glad if someone could precisely explain the meaning of devDepencies and optionalDependencies. From the npm-docs I could only conclude that you'd need devDeps to run the tests...
As far as I can tell, only buster-html-doc has the devDependency.
I thought I also made jsdom an optionalDependency in those projects, but if it's still a devDependency, then that seems wrong.
My understanding was that NPM would ignore errors from optionalDependencies, but alas that does not look to be the case.
No no, it really is like you both say. It just looked strange since you thought "it is optional" (which it factually is) but did not declare so in package.json (i.e. optionality ain't "official", like npm only knows what you're telling it, not what you're thinking).My understanding (it's undocumented) of optionalDependencies is that it tries to install it but ignores any errors. If a failing dependency of a dependency bubbles out of this behaviour, it sounds like a bug in npm to me.
I thought I also made jsdom an optionalDependency in those projects, but if it's still a devDependency, then that seems wrong.
Funny, didn't think that variant would occur :) So what you did is make the actual dependency in fact optional but just did not declare so in package.json - easy fix.
Well, actually, to run buster-html-doc's tests, you do need jsdom, so it kinda is a devDependency. But not if that breaks installation...
But skipping devDependencies for buster-dev-tools makes little sense..
Perhaps we should remove buster-html-doc from the core projects you get with buster-dev-tools.
Perhaps we should remove buster-html-doc from the core projects you get with buster-dev-tools.
Hmm, cannot really judge. But I'm thinking that having a proliferation of dev envs like "really real dev, all there is" (you), "somewhat crippled with hacks" (me), "easy" (whoever) - now multiply that with platforms, maybe even with node/npm versions - might turn out as not so good an idea.
W.r.t. buster-dev-tools it appears better to stick with the "Highlander way": there can only be one... and if you're outside Scotland (= on a setup we don't yet support) - then go contribute and make your particular spot part of the kingdom.
Managing differences across setups in production is hard enough, so I'd say let's try to keep the dev as common as possible and on as many platforms as possible. This should be only good for a) ease of communication and b) the confidence that we can have in how well a particular platform will be supported in production.
But buster-html-doc is a stand-alone extension. It is not released as part of buster, you have to install it separately. I think it's reasonable to leave it out if it means dev setup works on Windows.Think that convinces me, particularly the "not released as part of buster" part. Had absolutely no idea, so your initiative re architecture overview is great! Maybe you can move that up a bit on your todo list?
[...] (unless we discover bugs)Well, there won't be many then who would or even could... But, as said, you decide.