How to get a dev environment on Windows

239 views
Skip to first unread message

meisl...@googlemail.com

unread,
Mar 10, 2012, 3:44:10 PM3/10/12
to buster...@googlegroups.com
In Making Buster.JS easy to install on Mac OS X and Windows XP,Vista,7 it turned out that even only getting a usable development environment on Windows ain't trivial.
So I'd like to open a separate one for this; plz also check issue #127 on github. To summarize what we have so far:
  • node's fs.symLink(..) and fs.symLinkSync(..) aren't available on Win. It's pretty settled to use a replacement based on SysInternal's junction.exe for XP or on Vista and 7 alternatively mklink. There must be no circular dependencies for this to work (already fixed).
  • npm link is not implemented on Win. A replacement can be built based on the above.

Manually invoking junction.exe/mklink is not an option IMHO; I've hacked together some prototype of the above (as node script) and made it to node buster-dev-tools/update.js telling me "Finished!" :D But no further... Of course I'll make available what I have but prior to that I need some questions answered:

  • I found that recursively deleting what's in a node_modules folder before making the symlinks is a bad idea, at least on Win (functions.js does a rmdir /s /q on Win and rm -rf on Unixoids). Shouldn't just the link be removed, not the contents of what it points to? Are we expecting to ever have something else than symlinks in the various node_modules/ of our dev env?
  • How are npm link and npm link <package-name> supposed to work in case there already exists something (link or real file/folder) where the symlink would be put? Overwrite, delete-then-overwrite or error? Couldn't find it npm's docs and well, cannot just try either...
  • What about all the external dependencies (e.g. when, glob, ...), they're not handled automatically by update.js, right? Do I really need to npm-install them separately or what am I missing here?
  • Over here, global npm installs happen in %APPDATA%\npm\node_modules and I also have a %NODE_PATH% pointing there. Can't remember if that was set automatically when I installed node or if it was me. Could anyone tell me? 8-}...
  • should the initial clone of buster-dev-tools be from gitorious (as busterjs.org suggests, buster-dev-tools/README.txt points you there) or better from github? E.g. August's fix re. circular dependencies (thanks!) is on github but not yet on gitorious.

Oh, just learned that node's latest is now v0.6.12, npm (which is included with node for Win) also updated. Need to check whether that solves some of the problems.

Anyways, looking forward to your replies :)

___

My current setup is:

- WinXP Pro SP3 32bit, node 0.6.11 via .msi installer, npm is 1.1.1 (came with the .msi)
- buster 0.4.5 beta2 installed via npm with a few hacks by hand to get it going:
- alongside buster/: pre-built contextify (ATTENTION: needs some more, not to be used as-is)
- watch-tree installed via npm into buster/node_modules (used by buster-autotest, missing from deps) - yet this is *defunct on Win*
- simple browser tests (html-scaffold only) use the 0.4.5 buster-test.js and buster-test.css from busterjs.org
- mysysgit v1.7.6 and TortoiseGit v1.7.7.0-32bit
- buster-dev-tools cloned from gitorious, everything else is then pulled from github


Christian Johansen

unread,
Mar 10, 2012, 4:38:13 PM3/10/12
to meisl...@googlemail.com, buster...@googlegroups.com
Fantastic, love the initiative! See my responses below.
  • I found that recursively deleting what's in a node_modules folder before making the symlinks is a bad idea, at least on Win (functions.js does a rmdir /s /q on Win and rm -rf on Unixoids). Shouldn't just the link be removed, not the contents of what it points to? Are we expecting to ever have something else than symlinks in the various node_modules/ of our dev env?
Yes. When you do a `npm link` (or `npm install`), all the dependencies listed in the project's package.json will be installed locally in node_modules. The dev tools make sure that any buster- dependencies are symlinked (as to only have one copy of each project at all times), and other dependencies are installed as normal. Recursively removing node_modules removes all dependencies. Whether or not it is necessary I'm not 100% sure. August can probabl chip in here.
 
  • How are npm link and npm link <package-name> supposed to work in case there already exists something (link or real file/folder) where the symlink would be put? Overwrite, delete-then-overwrite or error? Couldn't find it npm's docs and well, cannot just try either...
`npm link` skips already installed dependencies. So if you do `npm link` twice in a row, the second invocation will be quick as it doesn't install any dependencies.
  • What about all the external dependencies (e.g. when, glob, ...), they're not handled automatically by update.js, right? Do I really need to npm-install them separately or what am I missing here?
package.json :) As I said, `npm link` installs all dependencies from package.json.
  • Over here, global npm installs happen in %APPDATA%\npm\node_modules and I also have a %NODE_PATH% pointing there. Can't remember if that was set automatically when I installed node or if it was me. Could anyone tell me? 8-}...
Not really sure. Whether or not the global repo is included in NODE_PATH seems to vary between platforms and installation mechanism. It's not included on my (Ubuntu) machine, but I built it from source... They used to recommend not doing that, now I don't know what's "the right thing".
  • should the initial clone of buster-dev-tools be from gitorious (as busterjs.org suggests, buster-dev-tools/README.txt points you there) or better from github? E.g. August's fix re. circular dependencies (thanks!) is on github but not yet on gitorious.
We try to mirror on both places, but we sometimes forget. GitHub is the primary hosting now, so go for that. But Gitorious really should be up to date too IMO.

Christian

meisl...@googlemail.com

unread,
Mar 10, 2012, 5:32:40 PM3/10/12
to buster...@googlegroups.com, meisl...@googlemail.com
:D, my pleasure!

  • I found that recursively deleting what's in a node_modules folder before making the symlinks is a bad idea, at least on Win (functions.js does a rmdir /s /q on Win and rm -rf on Unixoids). Shouldn't just the link be removed, not the contents of what it points to? Are we expecting to ever have something else than symlinks in the various node_modules/ of our dev env?
Yes. When you do a `npm link` (or `npm install`), all the dependencies listed in the project's package.json will be installed locally in node_modules. The dev tools make sure that any buster- dependencies are symlinked (as to only have one copy of each project at all times), and other dependencies are installed as normal. Recursively removing node_modules removes all dependencies. Whether or not it is necessary I'm not 100% sure. August can probabl chip in here.

Oh-ah - seems I underestimated quite a bit of what npm link does, though I'd rather deem it their fault - no word about recursively resolving/installing/linking sub-dependencies while kind of "staying aware that initially this was an npm link". Ok, that explains why it's just not implemented on Win, haha. Someone PLZ give me a thorough spec!

Anyways, I said "bad idea, at least on Win" but didn't explain: on a subsequent run (i.e. when there actually are already symlinks to buster- dependencies) update.js brutally erases everything it just pulled from the repos, eventually croaking when trying to symlink something that's not there anymore. Still wondering why/how this can be working on *nix...
However, I think I'm starting to get the point/intention. Will try to wrap my head around it, hoping for advice from August.

The rest of your answers are - just clear, thanks! (the NODE_PATH question isn't a show-stopper)

And Thank You for the quick reply :)
__
meisl

August Lilleaas

unread,
Mar 10, 2012, 8:48:33 PM3/10/12
to buster...@googlegroups.com
I found that recursively deleting what's in a node_modules folder before making the symlinks is a bad idea, at least on Win (functions.js does a rmdir /s /q on Win and rm -rf on Unixoids). Shouldn't just the link be removed, not the contents of what it points to? Are we expecting to ever have something else than symlinks in the various node_modules/ of our dev env?

If we screw up dependencies, the "npm link" may download an internal buster package from npm, so this is just a safeguard against that. For exaple, we bump buster-core to 0.6.6, but depend on 0.5 in a project. In that case, "npm link" will dowload 0.5 from npm, even if we already have a symlink in place for our local 0.6.6.

The real solution would of course be to have a utility to notify us when we screw up the dependencies.

August Lilleaas

unread,
Mar 10, 2012, 8:52:11 PM3/10/12
to buster...@googlegroups.com
In general, the current buster-dev-tools should work. We do a "npm install" on windows now, instead of "npm link". This will install all the dependencies to node_modules of the various projects, but it won't symlink the binaries and what not into the global npm path. That's OK for now, we just have to use "node code/buster/buster/bin/buster-test" etc to run the binaries. This is only for the development env of course, when installing buster normally with npm this is not an issue (I hope).

So, the only thing I need to fix is faye 0.8 so we don't depend on the node-waf redis module, and we should be good to go for a windows dev environment.


On Saturday, March 10, 2012 9:44:10 PM UTC+1, meisl...@googlemail.com wrote:

meisl...@googlemail.com

unread,
Mar 12, 2012, 12:18:52 PM3/12/12
to buster...@googlegroups.com
Yup, and I think I can come up with something for the linking as well.
Then npm install croaks in buster-syntax, buster-test and buster-html-doc because of jsdom, which depends on contextify. As you know, I've been able to somehow use a prebuilt-one so I'll try to do this again for dev env.

Found an answer to my own q re NODE_PATH (see initial post): node root or, respectively, node root -g do the job.

Regarding
  • How are npm link and npm link <package-name> supposed to work in case there already exists something (link or real file/folder) where the symlink would be put? Overwrite, delete-then-overwrite or error? Couldn't find it npm's docs and well, cannot just try either...
`npm link` skips already installed dependencies. So if you do `npm link` twice in a row, the second invocation will be quick as it doesn't install any dependencies.

I (now) understand that `npm link` is like `npm install` plus some more. So w.r.t. to the `npm install` part your answer is clear. But there's also the "some more" part, including (but not limited to, as I can see from August's post) the creation of a "globally-installed symbolic link from prefix/package-name to the current folder". Now, my question was: does it really do and say nothing, if there is already one (symlink, folder)? That is, how do I do a "re-link" then, if e.g. I want to have a different folder being pointed at? I'd be enthused if you could do that quick test for me :)

Thanks,
meisl

meisl...@googlemail.com

unread,
Mar 12, 2012, 12:21:23 PM3/12/12
to buster...@googlegroups.com
Sorry for breaking the citation, plz kindly ignore anything below "Thanks, meisl".

meisl...@googlemail.com

unread,
Mar 12, 2012, 12:51:33 PM3/12/12
to buster...@googlegroups.com
On the "really rm -r symlinks question:
IEEE says "The rm utility shall not traverse directories by following symbolic links into other parts of the hierarchy, but shall remove the links themselves."
So I guess we should indeed better treat junction points on Win like that...

August Lilleaas

unread,
Mar 12, 2012, 12:58:40 PM3/12/12
to meisl...@googlemail.com, buster...@googlegroups.com
I think I've come up with a sensible solution.

As we all know, we want to have installers for Windows and OS X.  The shortest route to that goal is probably to set up a folder structure with all buster module and all dependencies in one folder. So no more symlinks, and no more installing of npm packages in node_modules of the various projects. Then we set NODE_PATH to that directory, possibly by wrapping the "node" binary. We'll discuss how specifically to do that later.. But this will give us a globally available install, and then all the installer really needs to do is put the folder in a suitable location, and create an uninstaller item on Windows.

This setup of having everything in one folder, can also be reused for buster-dev-tools. The tradeoff is that individual packages will have to depend on the same version for various modules. So if buster-capture-server depends on when.js 0.4 and buster-test-cli on when.js 0.5, we're in trouble. But this should never happen (tm).

I'll start working on a rewrite of buster-dev-tools that uses this procedure.

August Lilleaas

unread,
Mar 12, 2012, 12:59:50 PM3/12/12
to meisl...@googlemail.com, buster...@googlegroups.com
By the way, I love your checking of IEEE specs in your research. Comforming to specs > hacking! :)

Christian Johansen

unread,
Mar 12, 2012, 1:26:18 PM3/12/12
to August Lilleaas, Buster. JS development, meisl...@googlemail.com

It just occurred to me that we've overlooked the simplest solution available:

buster/
  node_modules/
    buster-assertions/
    buster-capture-server/
...

Simple, one copy of each project, works everywhere. Non-buster dependencies can be installed locally in individual projects.

Christian

August Lilleaas

unread,
Mar 12, 2012, 1:31:39 PM3/12/12
to buster...@googlegroups.com, August Lilleaas, meisl...@googlemail.com
Installing the non-buster dependencies in the individual projects makes sense. I also like your suggestion better, so we don't have to set NODE_PATH. We just need to add buster/bin to PATH. Doing it!

meisl...@googlemail.com

unread,
Mar 12, 2012, 1:45:39 PM3/12/12
to buster...@googlegroups.com, August Lilleaas, meisl...@googlemail.com
Generally I think agree w.r.t. the installers, with both of you. @Christian: looks like what `npm install buster` produces, doesn't it ;)
However, I personally want to use node for other stuff as well, and I need (the functionality of) `npm link`. I can imagine buster users maybe not starting out like that but eventually getting there.
So I'll keep fiddling with that a bit more.

I'd like to have had it a bit more tidied-up and stuff, but now as August is ready-for-takeoff :) maybe you'd want to take a look at git://github.com/meisl/buster-dev-tools.git , branch linking_for_Windows.

re NODE_PATH: Actually I was proposing to not use it but rather take the output of node root / node root -g

Then, is it planned to bundle node (and thus npm) with the installer?
If so, care must be taken on uninstall, I think user must be asked if they go as well.

meisl...@googlemail.com

unread,
Mar 12, 2012, 1:48:16 PM3/12/12
to buster...@googlegroups.com, August Lilleaas, meisl...@googlemail.com
It's npm root / npm root -g , of course. Sorry for that.

meisl...@googlemail.com

unread,
Mar 12, 2012, 1:52:34 PM3/12/12
to buster...@googlegroups.com, meisl...@googlemail.com
Hehe, first I went through the painful experience of having my the stuff erased, including .git/ folders...

August Lilleaas

unread,
Mar 12, 2012, 1:54:25 PM3/12/12
to meisl...@googlemail.com, buster...@googlegroups.com
The installer should bundle node and npm. But I don't think the bundled node and npm commands should be exposed in PATH. They should be completely internal. So then the uninstaller doesn't need to ask the user if node should also be removed, as it's just our internal node "install" that gets wiped.

Not sure how the output of "npm root" relates to the use of NODE_PATH, but with the setup Chris suggested, we don't need to alter NODE_PATH anyway (just PATH).

Regarding your linking_for_Windows branch, it seems to mostly be about making "npm link" work on windows, but with the approach we're discussing now we aren't going to use npm link at all. Relying on a big nasty hairball graph of symlinks was kind of dirty anyway... I must say I really appreciate you contributing though, so it's a shame if we don't end up using it. :)

meisl...@googlemail.com

unread,
Mar 12, 2012, 2:36:34 PM3/12/12
to buster...@googlegroups.com, meisl...@googlemail.com
The installer should bundle node and npm. But I don't think the bundled node and npm commands should be exposed in PATH. They should be completely internal. So then the uninstaller doesn't need to ask the user if node should also be removed, as it's just our internal node "install" that gets wiped.
Perfect. And, BTW, that's "good windows style", too. You know, "what the heck's up with these so-called 'package-mangers' that those Unix people are so fond of?!"...

Not sure how the output of "npm root" relates to the use of NODE_PATH, but with the setup Chris suggested, we don't need to alter NODE_PATH anyway (just PATH).
It's where npm link puts its links and it's quite possible that NODE_PATH is my own invention, see initial post. Even better if you rely on neither one.
 
Regarding your linking_for_Windows branch, it seems to mostly be about making "npm link" work on windows, but with the approach we're discussing now we aren't going to use npm link at all. Relying on a big nasty hairball graph of symlinks was kind of dirty anyway... I must say I really appreciate you contributing though, so it's a shame if we don't end up using it. :)
Right & no problem at all. Your current dev layout (= flat) is good as it is IMHO - for dev. So in order to get going on Win (for me) it's necessary to have some sort of `npm link`. So I'll be using it, at least for buster-dev, and you're invited to change your mind, at any time.
I'm aware that it's rather a node/npm thing rather than a buster thing but well, I just started out with all this so for now...

Oh, and a big nasty hairball graph of dependencies you'll get, promised. Maybe this evening... :)

meisl...@googlemail.com

unread,
Mar 12, 2012, 4:26:15 PM3/12/12
to buster...@googlegroups.com, meisl...@googlemail.com
Maybe to clarify:
Sure the ultimate goal is to get this super-easy and shiny installer. Sure you should avoid unnecessary detours on the way - the keywords here being unnecessary (to be defined) and you (i.e. I personally may take some detours which you, of course, need not adopt).
I do understand that, especially for the TDD-inclined, it's tempting to equate "necessary" with "most simple you can think of". But hey, surely it's not on me to tell you what that presupposes. I mean tests, tests, tests and then - more tests :)

So, from my perspective, having a usable and working dev env on Windows (XP && Vista && Win7) first is an absolute "conditio sine qua non" before putting together the installer. I, for one, found that having a hodge-podge of ever-nested and ever-repeated node_modules/../node_modules/.. does NOT qualify as a usable dev env (to emphasize: speaking of dev env, NOT the installer!). I think I know what "not usable" means as I have put quite some effort into getting `npm install buster` to work somehow, then tried to do dev with it...
So, as said, I think your current dev env layout is fine, hence
  1. I'm trying to get that on disk (!), with all external deps.
  2. Once this is done I'll aim for plugging it in for basic buster usage ("basic" meaning whatever, to be defined).
  3. After that I want all tests that internal to buster to pass. At that point I'd call it not only usable but working as well.
  4. Only after that is achieved I'd bother about the bundling (be it Windows or whatever OS).
Again, I am new to mostly all of the stuff involved so I might well be missing a great deal of things. Furthermore, it's only my own humble view of things; you're the ones to decide which route to go.
I'm just trying to make you understand why I am following those "detours" (seemingly?) - hoping to get your help with one thing or another that I encounter on my way, even it's irrelevant for your route.
Being able to essentially help you would of course be even greater :)

Cheers,
meisl

August Lilleaas

unread,
Mar 12, 2012, 4:45:22 PM3/12/12
to meisl...@googlemail.com, buster...@googlegroups.com
I don't agree that a working dev env for windows is more important than an easy to use installer for windows. If I had to choose, I would without a shadow of a doubt choose the installer, and rather require that the relatively few Buster.JS contributors installed Linux or used cygwin.

But thankfully we can have both :) And as outlined in previous mails, we can reuse the development environment in the installer. Put diffrently: the installer installs the development environment.

August Lilleaas

unread,
Mar 12, 2012, 6:54:46 PM3/12/12
to meisl...@googlemail.com, buster...@googlegroups.com
I did the rewrite now, in the "rewrite" branch of buster-dev-tools.

Since we use npm install, and contextify is an optional dependency, contextify will no longer cause buster-dev-tools breakage on Windows - npm simply ignores failures for optional dependencies.

Faye 0.7 still fails though, but the faye 0.8 update is just around the corner.

meisl...@googlemail.com

unread,
Mar 13, 2012, 8:51:33 PM3/13/12
to buster...@googlegroups.com, meisl...@googlemail.com
Great, really good job!
Way more clean now and not using symlinks at all is also good. You know, e.g. properly removing them Windows junction points can be quite a pain in the a**...

Ok, re the promised "hairball": below it is included as a .png. As you can see there's still one dev-dependency on contextify, through buster-html-doc -> jsdom,
and there are some circular dev-deps left, too.
I'll provide it as .svg too, which zooms more nicely and it also provides tooltips.




meisl...@googlemail.com

unread,
Mar 13, 2012, 8:55:24 PM3/13/12
to buster...@googlegroups.com, meisl...@googlemail.com
btw, I managed to plug my prebuilt-contextify in my dev env; howto is to come. So the only real problem left is faye, as you said.

meisl...@googlemail.com

unread,
Mar 13, 2012, 9:01:58 PM3/13/12
to buster...@googlegroups.com, meisl...@googlemail.com
Plz find the .svg s attached, several combinations of dependencies kinds.
I wanted to provide them on github ... but screwed up somehow.
buster-dependencyGraphs.zip

August Lilleaas

unread,
Mar 14, 2012, 5:30:40 AM3/14/12
to buster...@googlegroups.com, meisl...@googlemail.com
Regarding dev dependencies, I put "buster" as a devDependency in some projects at least. With the new setup that is no longer necessary, since buster will be in NODE_PATH. So that's a bunch of circular deps removed right there.

Also, yay, graphic visualizations! Love 'em, never gotten into learning to make my own.

Contextify: Since it's an optionalDependency, it should work just fine on windows. npm will attempt to install all optionalDependencies, but bail and continue (instead of bail and exit like it does for dependencies and devDependencies) if the install fails. buster is aware of this, and gracefully skips syntax checking (which is what contextify is used for) if contextify can not be found. So even if having contextify is ideal, I'm pretty sure we can do buster dev on windows without it.

meisl...@googlemail.com

unread,
Mar 15, 2012, 3:18:16 PM3/15/12
to buster...@googlegroups.com, meisl...@googlemail.com
Contextify: Since it's an optionalDependency, it should work just fine on windows.

No, it doesn't. What I'm (desparately!) trying to tell you is that contextify is a regular dependency of jsdom, which itself is a devDependency of buster-html-doc, NOT optional. So...

Yet, I found a temporary workaround, with a prebuilt contextify which you can download here. Plz read the description there on how the procedure goes. No warranties for that though, I only know it's works on my machine with my particular version of node etc.

Then I think there's a difference between simply removing a line from package.json vs. making a module in fact NOT depend on another. We've been using the term "removing a dependency" a bit sloppy lately w.r.t. that, haven't we? Doing only the former while the module factually still does have the dependency - doesn't really solve anything, rather on the contrary!
So now I'm a bit puzzled what to do with those devDepencies on buster. In fact, I'd be glad if someone could precisely explain the meaning of devDepencies and optionalDependencies. From the npm-docs I could only conclude that you'd need devDeps to run the tests...



Christian Johansen

unread,
Mar 15, 2012, 3:32:53 PM3/15/12
to meisl...@googlemail.com, buster...@googlegroups.com
Then I think there's a difference between simply removing a line from package.json vs. making a module in fact NOT depend on another. We've been using the term "removing a dependency" a bit sloppy lately w.r.t. that, haven't we? Doing only the former while the module factually still does have the dependency - doesn't really solve anything, rather on the contrary!

I recently made sure jsdom is optional by requiring it in a try/catch and failing somewhat gracefully if you tried to force buster to use it. jsdom is used only for a few things:
  1. Better error messages for syntax errors. Completely optional, and the code only uses jsdom if it can
  2. HTML reporter on the comman line. This simply will not work without jsdom, no biggie
  3. The html-doc extension's tests

I thought I also made jsdom an optionalDependency in those projects, but if it's still a devDependency, then that seems wrong. As far as I can tell, only buster-html-doc has the devDependency.

My theory is that the whole optionalDependency business only works if jsdom fails to install, not its dependencies. But I have not investigated.

So now I'm a bit puzzled what to do with those devDepencies on buster. In fact, I'd be glad if someone could precisely explain the meaning of devDepencies and optionalDependencies. From the npm-docs I could only conclude that you'd need devDeps to run the tests...

You would. My understanding was that NPM would ignore errors from optionalDependencies, but alas that does not look to be the case.

Christian

August Lilleaas

unread,
Mar 15, 2012, 4:23:46 PM3/15/12
to Christian Johansen, meisl...@googlemail.com, buster...@googlegroups.com
My understanding (it's undocumented) of optionalDependencies is that it tries to install it but ignores any errors. If a failing dependency of a dependency bubbles out of this behaviour, it sounds like a bug in npm to me.

meisl...@googlemail.com

unread,
Mar 15, 2012, 4:52:59 PM3/15/12
to buster...@googlegroups.com, Christian Johansen, meisl...@googlemail.com
As far as I can tell, only buster-html-doc has the devDependency.
Right. And it is for sure; by-product of making the graphs.


I thought I also made jsdom an optionalDependency in those projects, but if it's still a devDependency, then that seems wrong.
Funny, didn't think that variant would occur :) So what you did is make the actual dependency in fact optional but just did not declare so in package.json - easy fix.


My understanding was that NPM would ignore errors from optionalDependencies, but alas that does not look to be the case.
 
My understanding (it's undocumented) of optionalDependencies is that it tries to install it but ignores any errors. If a failing dependency of a dependency bubbles out of this behaviour, it sounds like a bug in npm to me.
 No no, it really is like you both say. It just looked strange since you thought "it is optional" (which it factually is) but did not declare so in package.json (i.e. optionality ain't "official", like npm only knows what you're telling it, not what you're thinking).
Just verified this by deleting buster-html-doc's node_modules entirely, then moving jsdom into optionalDependencies in buster-html-doc's package.json and finally running buster-dev-tools deps, ending up with jsdom simply left out - without error.
Again, IMHO this once more calls for us being a bit more precise when speaking about dependencies.

Ok, all that does make the semantics of both, devDeps and optionalDeps more clear. What's missing is - speaking with August - documentation, i.e. like Christian has detailed re the use of jsdom in buster-html-doc. But well, I'm fine with it, not of top priority.

To conclude: Thanks for your answers! Looks to me like collaboration paying off :)

Christian Johansen

unread,
Mar 15, 2012, 4:56:32 PM3/15/12
to meisl...@googlemail.com, buster...@googlegroups.com
I thought I also made jsdom an optionalDependency in those projects, but if it's still a devDependency, then that seems wrong.
Funny, didn't think that variant would occur :) So what you did is make the actual dependency in fact optional but just did not declare so in package.json - easy fix.

Well, actually, to run buster-html-doc's tests, you do need jsdom, so it kinda is a devDependency. But not if that breaks installation...

Christian

August Lilleaas

unread,
Mar 15, 2012, 5:16:13 PM3/15/12
to Christian Johansen, meisl...@googlemail.com, buster...@googlegroups.com
I just learned about "npm install --production". But skipping devDependencies for buster-dev-tools makes little sense..

Perhaps we should remove buster-html-doc from the core projects you get with buster-dev-tools.

meisl...@googlemail.com

unread,
Mar 15, 2012, 5:52:20 PM3/15/12
to buster...@googlegroups.com, Christian Johansen, meisl...@googlemail.com
Well, actually, to run buster-html-doc's tests, you do need jsdom, so it kinda is a devDependency. But not if that breaks installation...
Ah, uhm.. getting complicated it is. In a dev env I'd expect to be able to run all the tests, s.t. I can always see what impact any of my local modifications have.

So with

But skipping devDependencies for buster-dev-tools makes little sense..
I definitly agree.


Perhaps we should remove buster-html-doc from the core projects you get with buster-dev-tools.
Hmm, cannot really judge. But I'm thinking that having a proliferation of dev envs like "really real dev, all there is" (you), "somewhat crippled with hacks" (me), "easy" (whoever) - now multiply that with platforms, maybe even with node/npm versions - might turn out as not so good an idea.
W.r.t. buster-dev-tools it appears better to stick with the "Highlander way": there can only be one... and if you're outside Scotland (= on a setup we don't yet support) - then go contribute and make your particular spot part of the kingdom.
Managing differences across setups in production is hard enough, so I'd say let's try to keep the dev as common as possible and on as many platforms as possible. This should be only good for a) ease of communication and b) the confidence that we can have in how well a particular platform will be supported in production.

Christian Johansen

unread,
Mar 15, 2012, 5:55:54 PM3/15/12
to meisl...@googlemail.com, buster...@googlegroups.com
Perhaps we should remove buster-html-doc from the core projects you get with buster-dev-tools.
Hmm, cannot really judge. But I'm thinking that having a proliferation of dev envs like "really real dev, all there is" (you), "somewhat crippled with hacks" (me), "easy" (whoever) - now multiply that with platforms, maybe even with node/npm versions - might turn out as not so good an idea.
W.r.t. buster-dev-tools it appears better to stick with the "Highlander way": there can only be one... and if you're outside Scotland (= on a setup we don't yet support) - then go contribute and make your particular spot part of the kingdom.
Managing differences across setups in production is hard enough, so I'd say let's try to keep the dev as common as possible and on as many platforms as possible. This should be only good for a) ease of communication and b) the confidence that we can have in how well a particular platform will be supported in production.

In general I agree. But buster-html-doc is a stand-alone extension. It is not released as part of buster, you have to install it separately. It is unlikely to need any more features than it already has. I think it's reasonable to leave it out if it means dev setup works on Windows. In reality, I'm probably the only who event needs buster-html-doc checked out (unless we discover bugs)...

Christian

meisl...@googlemail.com

unread,
Mar 16, 2012, 4:03:36 PM3/16/12
to buster...@googlegroups.com, meisl...@googlemail.com
But buster-html-doc is a stand-alone extension. It is not released as part of buster, you have to install it separately. I think it's reasonable to leave it out if it means dev setup works on Windows.
Think that convinces me, particularly the "not released as part of buster" part. Had absolutely no idea, so your initiative re architecture overview is great! Maybe you can move that up a bit on your todo list?

[...] (unless we discover bugs)
Well, there won't be many then who would or even could... But, as said, you decide.

Anyways, possibly documentation deserves more focus, be it only to state where things are in the working / where to go or what to do to start contributing. I mean it's not so much about a particular decision like leaving out buster-html-doc or not. General thoughts is the only thing I can provide, at least at the moment. No, what I'm after here is rather how to really leverage the open source, and then, once you're being hit by a whole lot of (seemingly) marvellous contributions, how to manage them. However, I'm not in the posession of a panacea.

So, to conclude: I'd be quite fine with having buster-html-doc simply be removed from dev tools. But doing so might require some thought on your part about what the possibly negative implications it'd have on the particular module, in a larger-scale collaboration context (assuming you want that to grow, don't you?).

August Lilleaas

unread,
Mar 16, 2012, 4:13:30 PM3/16/12
to meisl...@googlemail.com, buster...@googlegroups.com
Basically, the only problem now is that we haven't yet reached 1.0 yet :) We're committed to the docs, and won't release 1.0 until we have high quality documentation.

Christian Johansen

unread,
Mar 16, 2012, 7:58:06 PM3/16/12
to August Lilleaas, meisl...@googlemail.com, buster...@googlegroups.com
I've put this on the top of my list, and started work tonight. Hopefully publish something tomorrow. I think this is a pressing issue right now, just to have a high-level overview of all the packages.
--
MVH
Christian

meisl...@googlemail.com

unread,
Mar 16, 2012, 8:22:00 PM3/16/12
to buster...@googlegroups.com, August Lilleaas, meisl...@googlemail.com
Great, love to hear that :D
But there's no need for haste. "pressing" I'd agree with, but it's relative. In fact the "community" that's urgently requesting it is rather small, at least right now :) So just take the time it needs - and thanks!

meisl...@googlemail.com

unread,
Mar 17, 2012, 3:26:27 PM3/17/12
to buster...@googlegroups.com, August Lilleaas, meisl...@googlemail.com
Re the buster-html-doc / contextify problem on Win:
we'd leave out not only buster-html-doc but also buster-jstestdriver, since it regularly depends on buster-html-doc, right?

If I can find a way of automatically putting in the Win workaround I'll let you know. If not - well, then be it so.

Christian Johansen

unread,
Mar 17, 2012, 6:34:03 PM3/17/12
to meisl...@googlemail.com, buster...@googlegroups.com, August Lilleaas
We can leave buster-jstestdriver out too, yes. It's an optional extension that you have to install separately.

Christian
--
MVH
Christian
Reply all
Reply to author
Forward
0 new messages