buster-dev-tools/run-tests quite broken

[m.k...@sprax.eu]

Hi,

had set up a fresh dev-env on WinXP, node 0.8.16 (npm 1.1.16), then buster-dev-tools/run-tests.

First off, it needs buster in project-list.js. This seems to have been busted out of there sometime in the past. Any reason or just accident?

Then, with buster in project-list.js, it's still a complete fail: not with test failures but rather nearly everything exploding with an uncaught exception like this:

Uncaught exception in MINGW32:/E/busterJS/devEnv/buster-dev-tools v0.8.16, win32 ia32:
 
  TypeError: test:failure listener threw error: Cannot call method 'split' of undefined
    TypeError: test:failure listener threw error: Cannot call method 'split' of undefined
          at filterStack (e:\busterJS\devEnv\buster-test\lib\reporters\brief.js:73:39)
          at formatStack (e:\busterJS\devEnv\buster-test\lib\reporters\brief.js:134:17)
          at reportException (e:\busterJS\devEnv\buster-test\lib\reporters\brief.js:197:26)
          at Object.module.exports.test:failure (e:\busterJS\devEnv\buster-test\lib\reporters\brief.js:293:9)
          at notifyListener (e:\busterJS\devEnv\bane\lib\bane.js:54:35)
          at Object.object.emit (e:\busterJS\devEnv\bane\lib\bane.js:150:17)
          at emit (e:\busterJS\devEnv\buster-test\lib\test-runner.js:62:16)
          at testResult (e:\busterJS\devEnv\buster-test\lib\test-runner.js:86:9)
          at Object.<anonymous> (e:\busterJS\devEnv\buster-test\lib\test-runner.js:601:21)
          at bound (e:\busterJS\devEnv\buster-test\node_modules\lodash\lodash.js:2082:19)

Like a hundred of these, all the same.

At first I thought it might be due to the changed API of assert.exception (docs should be adjusted!).

But: shouldn't the old style ( assert.exception( someFunc() { ... }, "ErrorName", "message-for-test-fail") ) still work with the new API?

Then, changing assertions to e.g. assert.exception( someFunc() { ... }, { name: "TypeError", message: "exception-message-copied-from-code"}) doesn't help much.

Well, it's now not bailing out completely anymore but I just cannot get it to pass. Instead I'm getting this somewhat contradictory (?) failure message:

$ node ./test/fn-helpers-test.js
Failure: fn-helpers partialApply throws TypeError when called with no arg
  [assert.exception] when called with no arg: Expected {
  message: "first argument must be of type function! got 'undefined' of type undefined",
  name: "TypeError"
} but threw TypeError (first argument must be of type function! got 'undefined' of type undefined)
    AssertionError: [assert.exception] when called with no arg: Expected {
        message: "first argument must be of type function! got 'undefined' of type undefined",
        name: "TypeError"
      } but threw TypeError (first argument must be of type function! got 'undefined' of type undefined)
          at Object.ba.fail (e:\busterJS\devEnv\buster\node_modules\buster-assertions\lib\buster-assertions.js:257:25)
          at Object.fail (e:\busterJS\devEnv\buster\node_modules\buster-assertions\lib\buster-assertions.js:124:12)
          at Function.assert.exception (e:\busterJS\devEnv\buster\node_modules\buster-assertions\lib\buster-assertions.js:612:25)
          at Object.testCase.partialApply.throws TypeError when called with no arg (e:\busterJS\devEnv\buster-dev-tools\test\fn-helpers-test.js:14:20)
          at asyncFunction (e:\busterJS\devEnv\buster-test\lib\test-runner.js:230:19)
          at callTestFn (e:\busterJS\devEnv\buster-test\lib\test-runner.js:340:27)
          at p.then (e:\busterJS\devEnv\buster-test\node_modules\when\when.js:71:31)
          at Object.then (e:\busterJS\devEnv\buster-test\node_modules\when\when.js:177:11)
          at Object.bane.createEventEmitter.runTest (e:\busterJS\devEnv\buster-test\lib\test-runner.js:608:26)
          at Object.bane.createEventEmitter.runTests (e:\busterJS\devEnv\buster-test\lib\test-runner.js:576:29)

(in MINGW from msysgit)

At this point, I'm quite a bit lost. Anybody got a clue?

Thanks in advance,

___

meisl

p.s.: Generally I'm trying to work on Windows support, taking a workable dev-env as a first goal. It's got code itself - which should be guarded by tests, so... But I need not really argue for tests for everything in a project like this, or should I?

[Christian Johansen]

m.k...@sprax.eu writes:

> Hi,
>
> had set up a fresh dev-env on WinXP, node 0.8.16 (npm 1.1.16), then
> buster-dev-tools/run-tests.
>
> First off, it needs buster in project-list.js. This seems to have been
> busted out of there sometime in the past. Any reason or just accident?

It's _temporarily_ taken out. Reason is that August and I decided to get
all the 0.7 stuff on master a while ago to finish it up. So we updated
dev-tools to only include things ready for 0.7. Unfortunately, this work
has been quite delayed, so dev-tools have stayed incomplete for some
time.

>
> Then, with buster in project-list.js, it's still a complete fail: not with
> test failures but rather nearly everything exploding with an uncaught
> exception like this:

Yes, I've fixed run-tests now, by using buster-node to run tests
instead. Reason is that buster-node is a new module that ships the bare
necessities to get node testing going. It does not depend on the CLI
stuff from buster-test-cli, which is the final part missing from 0.7
(and the reason why the buster module is still not in project-list).

> At first I thought it might be due to the changed API of assert.exception (docs

> <http://docs.busterjs.org/en/latest/modules/buster-assertions/#assert.exception>should

> be adjusted!).
> But: shouldn't the old style ( assert.exception( someFunc() { ... },
> "ErrorName", "message-for-test-fail") ) still work with the new API?

It should. When I fixed the test setup, I only saw one failure. Try
pulling down my changes.

>
> p.s.: Generally I'm trying to work on Windows support, taking a workable
> dev-env as a first goal. It's got code itself - which should be guarded by
> tests, so... But I need not really argue for tests for everything in a
> project like this, or should I?

Cool, no need for arguments ;)

Sorry for the inconvenience

Christian

[m.k...@sprax.eu]

Great, that got me going - Thanks!

I was able to fix that last failure,

Failure: dev-utils installNpmDummy leaves existing package.json untouched
  [assert.equals] "contents of existent file: "asefdio{[%]}uzviubzizrene 7774$ expected to be equal to "asefdio{[%]}uzviubzizrene 7774$"

which really confused me. The character sequences are in fact equal, so..?!

Well, if you look closely enough you'll notice that the first (actual) is not enclosed in double quotes. 

This is even harder to see as there is indeed a double-quote right in front of it. But that is the closing one of the message prefix...

Now this is quite a faint hint that the "actual" isn't a string. In fact it's a buffer, which is due to an API change in node's fs.readFileSync.

I found this remarkably nasty, so maybe the reporting of such failures could be improved:

• omit the double quotes around the message prefix
• somehow make more clear that "actual" isn't really a string and what we're seeing is merely the result of its toString(). Maybe add some type info if it ain't a string, number or boolean?

Anyhow, the fix is here.

btw: I noticed that I still seem to have push rights on busterjs/buster-dev-tools but put it as a pull request so as not to get across overly obtrusive. It's been a while since, after all :)

__

meisl

[Christian Johansen]

> I found this remarkably nasty, so maybe the reporting of such failures
> could be improved:
>

> - omit the double quotes around the message prefix
> - somehow make more clear that "actual" isn't really a string and what

> we're seeing is merely the result of its toString(). Maybe add some type
> info if it ain't a string, number or boolean?

I agree. The strange use of quotes in assertion error messages is on my
radar, I'm going to fix that. I also think that adding a type hint to
these messages will be really useful, especially now that assert.equals
no longer does coercion.

>
> Anyhow, the fix is here<https://github.com/busterjs/buster-dev-tools/pull/15>
> .

>
> btw: I noticed that I still seem to have push rights on
> busterjs/buster-dev-tools but put it as a pull request so as not to get
> across overly obtrusive. It's been a while since, after all :)

Feel free to go ahead, we appreciate the help :) Pulled your pull
request though.

Christian

[m.k...@sprax.eu]

Very cool, thanks!

__

meisl