Behaviour of assert.exception()

[Malcolm Locke]

Hi,

I'm wondering what peoples thoughts are about the flexibility of this
assertion. To save yous all looking it up, it currently stands like
this:


assert.exception(callback[, type])

Fails if callback does not throw an exception. If the optional type is
provided, the assertion fails if the callback either does not throw an
exception, or if the exception is not of the given type (determined by
its name property).

// Passes
assert.exception(function () {
throw new TypeError("Ooops!");
}, "TypeError");



My main issue with this is there is no way to test exception messages.

It looks like other frameworks allow this, e.g.:

// Jasmine
expect(function() { throw new Err("Yo"); }).toThrow();
expect(function() { throw new Err("Yo"); }).toThrow("Yo");
expect(function() { throw new Err("Yo"); }).toThrow(new Err("Yo"));

// QUnit
raises(function() { throw new Err("Yo"); });
raises(function() { throw new Err("Yo"); }, /Yo/);
raises(function() { throw new Err("Yo"); }, Err);
// Custom checks against the error via a callback
raises(function() { throw new Err("Yo"); }, function(err) { .. });

Thoughts?

Malc

[Fábio M. Costa]

I think it would be better to test the typerror than the message itself. I mean the message itself does not change how your program is going to execute.

But I agree that there should be a way to do it, since its nice to make sure that your messages are informative.

So, maybe creating a new type of assertion? assert.exceptionMessage?

Just my thoughts...

Best,

--

Fábio Miranda Costa

Front-end @ Yipit.com
twitter: @fabiomiranda
github: fabiomcosta

[Christian Johansen]

When testing for exceptions, I frequently want to test against the message. Not glue the exact message to the test, but "sweep" to make sure a few key words are part of the message. There's a problem with the signature that explains why the message is not there today:

assert.exception(func[, type[, message]]);

Where 'message' is a custom assertion message. It's impossible to tell if this should match the exception message with the current syntax. Do you have some concrete suggestion as to how we can fix this? I'd be happy to change the entire signature if we have a good enough solution.

Christian

--
MVH
Christian

[Malcolm Locke]

On Tue, May 01, 2012 at 08:07:56PM +0200, Christian Johansen wrote:
> When testing for exceptions, I frequently want to test against the message.
> Not glue the exact message to the test, but "sweep" to make sure a few key
> words are part of the message. There's a problem with the signature that
> explains why the message is not there today:
>
> assert.exception(func[, type[, message]]);
>
> Where 'message' is a custom assertion message. It's impossible to tell if
> this should match the exception message with the current syntax. Do you
> have some concrete suggestion as to how we can fix this? I'd be happy to
> change the entire signature if we have a good enough solution.

I think modelling based on the QUnit signature would be a nice solution.

// Just match an exception
assert.exception(func);

// Match the exception message against a regular expression
assert.exception(func, /expected message/);

// Match the exception message exactly against a string
assert.exception(func, "expected message");

// Match the exception type. Note TypeError, not "TypeError"
assert.exception(func, TypeError);

// Call a custom callback to check the exception
assert.exception(func, function(err) {
return err.isValidException(); // Return true or false
});

Here's the QUnit implementation, it's pretty simple:

https://github.com/jquery/qunit/blob/master/qunit/qunit.js#L397

Shall I put it in a pull request? :)

[Christian Johansen]

 // Match the exception type.  Note TypeError, not "TypeError"
 assert.exception(func, TypeError);

Ah, right. That makes it slightly harder to test with custom exceptions that simply override name, but solves the "impossible" API problem.
 

 // Call a custom callback to check the exception
 assert.exception(func, function(err) {
     return err.isValidException(); // Return true or false
 });

That's a nice touch, I've thought about that before.

Shall I put it in a pull request? :)

Please do :)

So, it will accept a function (required), and then optionally a regexp/string/constructor/function? How will you disambiguate a constructor, e.g. TypeError from a custom function? You can't specify more than one of these right?

Christian

[Malcolm Locke]

On Tue, May 01, 2012 at 11:46:39PM +0200, Christian Johansen wrote:
> Shall I put it in a pull request? :)
> >
>
> Please do :)
>
> So, it will accept a function (required), and then optionally a
> regexp/string/constructor/function? How will you disambiguate a
> constructor, e.g. TypeError from a custom function? You can't specify more
> than one of these right?

Somewhat hacky, but if function.name === "", i.e. anonymous, consider it
a custom validation function, otherwise consider it a constructor.

Anyway, pull request is in:

https://github.com/busterjs/buster-assertions/pull/6

Malc

[Christian Johansen]

Somewhat hacky, but if function.name === "", i.e. anonymous, consider it
a custom validation function, otherwise consider it a constructor.

Hmm, ok. I guess that'll work. I don't have any better suggestions... Will look at pull request now, thanks.

Christian

[meisl]

Somewhat hacky, but if function.name === "", i.e. anonymous, consider it
a custom validation function, otherwise consider it a constructor.

It also means that you can't pass in a helper function that you defined in your tests as it'd be taken as a constructor.
Just seen it, don't know if it's important; up to you to evaluate the trade-off.

[August Lilleaas]

I'd much rather prefer an API with either an object, or more arguments.

assert.exception(function () { ... }.{type: TypeError});
assert.exception(function () { ... }.{test: function (e) {}});

Or

assert.exception(function () {}, TypeError)
assert.exception(function () {}, null, function (e) {})
assert.exception(function () {}, TypeError, function (e) {})

[Christian Johansen]

I very much like the object approach. That also opens the possibility of checking both the type and the message in one go too. +1!

--
MVH
Christian

[Malcolm Locke]

Agree, I'll rework it.

Malc

[August Lilleaas]

I just noticed that I didn't include any rationale whatsoever along with my suggestion, so I'm glad/lucky that you both liked it :)

[Christian Johansen]

Here's my revised suggestion, I'm about to implement it, so scream out immediately if you disagree :)

assert.exception(fn); // Call fn, fail if it does not throw
assert.exception(fn, { ... }); // Call fn, compare exception with provided object in the same way as assert.match(exception, {}) would. Fail if no exception
assert.exception(fn, fn2); // Call fn, pass exception to fn2, fail if no exception or fn2 returns anything other than true

Christian

[Christian Johansen]

Forgot the custom message:

assert.exception(fn[, message]);
assert.exception(fn, { ... }[, message]);
assert.exception(fn, fn2[, message]);