Asynchronous custom assert methods

[Ryan Schmidt]

I'm trying to write tests for my graphics library. In each test, I'll use my library to draw something to an HTML 5 canvas, then compare it to a reference PNG. So, using the example of just drawing a circle with normal 2-D canvas commands, I want to be able to write a test that looks like this:

buster.testCase('foo', {
setUp: function() {
this.canvas = document.createElement('canvas');
this.canvas.width = this.width = 100;
this.canvas.height = this.height = 100;
document.body.appendChild(this.canvas);
this.ctx = this.canvas.getContext('2d');
this.ctx.fillStyle = 'white';
this.ctx.fillRect(0, 0, this.width, this.height);
},
'circle': function() {
this.ctx.lineWidth = 2;
var cx = this.width / 2;
var cy = this.height / 2;
var radius = Math.min(this.width, this.height) / 3;
this.ctx.arc(cx, cy, radius, 0, 2 * Math.PI);
this.ctx.stroke();
assert.matchImage(this.canvas, 'circle.png');
}
});

The custom matchImage assert method that I want to write will load the given "expected" image into a new canvas, then compare its pixels to those of the given "actual" canvas. I can start writing this method like this:

buster.assertions.add('matchImage', {
assert: function(actualCanvas, expectedImageName) {
return ...
},
assertMessage: 'Expected ${0} to match image ${1}!',
refuteMessage: 'Expected ${0} not to match image ${1}!',
expectation: 'toMatchImage'
});

But the problem is that the assert method wants me to return something immediately -- synchronously -- but loading an image is an asynchronous operation:

assert: function(actualCanvas, expectedImageName) {
var expectedCanvas = document.createElement('canvas');
expectedCanvas.width = 100;
expectedCanvas.height = 100;
document.body.appendChild(expectedCanvas);
var expectedCtx = expectedCanvas.getContext('2d');

var expectedImage = new Image();
expectedImage.onload = function() {
expectedCtx.drawImage(expectedImage, 0, 0);
...
};
expectedImage.src = buster.env.path + 'test/' + expectedImageName;
},

I understand that individual tests, and the setUp method, can be asynchronous, but can I write a custom assert method that is asynchronous? If not, how would you approach this problem?

[August Lilleaas]

There's no APIs for asynchronous asserts. You can always just make a plain function for your custom assertion.

function myCustomAssert(foo, bar, done) {
    assert(foo);
    refute(bar);
    asyncThing(done(function () {
        assert(somethingElse);
    }));
}

// ...

"my test": function (done) {
    myCustomAssert(1, 2, done);
}

You can only have one assertion per test this way, but one assertion per test is a good idea anyway :)

There are no plans for an API for individual asynchronous asserts. I would vote against it, as you can always just split it into multiple contexts with small async test cases, and use custom assert functions. But this is just my take on it, it's not just my decision.

[Christian Johansen]

I've thought about this, and I don't think async assertions would make much of a difference to your tests, but it would make quite an impact on the assertions api.

I suggest you solve your case by creating a custom assertion that compares the values when loaded, then call it from an async helper function that also loads the data, much like August suggested.

I've also been thinking about how to make it easier to have two async calls to complete a function (ie a done you must call twice), but didn't come up with a good solution so far.

[Ryan Schmidt]

On Mar 5, 2012, at 13:11, August Lilleaas wrote:

> There's no APIs for asynchronous asserts. You can always just make a plain function for your custom assertion.
>
> function myCustomAssert(foo, bar, done) {
> assert(foo);
> refute(bar);
> asyncThing(done(function () {
> assert(somethingElse);
> }));
> }

What would be the difference between these two:

function myCustomAssert(foo, bar, done) {
asyncThing(done(function() {
assert(somethingElse);
}));
}

function myCustomAssert(foo, bar, done) {
asyncThing(function() {
assert(somethingElse);
done();
});
}

The second is clearer to me, but I've seen the first suggested before and I'm wondering if there's some desirable side-effect of it that I'm not seeing. Is it that the anonymous function will be over as soon as an assert() fails, so done() would never be called in that case? That did not seem to happen when I tested it, but I couldn't think of any other reason.

> "my test": function (done) {
> myCustomAssert(1, 2, done);
> }
>
> You can only have one assertion per test this way, but one assertion per test is a good idea anyway :)

Agreed; I don't mind that restriction at all. I'll try it this way.

[Christian Johansen]

What would be the difference between these two:


function myCustomAssert(foo, bar, done) {
   asyncThing(done(function() {
       assert(somethingElse);
   }));
}


function myCustomAssert(foo, bar, done) {
   asyncThing(function() {
       assert(somethingElse);
       done();
   });
}


The second is clearer to me, but I've seen the first suggested before and I'm wondering if there's some desirable side-effect of it that I'm not seeing. Is it that the anonymous function will be over as soon as an assert() fails, so done() would never be called in that case? That did not seem to happen when I tested it, but I couldn't think of any other reason.

When the done function wraps the final callback, Buster has more control over what happens inside. So we're able to provide more information about exceptions and so on, as we don't have to rely on window.onerror to catch async exceptions.

Christian