stop/start vs./in addition to wait

[Steven Parkes]

Have you looked at qunit, the way it handles asynchrony? Rather than having a fixed wait, you simply stop the task queue, with the expectation that you'll restart it in the future in a callback.

So something like

it(...,function(){
mycode(function(){ expect(...); start() });
stop();
})

The actual placement of the start() and stop() are fairly loose: they're don't do anything, just pause/restart the qunit queue. mycode calls the callback at som point in the future. By executing stop() the test has told the qunit scheduler that the test isn't complete. After the callback occurs, the start() tell qunit to restart the scheduler again.

This works out better in a lot of cases than a wait with a time for me. In many cases, the tested code will run immediately, but with a timeout of 0 so it is not inline. Putting a real timer on there makes the tests take a lot longer.

Something else I did in the old screw.unit days was to have timeouts I could cancel so if the expected callback came back early, I could continue immediately. But now I kinda like the qunit way better.

Any thoughts?

[Erik Hanson]

On Thu, Nov 12, 2009 at 3:56 PM, Steven Parkes <smpa...@smparkes.net> wrote:
>
> Have you looked at qunit, the way it handles asynchrony? Rather than having a fixed wait, you simply stop the task queue, with the expectation that you'll restart it in the future in a callback.

Jasmine has (or had?) a waitsFor function that will wait for a given
latch function to return true. Something like:

var users;
runs(function() {
users = loadUsersAsynchronously();
});

var timeout = 2000;
waitsFor(timeout, function() {
return users.length > 0;
});

runs(function() {
expects(users.join(",")).toEqual("bob,joe");
});

Or something like that. It's been quite a while since I did any async
stuff in Jasmine.

-- Erik

[Steven Parkes]

> Jasmine has (or had?) a waitsFor function that will wait for a given
> latch function to return true. Something like:
>
> var users;
> runs(function() {
> users = loadUsersAsynchronously();
> });
>
> var timeout = 2000;
> waitsFor(timeout, function() {
> return users.length > 0;
> });
>
> runs(function() {
> expects(users.join(",")).toEqual("bob,joe");
> });

Yeah, it has stuff like that, but that's fundamentally time-based. I have cases where I need to wait for two events to fire and then continue. They both happen quickly, but asynchronously, in random order. Time-based timeouts aren't ideal for that.

I haven't tested it extensively, but I think it took about 10 lines of code to support stop/start, at least what I need for now.

[Christian Williams]

The timeout in waitsFor is just an upper bound, the test will actual proceed as soon as the condition is satisfied.  Does that do what you want?

(Maybe we should make the timeout an optional arg?)

--X [typos courtesy of my iPhone]

[Steven Parkes]

On Nov 13, 2009, at Nov 13,11:21 AM , Christian Williams wrote:

> The timeout in waitsFor is just an upper bound, the test will actual proceed as soon as the condition is satisfied. Does that do what you want?

I haven't looked at the code: for what definition of "as soon as"? Polling?

[Christian Williams]

Yup.  It polls every 100ms right now; we could safely bump that down to make tests using lots of waitsFor's quicker.

--X [typos courtesy of my iPhone]

[Steven Parkes]

> Yup. It polls every 100ms right now; we could safely bump that down to make tests using lots of waitsFor's quicker.

I think providing start/stop as an alternative is a good idea ... though I don't expect everyone (or anyone) to agree with that ...

[Christian Williams]

here's their example from the stop() docs:

test("a test", function() {
  stop(1000); // wait 1 second
  $.getJSON("/someurl", function(result) {
    equals(result.value, "someExpectedValue");
    start();
  });
});

in Jasmine I might write it like this:

it("fetches something", function() {
  var callback = jasmine.createSpy('callback');
  $.getJSON("/someurl", callback);
  waitsFor(1000, function() { return callback.wasCalled(); });
  runs(function() {
    expect(callback).wasCalledWith("someExpectedValue");
  });
});

BTW, it would be nicer if the waitsFor line read like this, but we'd have to change wasCalled from a boolean to a function:

  waitsFor(callback.wasCalled);

Anyway.  The words starts/stops aren't super obvious to me.  Maybe something like this?

it("fetches something", function() {
  jasmine.testHasNotCompletedYet();
  $.getJSON("/someurl", function() {
    expect(callback).wasCalledWith("someExpectedValue");
    jasmine.testHasCompleted();
  });
}).timeout(1000);

--X [typos courtesy of my iPhone]

[Steven Parkes]

> \here's their example from the stop() docs:

>
>
> test("a test", function() {
> stop(1000); // wait 1 second
> $.getJSON("/someurl", function(result) {
> equals(result.value, "someExpectedValue");
> start();
> });
> });

Not necessarily terribly well documented. Stop doesn't wait for a second. That's actually the maximum it will wait. After that, it aborts the test. None of the jquery tests that use qunit use stop with a parameter: they just say stop() which means wait indefinitely.

Note stop() only sets a flag/callback in line. It stops the scheduler, the next time it gets control (after return from the test). This does cause many people (including me) confusion at first.

The bigger issue to me is that it doesn't use polling. It starts as soon as you say start(). It's implemented by having the scheduler return rather than loop on detecting stop() and to have the start() function do the setTimeout(next_,0) to restart it (mixing jaz and qunit here). That's what my patch does as well.

Again, I don't expect any/everyone to agree, but I prefer to avoid polling, including having a predicate that's repeatedly evaluated. My code is highly event driven, and the stop/start model fits that very well, where stop and start are viewed as events sent to the scheduler. None of my code has an explicit concept of "wait".

> Anyway. The words starts/stops aren't super obvious to me. ]]

100% on this. Once you understand what it's doing, it does kinda make sense, but I think a lot of people struggle with it at first. Doesn't help that they put stop() as the first line. Placement within the block doesn't matter, so maybe it's less confusing to highlight that by putting it first, but ...

> it("fetches something", function() {
> jasmine.testHasNotCompletedYet();
> $.getJSON("/someurl", function() {
> expect(callback).wasCalledWith("someExpectedValue");
> jasmine.testHasCompleted();
> });
> }).timeout(1000);

- Given the amount I call this, feels kinda wordy.
- The timeout at the end is kinda interesting. It makes the first *HasNot* redundant, which is similar to the qunit's qunit.async which is qunit.test with an implicit stop. I'm going to call it with nothing a lot (I don't timeout tests that shouldn't take time), so .timeout() feels strange. I'm split over the tail position of the call. I kinda like it because it doesn't dirty the signature, but it's an important issue when reading the test and putting it at the end is subtle.
- other possible terms for stop/start: async()/sync(). incomplete()/complete().

I kinda like this:

it("...",function(){
...
$.get(...,function(){
...
complete();
});
incomplete(<optional timeout>);
});

I think I could actually explain that to people. It sounds (mostly) declarative rather than imperative which seems like a better match to what's going on. I know it's bashing the global namespace a little (more), but jasmine.complete() is semantically wrong. You could, of course, just bash test and end up with test.complete() and test.incomplete().

[Erik Hanson]

On Fri, Nov 13, 2009 at 12:34 PM, Christian Williams
<antix...@gmail.com> wrote:
> BTW, it would be nicer if the waitsFor line read like this, but we'd have to
> change wasCalled from a boolean to a function:
>
> waitsFor(callback.wasCalled);

waitsFor could check the type of the parameter and if it's a function,
call it and use its result, otherwise just assume the parameter is
truthy/falsey.

-- Erik

[Christian Williams]

Kinda, except that the truthiness/falsiness evaluation needs to be deferred so it can be checked over and over, so in a function...

--X [typos courtesy of my iPhone]

[Steven Parkes]

On Nov 13, 2009, at Nov 13,1:07 PM , Steven Parkes wrote:

> it("...",function(){
> ...
> $.get(...,function(){
> ...
> complete();
> });
> incomplete(<optional timeout>);
> });

The more I think about this, the more I like it.

A few more thoughts:

Basically, at the end of every code block, you could "declare" whether you were done or not, but ... (big but, see below. I could explain that to people: just say incomplete() before returning so jaz knows you're not done. Saying nothing leaves you with the previous state which defaults to complete.

But there's one issue that I'd like to address that I don't have an answer for in my current code, which is what if I call the callback synchronously, e.g., what if the above .get where synchronous. Then it executes the complete followed by the the incomplete and gets confused/hangs.

I'm actually thinking it'd be cool to have stack-like count, incomplete incrs, complete decs, and as long as the "net complete" was zero, jaz would continue. I think that would work regardless of whether the call back was async or sync. But, then I haven't thought all that hard about it yet ...

[Christian Williams]

I like complete() / incomplete(<optional timeout>), but yeah I feel
gross about how much we've already dumped into the global ns.

Personally I never use the waitsFor/runs stuff. I mostly mock out
anything that would be async cuz it's usually not something I want
happening in a unit test anyway (like network calls, animation delays,
etc.), and pass spies in as callbacks. I'll give an example when I'm
at a computer.

That said, the code for using spies that way looks clunky. I'd love to
come up with something nicer.

--X [typos courtesy of my iPhone]

[Christian Williams]

So I usually mock out whatever is async in a test, like this:

// var App = function() {};
// App.prototype.showAlert = function(msg) {};
// var $ = { getJSON: function(url, data, callback) {} };

App.prototype.logIn = function(name, password) {
  var self = this;
  $.getJSON('/login', {name: name, password: password}, function(response) {
    if (response.success) {
      self.loginToken = response.token;
    } else {
      self.showAlert(response.message);
    }
  });
};

describe('App login', function() {
  var app;
  var jsonCallback;

  beforeEach(function() {
    app = new App();

    spyOn($, 'getJSON').andCallFake(function(url, data, callback) {
      expect(url).toEqual('/login');
      expect(data).toEqual({name: 'username', password: 'pw'});
      jsonCallback = callback;
    });
  });

  it('correctly handles success response', function() {
    app.logIn('username', 'pw');
    jsonCallback({success: true, token: 'fake-token'});
    expect(app.loginToken).toEqual('fake-token');
  });

  it('correctly handles failure response', function() {
    app.logIn('username', 'pw');
    spyOn(app, 'showAlert');
    jsonCallback({success: false, message: 'bad password'});
    expect(app.showAlert).wasCalledWith('bad password');
  });
});

fwiw.

--X [typos courtesy of my iPhone]

[Steven Parkes]

> So I usually mock out whatever is async in a test, like this:

Thanks for the example. It actually looks pretty clean. I haven't done much mocking/spying/what-have-you in Jasmine at this point but it looks cleaner than a lot of the other JS mock stuff I've seen.

As to my stuff, this style doesn't fit as well with my current code, which is internally asynchronous. It's at least in part that that asynchrony that I need to test.