Test(expects=...) doesn't play well with async rules
David Wolever
play well with rules… For example, given a simple rule which always
throws an IOError shortly after starting:
public class ErrorRule extends MethodRuleBase implements IMethodRule {
override public function evaluate(parentToken:AsyncTestToken):void {
super.evaluate(parentToken);
proceedToNextStatement();
setTimeout(function():void {
myToken.sendResult(new IOError("asdf"));
}, 10);
}
override protected function
handleStatementComplete(result:ChildResult):void {
if (result.error)
super.handleStatementComplete(result);
}
}
And a test case which expects an error:
public class SomeTest {
[Rule]
public var errorRule:ErrorRule = new ErrorRule();
[Test(expects="flash.errors.IOError")]
public function testError(): void { /* all the work is done by
ErrorRule */ }
}
I get the result:
at org.flexunit.internals.runners.statements::MethodRuleBase/
proceedToNextStatement()[...statementsMethodRuleBase.as:50]
at org.flexunit.internals.runners.statements::ExpectException/
evaluate()[...statementsExpectException.as:152]
at org.flexunit.internals.runners.statements::InvokeMethod/evaluate()
[...statementsInvokeMethod.as:73]
at org.flexunit.token::AsyncTestToken/sendResult()[...
okenAsyncTestToken.as:119]
at org.flexunit.internals.runners.statements::ExpectException/
handleNextExecuteComplete()[...statementsExpectException.as:189]
Error: Expected exception: flash.errors.IOError
It seems like the problem is that the 'expects' is being checked soon
as the test case terminates instead of when the rule sends the result.
Michael Labriola
I don't think you are doing what you think you are doing :)
First, let's pretend the expecting exception argument isn't there.
As soon as you call proceedToNextStatement() it is proceeding to the
next statement, which is Afters/Before,Exception handling, timeout,
async and the method invoker. It is going to work backwards up that
same stack, so async, timeout, *exception handling*, Afters/Before and
then your handleStatementComplete. It will get to your
handleStatementComplete long before your timeout happens. Flash is a
single threaded machine. It doesn't interrupt code execution...
ever... so unless your method did something asynchronous, there is no
way your timeout function can ever run before your
handleStatementComplete.
Right now your handleStatementComplete won't do anything unless it
gets an error, so, when the code gets back to this point, it will just
stop for now.
Now, let's put your expecting exception argument back in.
When you call proceedToNextStatement() it works all the way down the
stack again, runs the test and then starts back up the stack. When it
gets to expectingException your code hasn't run, there isn't an
exception on the method, so it is going to throw an exception because
there wasn't an error and you said their would be.
Now, regardless of all of that, you still have one more issue. You are
sending a new error into the token way above the expecting exception
in the stack. Which means even if the code in the timeout could happen
before the return from the test run, it would still throw an error at
this point as the expecting exception code has already done its job
and returned for someone else to do theirs... it is past the point
where it is looking for an error in the stack.
Async code in Flash Player is complex and you have to keep your head
around a bunch of stuff. It took me 3 days to port JUnit to FlexUnit
4. It took an additional 2.5 months to refactor everything to work
with async.
Mike
David Wolever
On 24-Jun-10, at 5:16 PM, Michael Labriola wrote:
> I don't think you are doing what you think you are doing :)
Obviously not :)
> First, let's pretend the expecting exception argument isn't there.
> ...
> Now, let's put your expecting exception argument back in.
> When you call proceedToNextStatement() it works all the way down the
> stack again, runs the test and then starts back up the stack. When it
> gets to expectingException your code hasn't run, there isn't an
> exception on the method, so it is going to throw an exception because
> there wasn't an error and you said their would be.
Ah, right — so that's the issue. I had expected the "expects" code to
be "higher up" the stack, so it would be triggered when I explicitly
send an error.
So, how do you think FlexUnit could make it easy for custom rules to
respect the "expects=" metadata?
Michael Labriola
I have a proposal on this functionality that I do plan to put out
soon. It is somewhat based on the way parts of Adobe's OSMF works
where we might be able to register rules to replace functionality
instead of just augment it. It is not flushed out completely yet but I
will post what I have considered as soon as we get beta 3 out.
Mike