Testing Rules
Michael Labriola
I have been working on a few new rules which will be available with
FlexUnit. I will be checking a few prototype rules into my 4.1 branch
shortly. They are designed to cover a few scenarios including testing
events, callbacks and (if Robert has a bit of time to work with me)
as3-signals. They use hamcrest matchers to make things much more
flexible.. I would like everyone to take a look at the syntax I am
proposing below and give some feedback. While I think it is a big
improvement, I think it could be tweaked further:
[Rule]
public var expectEvent:EventRule = new EventRule();
//One thing I would like specific comment on is the use of name or
type to describe the event being dispatched...
expectEvent.from( dispatcher ).hasName( "Yo" ).atLeast(1);
expectEvent.from( dispatcher ).
hasName( "Yo" ).
instanceOf( CustomEvent ).
hasProperties( {username:"user",password:"abc123" } ).
twice();
expectEvent.from( dispatcher ).
ofType( "Yo" ).
never();
This now encapsulates the needed Async functionality, so you can also
do this:
expectEvent.from( dispatcher ).
hasName( "Yo" ).
instanceOf( CustomEvent ).
hasProperties( {username:"user",password:"abc123" } ).
timeOut( 500 ).
twice();
If you do not specify a timeout, then the event must occur
synchronously.
When you create the rule, you can specify an option constructor
argument which determines if the events should be considered in order,
so you can specify that you expect event A, followed by event B, etc.
There is more here, but I wanted to gather some initial thoughts.
Thanks,
Mike
Drew Bourne
The syntax looks much better than the current usage of Async.
I prefer 'type()' or 'hasType()' over 'hasName()' or 'ofType()'
because the parameter to the Event constructor is called 'type', and
'ofType("eventType")' reminds me too much of 'instanceOf()'.
I'd also favour calling the 'timeOut()' function 'timeout()' as it
better mirrors the capitalisation of setTimeout().
Do you intend to / need to add a Async.handleEvent()-style method or
will users use a normal 'dispatcher.addEventListener()'?
Are there examples yet for event ordering? Why would enforcement of
ordering be specified in the EventRule constructor? I had a point here
about it being nice from a usage and implementation point of view but
may limit flexibility when mixing ordered and unordered events, then I
realised mixing ordered / unordered could be done with separate
EventRule instances for ordered and unordered events.
cheers,
Drew
Michael Labriola
Drew-
I could work with hasType().. perhaps also withTimeout().
I wasn't planning on exposing a handleEvent() here as the goal is to
change the thinking around async a bit. I do want to provide a way to
specify a method to be called, so
expectEvent.from( dispatcher ).hasType( "Yo" ).calls( myFunction );
Which allows simultaneously setting the expectation and specifying a
function to be called.
Give me some feedback on these thoughts and then we can get into
ordering :)
Mike
Michael Labriola
Side note for thought:
The one area I am having problem with is declaring a success. For
example, what if someone declares that an event should only happen
once.. what does that mean?
once in the timeout duration?
once before the method ends?
The first means that every test has to wait the full method timeout to
determine if something was a success or failure
The second obviously won't work for async.
There are some cases which are easier, for instance is someone says
something happens at least twice, then we can declare a success the
moment that it reaches 2 times. However, if someone says at most
twice, then we have the same issue outlined above. This is the only
thing that has been holding me up on finalizing this technique for a
while, so I would appreciate thoughts.
Mike
Kris
I'm glad you are moving away from the Async. The only thing I'd suggest to keep in mind is to allow some low level integration for libraries like morefluent.
Regards,
Kris
Drew Bourne
I wrote these syntax examples while thinking about this,
http://gist.github.com/633667
In writing those I feel it is more natural to get users to stick with
adding their own event listeners instead of wrapping them with
handleEvent. Doing so uses syntax they are already familiar with, and
keeps the async event expectations separate from whatever event
handling they want to do as part of the test.
Adding .calls() is good, matches Mockolate.
On determining success, defining which event signifies the end of the
test may be a reasonable requirement. See the couple of tests at the
bottom of the gist.
As an alternate example of async in
http://github.com/drewbourne/spectacular I accumulate all calls to
async() and wait until each of their timeouts expire or all the
functions have been called. In this case these calls are only ever
onces. To support other counts and atLeast, atMost I'd lean towards
having a function that must be called to indicate test completion.
Qunit from jQuery uses a stop() / start() workflow to indicate each
test stopping while async calls are made, and then starting again in
the async result handlers. Works well in that context and may be a
pattern to consider.
cheers,
Drew
Michael Labriola
Kris-
Exactly my goal. The hope is to enable things like morefluent and get
my stuff out of your way :)
mike
Kris
I also noticed you no longer require a reference to a current test which I love. This was a huge road block whenever you wanted to use Async in BeforeClass, AfterClass.
Thanks,
Kris
Michael Labriola
Drew-
I see this more like setting expectations on a mock than anything
else. That was my goal.
So, to me, this rule is less about handling an event in a generic way,
and more about the confirming that an object under test emitted
certain events. This particular rule is not meant to be a general
async replacement.This is the reason I am attempting to avoid having
users use addEventListener() directly.
I think one of the problems with the current async model is that it
tries to accomplish too many goals at once and hence does none well.
So, looking through your examples, I am trying to say, this MyEvent
should dispatch once within the context of this test and it should
have these properties. The purpose of the timeout to me is to indicate
that the object under test didn't meet its obligations within a given
time.
I am not envisioning having things like proceedOnEvent() etc. here as
those are about async flow control.
Does this make sense?
Mike
> I wrote these syntax examples while thinking about this,http://gist.github.com/633667
> In writing those I feel it is more natural to get users to stick with
> adding their own event listeners instead of wrapping them with
> handleEvent. Doing so uses syntax they are already familiar with, and
> keeps the async event expectations separate from whatever event
> handling they want to do as part of the test.
>
> Adding .calls() is good, matches Mockolate.
>
> On determining success, defining which event signifies the end of the
> test may be a reasonable requirement. See the couple of tests at the
> bottom of the gist.
>
Michael Labriola
p.s. we can start another thread on the async stuff, however, many of
those patterns don't work as well in the flash world due to the way
our async and call stack works. I think I have looked at every pattern
I have been able to find in every other language I have examined and
they all make assumptions which are invalid here.
If you guys would like to start that discussion, I am game :)
> I wrote these syntax examples while thinking about this,http://gist.github.com/633667
> In writing those I feel it is more natural to get users to stick with
> adding their own event listeners instead of wrapping them with
> handleEvent. Doing so uses syntax they are already familiar with, and
> keeps the async event expectations separate from whatever event
> handling they want to do as part of the test.
>
> Adding .calls() is good, matches Mockolate.
>
> On determining success, defining which event signifies the end of the
> test may be a reasonable requirement. See the couple of tests at the
> bottom of the gist.
>
Michael Labriola
Yeh, that is one of the huge problems. I like the way we build pieces
in FlexUnit 4 but it meant we never had the local context of the test.
Since a rule is defined on the test, it has an implicit reference to
both the running test case and test, which makes the async stuff much,
much cleaner.
The reworked async rule pieces will be a base that can be further
composed or extended. So, you will still have a morefluent rule, but
eventually that will (internally) use the async rule and never need to
call out to the static class ever again.
Mike
Drew Bourne
Much clearer about the goal of this rule now. My reference to using
addEventListener was more with regard validation that this rule may
not provide. With the proposed api though, any validation that needs
to happen on the event should be possible by this rule.
Continuing with the success/ failure cases, I think the EventRule will
have to expose whether it should wait until all the timeouts occurs or
continue as soon as it can. Some scenarios like once(), times(5) could
continue as soon as their expected counts are met. Others like
atLeast(3) and atMost(3) should probably wait for the timeouts to hit.
That distinction seems a bit arbitrary though. Similar to ordered
events EventRule could have a flag to indicate it should continue as
soon as possible or wait for timeout. I'd like to be able to override
it per expected event though.
cheers
Drew
Michael Labriola
I imagined that the calls() is the escape hatch. If there is something
that is just beyond our scope (and that they can't write a matcher
for) then they could use calls() to finally just handle it by
themselves.
Regarding success/failure:
So, a few people have asked questions off list, so, for everyone to be
clear, this is only important for async operations. Anything
synchronous can evaluate at the end. It is just the concern of async
operations. So, if things are async, atLeast() is the only one that is
easy... we can then blow off the remainder of the events. The others
are easy int he failure condition, you tell me once() and I get it
twice, then I fail.
The worst scenario though is that we get to exactly the right number.
If we get 1 and you expected 1, we need to wait until the timeout to
make sure you don't get more than 1 or that matcher should fail. The
option Drew points out is that we could have some type of 'aggressive'
mode, wherein if we say once() and we get the first 1 we consider it a
success immediately.. In this way, when you say once() we are actually
saying atLeast(1). I see the value of this but also the danger.
A good example if the Flex 3.4 SDK. It had a bug where it called
responders twice from RPC services. We caught that because our code
waited and didn't declare it a success after the first one.
The other opportunity, which I agree is a little rough... is we could
just explain all of this and encourage the use of atLeast() whenever
using a timeout unless you want a real count verification...
Would really like more feedback on this if anyone is so inclined..
This code works, is tested and is ready to go, I just don't want to
turn loose of an API that we will regret.
Besides, after this one is over, we can discuss the other low-level
crazy Async ideas I have.
Mike