Better support for counting tests

[kris]

Hi,

I came across an issue with flex mojos runner for flex unit. They
simply go through all the functions from the test classes annotated
with [Test] and use this number to control when the unit testing app
goes down. This creates a serious issue when you want to build your
own runners. I don't blame flex mojos guys as essentially there is no
good way to figure out number of test cases from the runner api. My
proposal is to add a public function to the runner interface that
gives total count of test cases.

Regards,
Kris

[Michael Labriola]

Kris,

That's not true at all. We do have test counts, which is actually
where a lot of work goes to ensure things like Theories and
Parameterized testing works properly with this count. FlashBuilder,
FDT and IntelliJ all seem to use this count correctly, which is how
they report the number of tests run at the beginning of the test run.

Counting the number of Test methods, as you pointed out, won't work at
all.

Regarding a good way to get this value, there is a property on the
Description object called testCount. Before the very first test is
run, this entire count is computed.

When the test run starts, all of the listeners have their
testRunStarted() method called and passed a description for this
entire set of tests. If you were to trace (description.testCount ) in
that method, you would see an accurate test count.

The description is where test information lives. All runners already
have a public API which provides the description.

Mike

[Kris]

All right, but that's not the way flex mojos has implemented this. I need to take a look at the latest stuff maybe they have it updated.

Thanks for sharing it with me. I'm planning on submitting a patch to flex mojos maybe it should include the changes for count as well.

Regards,
Kris

[Michael Labriola]

Kris,

Do you know where this code lives so that can fix it for them? We
don't need a major integration project handling this wrong and giving
us a black eye.

Mike

On Sep 15, 3:01 am, kris <kris.karczmarc...@gmail.com> wrote:

[Kris]

Here is a direct link to the file

http://svn.sonatype.org/flexmojos/trunk/flexmojos-testing/flexmojos-unittest-flexunit4/src/main/flex/org/sonatype/flexmojos/unitestingsupport/flexunit4/FlexUnit4Listener.as

Search for countTestCases() function.

Regards,
Kris

[Michael Labriola]

Thanks. I know we need better docs on a lot of this stuff which will
help

On Sep 15, 8:20 am, Kris <kris.karczmarc...@gmail.com> wrote:
> Here is a direct link to the file
>

> http://svn.sonatype.org/flexmojos/trunk/flexmojos-testing/flexmojos-u...

>
> Search for countTestCases() function.
>
> Regards,
> Kris
>

[Brian]

@Kris - Just a note. Since the flexmojos unit testing support is
written to support a variety of unit testing frameworks, Velo has
whipped up his own framework for bootstrapping the testing process.
All of our Ant build hooks, however, conform to the interfaces Mike
mentioned so there shouldn't be any issue with these tools.
Currently, our team doesn't contribute to the FM project or its unit
testing support.

Did you open up a JIRA ticket with FM or start a thread on the mailing
list? If you need any help interfacing with Velo to get a patch in,
let us know. Great find.

-Brian

On Sep 15, 8:49 am, Kris <kris.karczmarc...@gmail.com> wrote:
> All right, but that's not the way flex mojos has implemented this. I need to
> take a look at the latest stuff maybe they have it updated.
>
> Thanks for sharing it with me. I'm planning on submitting a patch to flex
> mojos maybe it should include the changes for count as well.
>
> Regards,
> Kris
>