TestNG behaviour when no methods are found

121 views
Skip to first unread message

Eran

unread,
Jun 20, 2006, 9:15:15 AM6/20/06
to testn...@googlegroups.com
Hi,

Today, if TestNG is invoked and it doesn't find any tests to run, it marks the test run as a success (0/0/0).
Obviously, if you open the reports you could see that no test was executed but if you run TestNG automatically by Ant and rely on the failureProperty to perform some actions, that can lead to a problem.
As it's common to make mistakes like misspelling a group, people might get the false impression that all tests pass when in fact no test was executed. Therefore, I believe that such an "empty run" should be considered as a failure.

What does everybody think?

Regards,
Eran

Alexandru Popescu

unread,
Jun 20, 2006, 9:46:17 AM6/20/06
to testn...@googlegroups.com
IMO a failing test is completely different from an empty test class.
What we can probably do, but I am not yet convinced is possible, is to
signal somehow that nothing was run.

./alex
--
.w( the_mindstorm )p.

Hani Suleiman

unread,
Jun 20, 2006, 9:49:09 AM6/20/06
to testn...@googlegroups.com
An exception or error message should be shown, explaining what was
searched and why tests weren't found, for example:

WARN: No tests found in groups 'foo', 'bar', 'poo'

Eran

unread,
Jun 20, 2006, 10:24:42 AM6/20/06
to testn...@googlegroups.com
Printing an warning/error message is definitely a must but unfortunately it requires a human to read it...
Perhaps we can set the failure property to a special value?

Regards,
Eran

Cédric Beust ♔

unread,
Jun 20, 2006, 11:10:43 AM6/20/06
to testn...@googlegroups.com
Hi Eran,

This is a good point and I've been bitten by it a few times as well.  I think it makes sense to make an exception for this case and have TestNG exit with an error message and an error code.

--
Cedric
--
Cédric
Reply all
Reply to author
Forward
0 new messages