Annotated test methods and return types

[Will McQueen]

Hi,

It seems that when I annotate a public method with @Test, then TestNG
doesn't consider it to be a test method unless the return type is
void. Why is that? I've found myself on several occasions wanting to
re-use a test method as a helper method for use in some other test
within the same class. I would expect that TestNG shouldn't care what
the return type is (ie, TestNG should just ignore the return value if
there is one).

Cheers,
Will

[Cédric Beust ♔]

Are you sure about this?  I can't test right now but I think I modified this a while back so that test methods don't have to be void.

--
Cedric

--

Cédric

[John A Thompson]

I saw this behavior last week.  Seemed like the test method was ignored, changing the return to void fixed the issue.

[Will McQueen]

Hi Cédric,

Short version:
***************
Output of test (with verbose=10) says:
"Method public java.lang.String com.quick.MyTest.foo() has a @Test
annotation but also a return value: ignoring it."

It seems that the code is ignoring the entire method rather than just
its return value?

Long version:
**************
I don't see my response here, so I think I mistakenly clicked "Reply
to author" instead of "reply", which probably sent my reply to John's
private email instead of to this newsgroup. I don't see a way to
retrieve sent posts, so I've created another response below...

I'm using testng-5.8-jdk15.jar on WinXP Pro with JDK 1.6.0_10, with
the following class:

package com.quick;
import org.testng.annotations.Test;
public class MyTest {
@Test
public String foo() {
System.out.println("This method isn't recognized by TestNG as a test
method.");
return "";
}
}

...and I get the following response (with verbose=10) showing that
the test method was skipped because it had a return value.

[Parser] Running:
C:\checkout\workarea\ws1.3.1\temp-testng-customsuite.xml

[RunInfo] Adding method selector:
org.testng.internal.XmlMethodSelector@1ad086a priority: 10
[TestClass] Creating TestClass for [ClassImpl com.quick.MyTest]
Method public java.lang.String com.quick.MyTest.foo() has a @Test
annotation but also a return value: ignoring it.
[SuiteRunner] Created 1 TestRunners
[TestRunner] Running test com.quick.MyTest on 1 classes, included
groups:[] excluded groups:[]
[TestClass]
======
TESTCLASS: com.quick.MyTest
[TestClass]
======

[TestRunner] Found 0 applicable methods

*********** INVOKED METHODS


***********

Creating C:\checkout\workarea\ws1.3.1\test-output
\ws1.3.1\com.quick.MyTest.html

===============================================
com.quick.MyTest
Tests run: 0, Failures: 0, Skips: 0
===============================================


===============================================
ws1.3.1
Total tests run: 0, Failures: 0, Skips: 0
===============================================

Creating C:\checkout\workarea\ws1.3.1\test-output\ws1.3.1\toc.html
Creating C:\checkout\workarea\ws1.3.1\test-output
\ws1.3.1\com.quick.MyTest.properties
Creating C:\checkout\workarea\ws1.3.1\test-output\ws1.3.1\index.html
Creating C:\checkout\workarea\ws1.3.1\test-output\ws1.3.1\main.html
Creating C:\checkout\workarea\ws1.3.1\test-output\ws1.3.1\groups.html
Creating C:\checkout\workarea\ws1.3.1\test-output\ws1.3.1\methods.html
Creating C:\checkout\workarea\ws1.3.1\test-output\ws1.3.1\methods-
alphabetical.html
Creating C:\checkout\workarea\ws1.3.1\test-output\ws1.3.1\classes.html
Creating C:\checkout\workarea\ws1.3.1\test-output\ws1.3.1\reporter-
output.html
Creating C:\checkout\workarea\ws1.3.1\test-output\ws1.3.1\methods-not-
run.html
Creating C:\checkout\workarea\ws1.3.1\test-output
\ws1.3.1\testng.xml.html
Creating C:\checkout\workarea\ws1.3.1\test-output\index.html
Creating C:\checkout\workarea\ws1.3.1\test-output\testng-results.xml

Cheers,
Will

[Cédric Beust ♔]

You are right, I forgot that I'm displaying a warning but still ignoring the method.

This was made necessary for backward compatibility.  If TestNG suddenly started allowing methods that return a value as valid test methods, existing users of the class level @Test annotation would see a lot of unwanted methods be run.

The recommended workaround is to create a void @Test method that invokes the method with a result and then ignores that result.

--

Cédric

[Will McQueen]

Hi Cédric,

That's too bad. I do use helper methods (called from a @Test method)
as a workaround, but I end-up needing to use a lot of them within the
same class in order to get around this limitation, so the test code
starts looking sloppy and not as easy to maintain. I feel strongly for
having TestNG recognize all annotated @Test methods as test methods
regardless of their return type, but I understand that we need to
maintain backward compatibility. So what can we do? I propose 2
solutions:

1) Is it possible to add this option to TestNG as an experimental
feature that's activated with a command-line switch? That way, users
such as myself who would trade backward compatibilty for the
convenience of this feature would have this option.

2) If not #1 isn't possible, then if it's not too much trouble, could
you please point me to places in the source code that would need to be
changed in order to have this feature? I see myself needing to create
private builds with this feature each time a new revision of TestNG
becomes available, so that I can have this incredibly useful
capability.

Thank you.

Cheers,
Will

On Nov 11, 12:34 pm, Cédric Beust ♔ <cbe...@google.com> wrote:
> You are right, I forgot that I'm displaying a warning but still ignoring the
> method.
>
> This was made necessary for backward compatibility.  If TestNG suddenly
> started allowing methods that return a value as valid test methods, existing
> users of the class level @Test annotation would see a lot of unwanted
> methods be run.
>
> The recommended workaround is to create a void @Test method that invokes the
> method with a result and then ignores that result.
>
> --

> ***Cédric
> *

> > Will- Hide quoted text -
>
> - Show quoted text -

[Cédric Beust ♔]

Hi Will,

#1 is certainly possible, it's just a matter of determining how big a problem is it to users (and it doesn't seem to be from what I can tell).

As for #2, TestNG emits a warning when it ignores a test method with a return value, just grep that message in the source and you'll find the exact place where to patch.

--

Cédric

[Will McQueen]

Thanks, Cédric :-)

Cheers,
Will

On Nov 21, 2:24 pm, Cédric Beust ♔ <cbe...@google.com> wrote:
> Hi Will,
>
> #1 is certainly possible, it's just a matter of determining how big a
> problem is it to users (and it doesn't seem to be from what I can tell).
>
> As for #2, TestNG emits a warning when it ignores a test method with a
> return value, just grep that message in the source and you'll find the exact
> place where to patch.
>
> --

> ***Cédric
> *

> > > - Show quoted text -- Hide quoted text -