Why does @Test(enabled = false) at class level cause @Before.... methods to not run?

[martin.p...@gmail.com]

Hello,

From my understanding the @Test(enabled = false) causes all public
methods in a class not to run. That way if you have a bunch of public
non-test methods you simply add the enabled=false at the class level
and annotate only the remaining test methods. However, if you do this
then all the @Before... methods don't get invoked even when I've added
enabled = true, alwaysRun = true to them.

Is this a bug or have I missed something here?

Thanks

Here is some example code:

============================================
package my.test2;

import static org.testng.Assert.assertTrue;

import org.testng.annotations.AfterMethod;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;

@Test(testName = "Test1", enabled = false)
public class Test1 {
private boolean setup = false;
private boolean teardown = false;
private boolean setupTest = false;
private boolean teardownTest = false;

@Test(enabled = true)
public void test() {
System.out.println("test called");
assertTrue(setupTest, "setUpTest was not called");
assertTrue(setup, "setup was not called");
}

@BeforeTest(enabled = true, alwaysRun = true)
protected void setUpTest() {
System.out.println("setUpTest called");
setupTest = true;
}

@AfterTest(enabled = true, alwaysRun = true)
protected void tearDownTest() {
System.out.println("tearDownTest called");
teardownTest = true;
}

@BeforeMethod(enabled = true, alwaysRun = true)
protected void setUp() {
System.out.println("setUp called");
setup = true;
}

@AfterMethod(enabled = true, alwaysRun = true)
protected void tearDown() {
System.out.println("tearDown called");
teardown = true;
}

public void nonTestMethod() {
throw new RuntimeException("nonTestMethod should not have been
invoked!");
}
}
============================================

and here is the testng output
============================================

[Parser] Running:
My

[RunInfo] Adding method selector:
org.testng.internal.XmlMethodSelector@49c449c4 priority: 10
[TestClass] Creating TestClass for [ClassImpl my.test2.Test1]
[SuiteRunner] Created 1 TestRunners
[TestRunner] Running test Test1 on 1 classes, included groups:[]
excluded groups:[]
[TestClass]
======
TESTCLASS: my.test2.Test1
[TestClass] BeforeMethod: my.test2.Test1.setUp()
[TestClass] Test : my.test2.Test1.test()
[TestClass] Test : my.test2.Test1.nonTestMethod()
[TestClass] AfterMethod : my.test2.Test1.tearDown()
[TestClass]
======

[Invoker 1096171862] Skipping @BeforeTest my.test2.Test1.setUpTest()
because my.test2.Test1 is not enabled
[TestRunner] Found 2 applicable methods
[TestRunner] WILL BE RUN IN RANDOM ORDER:
[TestRunner] my.test2.Test1.nonTestMethod()
[TestRunner] on instances
[TestRunner] my.test2.Test1@6b606b6
[TestRunner] my.test2.Test1.test()
[TestRunner] on instances
[TestRunner] my.test2.Test1@6b606b6
[TestRunner] ===
[Invoker 1096171862] Skipping @BeforeMethod my.test2.Test1.setUp()
because my.test2.Test1 is not enabled
[Invoker 1096171862] Invoking my.test2.Test1.test
test called
[Invoker 1096171862] Skipping @AfterMethod my.test2.Test1.tearDown()
because my.test2.Test1 is not enabled
[Invoker 1096171862] Skipping @AfterTest my.test2.Test1.tearDownTest()
because my.test2.Test1 is not enabled

*********** INVOKED METHODS

my.test2.Test1.test() 112592566

***********

Creating C:\cygwin\tmp\testng-bug\bin\test-reports\My\Test1.html
FAILED: test
java.lang.AssertionError: setUpTest was not called expected:<true> but
was:<false>
at my.test2.Test1.test(Test1.java:20)
... Removed 25 stack frames

===============================================
Test1
Tests run: 1, Failures: 1, Skips: 0
===============================================


===============================================
My
Total tests run: 1, Failures: 1, Skips: 0
===============================================

Creating C:\cygwin\tmp\testng-bug\bin\test-reports\My\toc.html
Creating C:\cygwin\tmp\testng-bug\bin\test-reports\My\Test1.properties
Creating C:\cygwin\tmp\testng-bug\bin\test-reports\My\index.html
Creating C:\cygwin\tmp\testng-bug\bin\test-reports\My\main.html
Creating C:\cygwin\tmp\testng-bug\bin\test-reports\My\groups.html
Creating C:\cygwin\tmp\testng-bug\bin\test-reports\My\methods.html
Creating C:\cygwin\tmp\testng-bug\bin\test-reports\My\methods-
alphabetical.html
Creating C:\cygwin\tmp\testng-bug\bin\test-reports\My\classes.html
Creating C:\cygwin\tmp\testng-bug\bin\test-reports\My\reporter-
output.html
Creating C:\cygwin\tmp\testng-bug\bin\test-reports\My\methods-not-
run.html
Creating C:\cygwin\tmp\testng-bug\bin\test-reports\My\testng.xml.html
Creating C:\cygwin\tmp\testng-bug\bin\test-reports\index.html
Creating C:\cygwin\tmp\testng-bug\bin\test-reports\testng-failed.xml
Creating C:\cygwin\tmp\testng-bug\bin\test-reports\My\testng-
failed.xml
Creating C:\cygwin\tmp\testng-bug\bin\test-reports\testng-results.xml

[borojevic]

Hi Martin,
according to documentation:
"The effect of a class level @Test annotation is to make all the
public methods of this class to become test methods even if they are
not annotated"
so all public methods will be test methods, it does not matter if you
annotate some of the public methods with test annotation.

On the same way when you say enabled=false for all test methods on the
class level, you disabled all your test methods
so there is no reason to run before and after methods because you
don't have any tests to run in that class.
I think that is normal behavior, not a bug.

On Dec 4, 12:47 pm, "martin.petrov...@gmail.com"

[martin.p...@gmail.com]

Thanks for replying.

I think your second point is not entirely correct. If you set
enabled=false for all test on the class level, but override at least
one method by (enabled=true as on the test() method in the above
example), then there is one test that is enabled in the class (and is
executed by testng). For that reason the @Before and @After... methods
should have been triggered.

The behaviour should either be:
1) when enabled=false is set at the class level then all method in
the class are disabled regardless if this is overridden by a
enabled=true at a method level, or
2) when enabled=false is set at the class level then all method in
the class are disabled unless the behaviouir is ovrriden by
enabled=true at a method level - in which case the @Before... and
@After... methods should be triggered if at least one test method is
enabled as example above.

My reason for doing this was because my test class implemented an
interface that has a lot of public methods that are not actually
tests. So rather than annotating each non-test method with
enabled=false, I did it at a class level and just annotated the one or
two test methods with enabled=true ... yes I'm a lazy programmer.
Having a Test class implement some interface may not be good design
but I came across this while migrating someone's junit test cases. The
workaround is to annotate all the non-test methods with @Test(enabled
= false), but I thought it was still worth mentioning as a potential
bug.

[Alexandru Popescu ☀]

On Dec 5, 2007 1:33 PM, martin.p...@gmail.com

<martin.p...@gmail.com> wrote:
>
> Thanks for replying.
>
> I think your second point is not entirely correct. If you set
> enabled=false for all test on the class level, but override at least
> one method by (enabled=true as on the test() method in the above
> example), then there is one test that is enabled in the class (and is
> executed by testng). For that reason the @Before and @After... methods
> should have been triggered.
>
> The behaviour should either be:
> 1) when enabled=false is set at the class level then all method in
> the class are disabled regardless if this is overridden by a
> enabled=true at a method level, or

Then there is no overridding.

> 2) when enabled=false is set at the class level then all method in
> the class are disabled unless the behaviouir is ovrriden by
> enabled=true at a method level - in which case the @Before... and
> @After... methods should be triggered if at least one test method is
> enabled as example above.
>

Yes, I think you are right. If you can send out a small testcase
reproducing the problem then it will be easier and quicker for us to
get to it.

> My reason for doing this was because my test class implemented an
> interface that has a lot of public methods that are not actually
> tests. So rather than annotating each non-test method with
> enabled=false, I did it at a class level and just annotated the one or
> two test methods with enabled=true ... yes I'm a lazy programmer.
> Having a Test class implement some interface may not be good design
> but I came across this while migrating someone's junit test cases. The
> workaround is to annotate all the non-test methods with @Test(enabled
> = false), but I thought it was still worth mentioning as a potential
> bug.
>

The only thing I am not sure about your first example: why do you need
the top/class level @Test annotation? If you just remove it, and
annotate the 2-3 methods you have as tests things should work.

./alex
--
.w( the_mindstorm )p.
TestNG co-founder
EclipseTestNG Creator

[Saverio Miroddi]

I attach the test case.
I was just posting the same problem.

=======================================================

I also add another problem:

I am using the TestNG eclipse plugin.
I run the test class; and all tests fail. So I go on the "Failed test" tab, and try to run or debug a failed test randomly chosen, but nothing happens.

I _don't_ want to run testng-failed.xml because it contains all the test. All i neeeeeed is one.

To be more specific: the test is a factory one. Is it possible that this situation is not handled by testng eclipse?

Test case attached.

=======================================================

Cheers,
Saverio
---------------------------------------------------------------------
Posted via Jive Forums
http://forums.opensymphony.com/thread.jspa?threadID=173916&messageID=249122#249122