Test class execution : chronological order.

[MG]

Hi

I have 5 java classes where each one is having multiple @Test methods, @BeforeClass method & @AfterClass method. Each test class is independent class. But the @Test are dependent on the setup done in @BeforeClass method. So the order of these test classes doesnt matter. Following is my testng xml
<suite name="ff" verbose="1">
<parameter name="groupName" value="ff"/>
<test name="SmokeTest">
<groups>
<run>
<include name="ff"/>
</run>
</groups>
<classes>
<class name="a"/>
<class name="b"/>
<class name="c"/>
<class name="d"/>
<class name="e"/>
</classes>
</test>
</suite>

What I am observing the chronological order is, as follows:
@BeforeClass for a
@Test 1 for a
@Test 2 for a
@BeforeClass for b
@Test 1 for b <--- fails
@Test 3 for a <--- fails
@Test 4 for a <--- fails
@AfterClass for a
@AfterClass for b
@BeforeClass for c
@Test 1 for c
@Test 2 for c
@AfterClass for c
@BeforeClass for d
@Test 1 for d
@Test 2 for d
@AfterClass for d
@BeforeClass for e
@Test 1 for e
@Test 2 for e
@AfterClass for e

Last 3 classes are executed properly but order of execution for first 2 classes is wrong. Why is it doing that? I dont have any dependency between these two classes.

How do I resolve this? I really appreciate help on this.

Thanks
MG
---------------------------------------------------------------------
Posted via Jive Forums
http://forums.opensymphony.com/thread.jspa?threadID=104042&messageID=173157#173157

[Alexandru Popescu ☀]

What version do you use? I don't think this is a problem, so I will
need some testcases to reproduce it.

bests,
./alex
--
.w( the_mindstorm )p.
TestNG co-founder
EclipseTestNG Creator

[Cédric Beust ♔]

I think you have more logical dependencies than you think:

On 7/27/07, MG <testng...@opensymphony.com > wrote:

Why?

B.beforeClass was invoked, so the first method of B should not fail.  If it does, you have another dependency you didn't tell TestNG about.

@Test 3 for a  <--- fails

Why?

A.beforeClass was invoked, so any method on A should pass.

@Test 4 for a <--- fails

Ditto.

Maybe the problem is that all the methods from A need to be run before the methods from B get invoked?  In this case, you can put these in different <test> tags...

--
Cédric

[MG]

Ok. I will explain little more. In beforeClass I change my product state, I run the tests & in AfterClass I revert the product state to the original state. But if two beforeClasses are run one after other then the product state is A + B which gives me result Z. Thus my tests fail.

>
> @Test 3 for a <--- fails
>
>
> Why?
>
> A.beforeClass was invoked, so any method on A should
> pass.
>
> @Test 4 for a <--- fails
>
>
> Ditto.
>
> Maybe the problem is that all the methods from A need
> to be run before the
> methods from B get invoked? In this case, you can
> put these in different
> <test> tags...

Putting it in different test tags is working fine. But ideally I dont need to do that right?
My last question. If the test classes have no dependencies on each other then what should be the order of exetution? I mean will one class will be executed completely (including beforeClass, Test, AfterClass) before executing another class?

All along I was assuming that the order of test classes is randomn but once it picks up one class then it executes it completely & then picks up another class. (This is all assuming I have no dependencies in between the classes).

Thanks
MG
>
> --
> CÃ(c)dric

>
> >
---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=104042&messageID=173199#173199

[MG]

>
> What version do you use? I don't think this is a
> problem, so I will
> need some testcases to reproduce it.

I am using 5.5. What do you mean by this is not a problem? The default behavior of testNG is not what we are seeing here or it is what we are seeing?

I need to come up with the tests which you can run. I cannot give you my current tests as they are very product specific.

Thanks
MG

>
> bests,
> ./alex
> --
> .w( the_mindstorm )p.
> TestNG co-founder

http://forums.opensymphony.com/thread.jspa?threadID=104042&messageID=173201#173201

[Mathias B.]

Hello,
Yes i think it is the correct behavior of TestNG because the afterClass and beforeClass annotations only guaranty that the beforeClass test will be executed before all the tests of this class and the afterClass test will be executed after all the tests of this class. But it is possible that some tests of another class are executed between them. (Correct me if I'm wrong)
What is the problem with putting each class in different <test> tags? Because It will make what you expect.

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=104042&messageID=174029#174029

[MG]

I havent got any official answer yet about the execution orders.

I cannot put all the test classes in diff test tags as I am passing some parameters to each test tag (approximately 10 test tags at this moment). So my actual test tag contains multiple input parameters with same test classes inside. The maintaince will be really bad if I tried to put each test in one test tag. so there will be 10 current test tags multipled by 5, number of classes, i.e 50 test tags. Its a huge testng.xml

I am still waiting for the answer.

Thanks
MG

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=104042&messageID=175215#175215

[Alexandru Popescu ☀]

On 8/2/07, MG <testng...@opensymphony.com> wrote:
>
> I havent got any official answer yet about the execution orders.
>
> I cannot put all the test classes in diff test tags as I am passing some parameters to each test tag (approximately 10 test tags at this moment). So my actual test tag contains multiple input parameters with same test classes inside. The maintaince will be really bad if I tried to put each test in one test tag. so there will be 10 current test tags multipled by 5, number of classes, i.e 50 test tags. Its a huge testng.xml
>
> I am still waiting for the answer.
>

I have re-read the whole thread and I still fail to find enough
details so that I can answer your question. If you'll consider sending
more information (other than a "sample" suite definition file) I will
try to look into it. And I think this topic has been covered partially
on another thread these last days (the guy there have provided us with
enough information + test cases).

bests,
./alex
--
.w( the_mindstorm )p.
TestNG co-founder

EclipseTestNG Creator

[MG]

Hi

I am attaching the code for 2 java classes. Methods in both the classes are using dependsOnMethods annotation. Dependent method (Test) sets the product state so that next method (Test) can run.
e.g. test1 runs single sign On, while test2 queries the user details & test3 logouts.

But the execution looks like this:

[testng] classB.BeforeClass
[testng] classB.methodB1
[testng] classA.BeforeClass
[testng] classA.methodA1
[testng] classB.methodB2
[testng] classB.methodB3
[testng] classB.methodB4
[testng] classB.methodB5
[testng] classB.methodB6
[testng] classB.AfterClass
[testng] classA.methodA2
[testng] classA.methodA3
[testng] classA.methodA4
[testng] classA.methodA5
[testng] classA.methodA6
[testng] classA.AfterClass

Here I had expected that first one class will complete invocation with all the tests & then the other class will start executing.

The testng looks like this:
<suite name="ff-notification" verbose="1">
<test name="Test">

<groups>
<run>
<include name="ff"/>
</run>
</groups>
<classes>

<class name="com.sun.identity.qatest.notification.classA"/>
<class name="com.sun.identity.qatest.notification.classB"/>
</classes>
</test>
</suite>

Pls advice me on this.

Thanks in advance

MG
---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=104042&messageID=180223#180223