Inconsistent results with parallel test execution

[Tarun Bhadauria]

Disclaimer - I am not sure if this with TestNG or how badly I am designing tests.

I am using testng 6.3.1 jar and Eclipse 3.4

My tests involves three entities - 

Data object class -  data for integration tests involving data for forms. Herein each form on a page could be represented by a different data object.  

Helper class - which fills in data in a form on page

Test class - which uses data object and help class to perform test.

Here is a cut down version of tests -

public class ParallelDataObject {

HelperClass helperClass = new HelperClass();

Data data;

@BeforeMethod

public void setTestData() {

data = new Data();

helperClass.setData(data);

}

@Test

public void passM1() {

helperClass.verifyFlag();

}

@Test

public void failM2() {

data.setFlag(false);

helperClass.setData(data);

helperClass.verifyFlag();

}

@Test

public void passM3() {

helperClass.verifyFlag();

}

}

class HelperClass {

Data data;

public void setData(Data data) {

this.data = data;

}

public void verifyFlag() {

assert data.getFlag();

}

}

class Data {

private boolean flag;

public Data() {

flag = true;

}

public Data setFlag(boolean flag) {

this.flag = flag;

return this;

}

public boolean getFlag() {

return flag;

}

}

When I run tests in sequential order all goes well. that is, - passM1, passM3 succeed and failM2 fail.

But when I set it for parallel method execution I see different results. at times all fail and at times two fail and some times I get expected result.

I am not sure how I should be using @BeforeMethod in this scenario. Or is there some grave flaw in how I am designing tests.

~tarun

 

[Cédric Beust ♔]

Your code is not multithread safe. Consider:

@Test

public void passM1() {

helperClass.verifyFlag();

}

@Test

public void failM2() {

data.setFlag(false);

helperClass.setData(data);

helperClass.verifyFlag();

}

Imagine that the first line of failM2 is run, then its thread gets preempted by the one that runs passM1(), which will do a verifyFlag(): suddenly passM1 fails.

The problem here is that you're sharing mutable data among threads without protecting it, so either use synchronized blocks or a different parallel mode.

-- 

Cédric

~tarun

 

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/testng-users/-/UgYCAl3SO6kJ.
To post to this group, send email to testng...@googlegroups.com.
To unsubscribe from this group, send email to testng-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

[Tarun Bhadauria]

thanks Cedric