@BeforeMethod and multi-threaded testing

[joshua clingenpeel]

I've seen this brought up before but can't find a definitive answer.
Instead of necropost I thought I'd add some code and the things that
others have asked for in order to track this down. The situation - a
@BeforeMethod in a class with several tests. We expect @BeforeMethod
to run before each test method. It's seeding some test data. Run
sequentially, this works as expected. Run in parallel, we get odd
results.

Test Class:
@Test()
public class Foo {
private static final Logger logger = Logger.getLogger(Foo.class);

private float item;

@BeforeClass
public void setUp() {
logger.info("BeforeClass");
}

@BeforeMethod
public void beforeMethod() {
this.item = new Random().nextFloat();
logger.info(format("Setup {0}", this.item));
}

public void test01() {
logger.info(format("Run {0}", this.item));
}

public void test02() {
logger.info(format("Run {0}", this.item));
}

public void test03() {
logger.info(format("Run {0}", this.item));
}
}

TestNG XML:
<suite name="fooSuite" verbose="1" parallel="methods">
<test verbose="2" name="fooTest" annotations="JDK">
<classes>
<class name="Foo"/>
</classes>
</test>
</suite>

Logger Output:
2011-05-05 16:57:48,928 INFO (Foo:25) - BeforeClass
2011-05-05 16:57:48,937 INFO (Foo:31) - Setup 0.266
2011-05-05 16:57:48,937 INFO (Foo:31) - Setup 0.445
2011-05-05 16:57:48,939 INFO (Foo:31) - Setup 0.641
2011-05-05 16:57:48,946 INFO (Foo:35) - Run 0.266
2011-05-05 16:57:48,946 INFO (Foo:39) - Run 0.266
2011-05-05 16:57:48,948 INFO (Foo:43) - Run 0.266
--
Josh

[Cédric Beust ♔]

Hi Joshua,

You will need to be more specific to explain what you mean by "odd results".

The contract from TestNG's stand point is this: within the same thread, a @BeforeMethod will always be invoked before the corresponding @Test method. That's it.

Are you seeing something different?

-- 

Cédric

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to testng...@googlegroups.com.
To unsubscribe from this group, send email to testng-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

[Joshua Clingenpeel]

On May 5, 11:59 am, Cédric Beust ♔ <ced...@beust.com> wrote:
> Hi Joshua,
>
> You will need to be more specific to explain what you mean by "odd results".
>

Specifically: I have some random data I want to set for each test
case. In the example, I'm generating a random Float. I need that data
to be available when the test runs, but I don't want to repeat a block
of code in each test method. So I created the BeforeMethod and then
stored the Float in a member variable - `item`.

My expectations would be that each test would run with a different
value. When run sequentially - obviously - I get a new value before
each test method. When I run concurrently, the value of item is set
three times, but it's not guaranteed that each test will get access to
the value created for it in BeforeMethod before a different
BeforeMethod has come along and changed it again.

A fix we're using for this is to set the item in a DataProvider that's
used by each test method, but there again we have code repetition.

> The contract from TestNG's stand point is this: within the same thread, a
> @BeforeMethod will always be invoked before the corresponding @Test method.
> That's it.
>
> Are you seeing something different?
>

Given your description, I don't think TestNG is breaking its contract.
What I would like to see though is that when run in parallel mode,
each method getting its own thread means a new Foo class is
instantiated for each method to be run - state can be maintained and
we don't run into these race conditions. An option to force this
behavior would be great.

> --
> Cédric

[Cédric Beust ♔]

Hi Joshua,

In this case, I think that indeed, a data provider that returns random values is the best way to guarantee that each of your test method will be fed a different value, regardless of the parallel strategy.

-- 

Cédric

[Joshua Clingenpeel]

On May 12, 2:35 pm, Cédric Beust ♔ <ced...@beust.com> wrote:
> Hi Joshua,
>

> In this case, I think that indeed, a data provider that returns random
> values is the best way to guarantee that each of your test method will be
> fed a different value, regardless of the parallel strategy.
>
> --
> Cédric

Cool, and thanks. I guess no immediately plans to instantiate new test
classes in each thread then, eh? Keep up the good work.

[Cédric Beust ♔]

On Mon, May 16, 2011 at 1:00 PM, Joshua Clingenpeel <joshua.cl...@gmail.com> wrote:

Cool, and thanks. I guess no immediately plans to instantiate new test

classes in each thread then, eh? Keep up the good work.

Not really, no, mostly because you can obtain the same effect by initializing your state in @BeforeMethod, which is a better practice in my opinion, instead of scattering the initializations in 1) the field declaration, 2) the no arg constructor and 3) @Before, like JUnit encourages.

-- 

Cédric