Settings up Fixtures using ITestNGListener

[ali]

Hi,

I'm experimenting with annotation based declaration of test
dependencies using the testng Listener paradigm.
I have a reasonably involved code base that often has external
dependencies for many unit tests to run - an example contrived layout
would be

src/test/package-a/TestA1.java (Tests in this class depend on
FixtureX)
|- testA1a
|- testA1b
src/test/package-a/TestA2.java (Tests in this class depend on
FixtureY)
|- testA2a
|- testA2b
src/test/package-b/TestB1.java (Tests in this class depend on FixtureX
& Y)
|- testB1a
|- testB1b

To make this example a bit more relevant - lets say FixtureX is
memcache & FixtureY is redis
Currently, I'm using ISuiteListener to implement the setup and
teardown of my fixtures as follows

class FixtureX implements ISuiteListener
{
public void onStart(ISuite suite) {
// startup memcached
}
public void onFinish(ISuite suite) {
// shutdown memcached
}
}

.. and annotating each test class that depends on the fixture with
@Listeners ( {FixtureX.class} )
@Listeners ( {FixtureX.class, FixtureY.class} )
etc...

Now, the execution flow ends up looking like this when I run all the
tests in one run:
FixtureY->onStart
FixtureX->onStart
PASSED: testA1a
PASSED: testA1b
PASSED: testB1a
PASSED: testB1b
PASSED: testA2a
PASSED: testA2b
FixtureY->onFinish
FixtureX->onFinish

What I'm trying to achieve however - is to have a per class onStart /
onFinish for fixtures that each test class declares so as to guarantee
a clean state for each class of tests (I could do a per test setup/
teardown but that becomes overkill and slows down the tests
significantly). I guess I'm looking for an IClassListener type
behavior :)

The @BeforeClass & @AfterClass annotations would work well for this -
but result in a lot of copy paste business which I'm trying to avoid
in my tests.

Would really appreciate any insights on whether I'm going down the
wrong path completely - or if theres something obvious I've missed.

best,
./A

[Cédric Beust ♔]

Hi Ali,

I didn't read your email in details so apologies if I'm missing something, but I'm mostly commenting on:

The @BeforeClass & @AfterClass annotations would work well for this but result in a lot of copy paste business which I'm trying to avoid in my tests.

The traditional way of avoiding this with TestNG is to put these methods in a base class and have your test classes extend this class. Wouldn't this work for you?

If the answer is no, I'd like to hear why, and it might also point out toward the need for something similar to JUnit's rules (are you familiar with these?). Would rules help here? If yes, in what way that doesn't work with the base class approach?

-- 

Cédric

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to testng...@googlegroups.com.
To unsubscribe from this group, send email to testng-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

[ali]

Cedric,
Thank you for the quick response.

Re base class: this only works well if I only have one fixture that my
test class depends on (as you can probably tell by now, these tests
are more 'integration' than unit). I guess there is a possibility of
each test requiring any fixture to extend a common 'FixtureProvider'
base class which starts up and tears down a bunch of fixtures based on
parameters or even custom annotations - but to me thats starting to
lend more towards a 'test framework' feature than something one would
implement themselves.

Re junit Rules: I had not looked at them before, but now that you have
mentioned it @ClassRule from junit does almost exactly what I want -
ala:

class TestA1{
@ClassRule
public static FixtureX = new FixtureX();
@Test
public void testA1a(){};
@Test
public void testA1b(){};
}
class TestB1{
@ClassRule
public static FixtureX = new FixtureX();
@ClassRule
public static FixtureY = new FixtureY();
@Test
public void testB1a(){};
@Test
public void testB1b(){};
}

I guess one could argue that testng provides a near equivalent with
something like the sample below, however the @Rule / @ClassRule
annotation does seem to lend to a slightly cleaner test.

class TestA1{
@BeforeClass
public void setupFixtures(){
FixtureX.before()
}
@AfterClass
public void teardDownFixtures(){
FixtureX.after()
}

@Test
public void testA1a(){};
@Test
public void testA1b(){};
}

class TestB1{
@BeforeClass
public void setupFixtures(){
FixtureX.setUp();
FixtureY.setUp()
}
@AfterClass
public void teardDownFixtures(){
FixtureX.tearDown()
FixtureY.tearDown()
}
@Test
public void testB1a(){};
@Test
public void testB1b(){};
}

Based on the (very verbose) examples above - would you consider this
use case as something to support at the testng framework level?

best,
./A

[Cédric Beust ♔]

Hi Ali,

I'm open to discussing it but I'm not convinced we have exhausted all the other options yet :-)

For example, an ITestListener will notify you whenever a test method starts or ends. With some book keeping on your end, you could use onTestStart() to invoke code that would only be run the first time you see a certain class.

What do you think?

-- 

Cédric

On Tue, May 29, 2012 at 6:16 PM, ali <a...@indydevs.org> wrote:

[ali]

You're right - there are definitely a couple of ways to skin this cat - in fact, I've managed to narrow down my use case to be supported by:

- a Fixture annotation that takes as parameters an array of BaseFixture extension classes which provides a startup and shutdown implementation

- a standard FixtureProvider base class that inspects the annotations on whichever test class that extended it and provides a BeforeClass & AfterClass method which iterate over the Fixtures declared.

This implementation provides a way to use either implicit annotations or explicit initializations of the fixtures.

I guess the question now remains if my use case is unique or whether many people are implementing custom utility classes when working with testng.

best,

./A

p.s. I just realised we were both presenting at gtac 2007 ;)

-- 

Cédric

To unsubscribe from this group, send email to testng-users+unsubscribe@googlegroups.com.

[ali]

re: ITestListener - i personally want to stay away from that for BeforeClass/AfterClass style setup/teardowns as it it would be rather involved to ensure that the teardown of the fixture happens at the appropriate time.

./A