@Factory test execution question

[Vincent Ho]

Hi,

 

I have a @Factory method which accepts a file containing rows of: test_id, param1, param2.

 

Each row has a unique test_id and its set of parameters.  An example data file is as follows:

 

------------------------------------------------------

test_id_1    param_1_1     param_1_2

test_id_2    param_2_1     param_2_2

. . . .

test_id_n    param_n_1     param_n_2

------------------------------------------------------

 

The @Factory method will parse this data file and returns an array of ITest objects where each

object will contain its test_id, param_1, and param_2 to be used in its test.  I am wondering:

 

1. How can I share a setup method in the test class between all tests (test_id_1, test_id_2, . . .) ?

 

I found @BeforeClass, @BeforeTest and @BeforeMethod are all invoked again for each test_id.

 

2. If some of the tests failed, how can I easily rerun the failed tests via testng-failed.xml mechanism ?

 

I found that testng-failed.xml will not record the parameters for the failed test and will run all tests again.

 

Vincent

[Cédric Beust ♔]

Hi Vincent,

On Mon, Apr 25, 2011 at 2:31 PM, Vincent Ho <vince...@gmail.com> wrote:

Hi,

 

I have a @Factory method which accepts a file containing rows of: test_id, param1, param2.

 

Each row has a unique test_id and its set of parameters.  An example data file is as follows:

 

------------------------------------------------------

test_id_1    param_1_1     param_1_2

test_id_2    param_2_1     param_2_2

. . . .

test_id_n    param_n_1     param_n_2

------------------------------------------------------

 

The @Factory method will parse this data file and returns an array of ITest objects where each

object will contain its test_id, param_1, and param_2 to be used in its test.  I am wondering:

 

1. How can I share a setup method in the test class between all tests (test_id_1, test_id_2, . . .) ?

Why not have them extend a common base class?

 

 I found @BeforeClass, @BeforeTest and @BeforeMethod are all invoked again for each test_id.

 

2. If some of the tests failed, how can I easily rerun the failed tests via testng-failed.xml mechanism ?

 

I found that testng-failed.xml will not record the parameters for the failed test and will run all tests again.

Correct, the facility we were discussing in a different thread this morning only applies to data providers and not factories, at least today. I'm not quite sure how this would work for factories since by definition of @Factory, the test classes it creates are not mentioned in the testng.xml file. Let me think about that.

--
Cédric

[Vincent Ho]

Hi Cédric,

 

Thanks for the quick response.  Regarding question #1 below, I'd like to clarify my use case:

 

> > 1. How can I share a setup method in the test class between all tests (test_id_1, test_id_2, . . .) ?

> Why not have them extend a common base class?

There is only one test class as below.   The testng.xml will specify this one class in a test block.
In effect, I want to have the ability to setup and teardown some resources to be shared by all the

tests executed by the same factory (something along the line of @BeforeFactory).  Am I insane ?

 

public class MyTest implements ITest {

     @Factory

     public Object[] createInstances() {

          // parse the data file and create ITest objects

          // return the array of ITest object (ie. MyTest)

     }

 

     public void testSetup() {

          // doing some setup to be shared by all tests

          // Note: I want this method to be called ONCE

          // @BeforeClass or @BeforeTest does not work

     }

 

     @Test

     public void testMethod() {

          // doing the test here, using the testSetup() above

          // Note: I DON'T want testSetup() called for every test

     }

 

     public void testCleanUp() {

          // doing the cleanup of testSetup() at the end of all tests

     }

}

 

Vincent

 

2011/4/25 Cédric Beust ♔ <ced...@beust.com>

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to testng...@googlegroups.com.
To unsubscribe from this group, send email to testng-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

[Cédric Beust ♔]

Hi Vincent,

Ah okay. Indeed, the fact that there is only one class is quite important :-)

As much as it pains me to admit it, I'm afraid storing that state in a static is probably your best bet here:

private static m_state;

@BeforeMethod

public void bm() {

  if (m_state == null) {

    // initialize state from whatever came from the constructor

  }

}

By the way, I recently added a feature that makes writing @Factory code a bit easier, I'll send this to the list in a separate email shortly, keep an eye out for it. Nothing fundamental, just cutting down a bit of boilerplate.

-- 

Cédric

--
Cédric

[Vincent Ho]

Hi Cédric,

 

Your suggestion below works as long as I don't need a cleanup method.  I want to achieve this sequence:

 

. . . execute some SHARE setup for all tests once

. . . run test 1

. . . run test 2

. . . run test <n>

. . . execute the cleanup for the SHARED setup above

 

I tried @Before/After<Method|Class> with your suggestion of static variable below I get this sequence:

 

. . . setup

. . . test 1

. . . cleanup

. . . test 2

. . . test <n>

 

Is it possible for me to achieve the above with current testng 6.0.1 ?  Please let me know what I'm missing.

 

Thanks,

[Cédric Beust ♔]

Hi Vincent,

Hard to tell because you didn't give me enough information on when that clean up should occur. How do you decide it's time to clean up?

-- 

Cédric

[Vincent Ho]

Hi Cédric,

 

That clean up should occur at the end of all the tests within the factory.  I just got it working with @BeforeTest and @AfterTest annotation.

 

Thanks,

Vincent

2011/4/26 Cédric Beust ♔ <ced...@beust.com>

[Cédric Beust ♔]

On Tue, Apr 26, 2011 at 1:00 PM, Vincent Ho <vince...@gmail.com> wrote:

Hi Cédric,

 

That clean up should occur at the end of all the tests within the factory.  I just got it working with @BeforeTest and @AfterTest annotation.

Ok, good. Another way could have been to use groups.

--
Cédric