How can I get testNG to run tests sequentially before repeating?

[Eric Liu]

I am using a dataProvider and set up 3 tests with priorities 1,2,3.

I want the tests to run 1,2,3 and then repeat them with different data each time but as of now, it runs the first test 3 times, then second 3 times, and so on. My code looks something like this.

@Test(priority=1, dataProvider provider)

public void test1(asdf){ .. }

@Test(priority=2, dataProvider provider)

public void test1(asdf){ .. }

@Test(priority=3, dataProvider provider)

public void test1(asdf){ .. }

Is this possible with TestNG? It seems like a trivial thing but I can't find anything concerning this. Thanks

[Alexey Anischenko]

Short answer - no, there is no easy way in TestNG to do it ("it" means creating a sophisticated sequence of test methods)

Long answer - there is a glitchy, buggy, and not always working way of forging your own sequence of test methods:

1. annotate your test class with @Listeners({ com.yourPackageName.test.YourMethodInterceptor.class })

2. implement the interceptor, where you can do anything with that collections of test methods to be called.

There WILL be problems with setup-teardown methods if you change the total number of test methods (vs. the original content of a collection). So, you need to do your own setup-teardown, not relying on TestNG annotations.

The teardown fails, setup usually works.

Below is the working part of one of my tests:

-------------------------------------------

import java.util.LinkedList;

import java.util.List;

import org.testng.IMethodInstance;

import org.testng.IMethodInterceptor;

import org.testng.ITestContext;

/**

 * This receives the list of TestNG tests before execution, and can process the list, adding/removing the test methods.

 * 

 */

public class YourMethodInterceptor implements IMethodInterceptor {

private List<IMethodInstance> customTestsList = new LinkedList<IMethodInstance>();

@Override

public List<IMethodInstance> intercept(List<IMethodInstance> methods, ITestContext context) {

if (customTestsList.isEmpty()) { // forging our custom tests list from instances created by TestNG

customTestsList.add(findMethodInstance(methods, "login"));

customTestsList.add(findMethodInstance(methods, "goto1stPage"));

for (int loops = 0; loops < 4; loops++) { // 4 loops of...

int navLimit = (int) (Math.random() * 4 + 1);

for (int ix = 0; ix < navLimit; ix++) { // ... navigating 1 to 4 pages...

customTestsList.add(findMethodInstance(methods, "nextPage"));

}

boolean shouldExit = Math.random() < 0.5 ? true : false;

if (shouldExit) { // ... then exiting and logging back in, with 50% probability

customTestsList.add(findMethodInstance(methods, "exit"));

customTestsList.add(findMethodInstance(methods, "reLogin"));

}

}

customTestsList.add(findMethodInstance(methods, "nextPage"));

customTestsList.add(findMethodInstance(methods, "exit"));

customTestsList.add(findMethodInstance(methods, "testTearDown"));

}

return customTestsList; 

}

/**

* Finds method instance object in a list, by test method name

* 

* @param methods list of methods

* @param name    test name to search for

* @return method instance object, or null, if nothing found

*/

private IMethodInstance findMethodInstance(List<IMethodInstance> methods, String name) {

for (IMethodInstance method : methods) {

if (method.getMethod().getMethodName().equals(name)) {

return method;

//return new MethodInstance(method.getMethod().clone()); // this one creates new classes for each method I guess - chrome sessions are lost

//return new MethodInstance(method.getMethod());

}

}

return null;

}

}

-----------------------------------------

All the best,

Alexey

[⇜Krishnan Mahadevan⇝]

If you need two or more "@Test" methods to be executed for every iteration of a data provider, then you can explore leveraging the "@Factory" annotation.

Take a look at the documentation here: https://testng.org/doc/documentation-main.html#factories 

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribblings @ https://rationaleemotions.com/

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to testng-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/testng-dev/e3afef88-52c3-433e-b56b-781ccd26e144%40googlegroups.com.

[Eric Liu]

I checked out the factory annotation in the docs, but it still does not execute sequentially. 

Here is a small example taken/modified from the docs

https://pastebin.com/gahzSYwu 

https://pastebin.com/nRGZM8T2 

To view this discussion on the web visit https://groups.google.com/d/msgid/testng-dev/CANikZLkbe-N%3DtPq2-7%2BJEuQzn1zkPLjA0m%3DpD40ckGSzO5iPxg%40mail.gmail.com.

[⇜Krishnan Mahadevan⇝]

You need to be creating a suite xml file wherein you need to set the attribute "group-by-instances" = true and try again.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribblings @ https://rationaleemotions.com/

To view this discussion on the web visit https://groups.google.com/d/msgid/testng-dev/CAK%3DR5kbOnj9pgmjAAmMbQtbD8pSNXY00c81yLf6M1b9bTC5D4A%40mail.gmail.com.

[Eric Liu]

Got it up and running by specifying new tests in the XML file. This seems clunky to me as you would then have to specify the inputs in the XML right? I figured there would be a way to automatically use dataprovider to run through 1,2,3 with different sets of data, since dataprovider already automatically handles changing the data, just not the sequencing. I just started learning testNG so I may not be thinking about this right though.

To view this discussion on the web visit https://groups.google.com/d/msgid/testng-dev/CANikZLmrvR8qKvpfB02hQRMTTXj_Vf%2BhuNVoyWchMJ01nM%2Byew%40mail.gmail.com.

[⇜Krishnan Mahadevan⇝]

Eric,

I think you got it wrong.

The 3 <test> tags in https://testng.org/doc/documentation-main.html#factories was basically aimed at demonstrating to you that you could pass in parameters to your test as well.

In your case, you dont need that. 

So your altered example would be as below:

Here's the test class.

import org.testng.annotations.Test;

public class WebTest {
  private int m_numberOfTimes;

  public WebTest(int numberOfTimes) {
    m_numberOfTimes = numberOfTimes;
  }

  @Test
  public void testServer() {
    for (int i = 0; i < m_numberOfTimes; i++) {
      // access the web page
    }
  }
}

Here's the factory class.

import org.testng.annotations.Factory;

public class WebTestFactory {
  @Factory
  public Object[] createInstances() {
    Object[] result = new Object[10];
    for (int i = 0; i < 10; i++) {
      result[i] = new WebTest(i * 10);
    }
    return result;
  }
}

Here's the suite xml file.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="My_Suite" group-by-instances="true" verbose="2">
  <test name="My_Test">
    <classes>
      <class name="com.rationaleemotions.googleforums.factories.WebTestFactory"/>
    </classes>
  </test>
</suite>

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribblings @ https://rationaleemotions.com/

To view this discussion on the web visit https://groups.google.com/d/msgid/testng-dev/CAK%3DR5kYOp6moVXRT5GoQ7UKm%3Dm5aEidFrmzn%3DAM%2B2h8Vm%2B6s6Q%40mail.gmail.com.