Parallel execution not working
thirsch...@gmail.com
I have used TestNG in the past, mostly through Gradle.
Now I took over a legacy project, which runs TestNG programmatically.
This is the part where TestNG gets configured:
private TestNG getConfiguredTestNG(final Parameters parameters) throws IOException, URISyntaxException {
TestNG testNg = new TestNG();
// Implement our quarantine mechanism.
testNg.setAnnotationTransformer(new AnnotationTransformer());
// prevents skipping of test cases if @Before-/@AfterClass methods throwing exception
testNg.setConfigFailurePolicy(XmlSuite.CONTINUE);
if (parameters.getParallelMode() != null) {
configureParallelism(testNg, parameters);
} else {
// tests are grouped and executed by test class instance
testNg.setGroupByInstances(true);
}
// Set test classes
File[] testSuite = getTestSuite(parameters);
Class<?>[] testClasses = getTestClasses(parameters.getTestClasses(), testSuite);
testNg.setTestClasses(testClasses);
// Set directory for TestNG output
testNg.setOutputDirectory(getTestNgOutputDir());
// The list of test groups the user wants to start.
final String testGroups = parameters.getTestGroups();
// The list of test groups the user wants to exclude.
final String excludeGroups = parameters.getExcludedGroups();
// By default all test groups are about to be started, if the user did not request specific groups.
if (testGroups == null) {
// Exclude those test groups which should not run by default as per specification.
final String excludedGroups = getExcludedGroups(excludeGroups);
testNg.setExcludedGroups(excludedGroups);
} else {
// Run only selected test groups.
testNg.setGroups(testGroups);
// Exclude those test groups which should not run by default as per specification.
final String excludedGroups = getExcludedGroups(testGroups, excludeGroups);
testNg.setExcludedGroups(excludedGroups);
}
return testNg;
}
with:
private TestNG configureParallelism(TestNG testNg, Parameters parameters) {
setParallelMode(testNg, parameters);
setThreadCount(testNg, parameters);
return testNg;
}
private TestNG setParallelMode(TestNG testNg, EsfParameters esfParameters) {
String mode = esfParameters.getParallelMode();
switch(mode) {
case "methods":
System.out.println("OK");
testNg.setParallel(ParallelMode.METHODS);
break;
case "tests":
testNg.setParallel(ParallelMode.TESTS);
break;
case "classes":
testNg.setParallel(ParallelMode.CLASSES);
break;
case "instances":
testNg.setParallel(ParallelMode.INSTANCES);
break;
default:
throw new RuntimeException(String.format("Parallelism setting '%s' not recognized, see TestNG docs.", mode));
}
return testNg;
}
private TestNG setThreadCount(TestNG testNg, EsfParameters esfParameters) {
Integer count = esfParameters.getThreadCount();
if (count != null && count > 0) {
testNg.setThreadCount(count);
}
return testNg;
}
The second bit I added to configure parallelism.
But when this code is run, TestNG still only runs a single test at a time, no matter how many test classes get passed to it.
I also verified that m_useThreadCount and related variables have the expected values before run() gets called
How to debug this? What can I do to get more insight about why TestNG only runs one method at a time?
Krishnan Mahadevan
How are you ascertaining that your tests are not running in parallel ? Have you tried printing the thread id to confirm this ?
Here’s a sample :
import org.testng.ITestResult;
import org.testng.Reporter;
import org.testng.annotations.Test;
public class TestclassSample {
@Test
public void a() {
prettyPrint();
}
@Test
public void b() {
prettyPrint();
}
@Test
public void c() {
prettyPrint();
}
@Test
public void d() {
prettyPrint();
}
private static void prettyPrint() {
ITestResult result = Reporter.getCurrentTestResult();
String msg = "Running " + result.getMethod().getMethodName() + "() running on [" +
Thread.currentThread().getId() + "]";
System.err.println(msg);
}
}
import org.testng.TestNG;
import org.testng.annotations.Test;
import org.testng.xml.XmlSuite;
public class TestclassTestRunner {
@Test
public void testMethod() {
TestNG testng = new TestNG();
testng.setTestClasses(new Class[]{TestclassSample.class});
testng.setThreadCount(2);
testng.setParallel(XmlSuite.ParallelMode.METHODS);
testng.setVerbose(2);
testng.run();
}
}
Output:
Running a() running on [11]
Running b() running on [12]
Running c() running on [11]
Running d() running on [12]
PASSED: a
PASSED: b
PASSED: c
PASSED: d
===============================================
Command line test
Tests run: 4, Failures: 0, Skips: 0
===============================================
===============================================
Command line suite
Total tests run: 4, Failures: 0, Skips: 0
===============================================
===============================================
Default Suite
Total tests run: 1, Failures: 0, Skips: 0
===============================================
Process finished with exit code 0
Thanks & Regards
Krishnan Mahadevan
"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/
My Technical Scribbings @ http://rationaleemotions.wordpress.com/
--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.
To post to this group, send email to testng...@googlegroups.com.
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.
thirsch...@gmail.com
Is there any way I can see standard out or standard err in the TestNG output?
thirsch...@gmail.com
I can see now, that multiple threads are indeed started, when I run your Example test.
The problem lies in the way TestNG was implemented in this context.
There is some code somewhere that prevents normal Tests from running in parallel, it seems, however it does that...
At least I know now that TestNG is not broken!
Thanks and regards,
Thomas