Running TestNG programmatically
2,869 views
Skip to first unread message
prasanna bhat
Mar 29, 2010, 12:14:12 PM3/29/10
to testng...@googlegroups.com
Hi,
I have hit a road block when trying to run TestNG progrmmatically.
Earlier i had a testng.xml which i specifed in the pom.xml, and i had registered listeners and method-selectors. With this implementation the entry point used to be ITestMethodSelector.includeMethod
Now i wanted to replace this testng.xml with TestNG object and configuring my test run, from the documentation (http://testng.org/documentation-main.html#running-testng-programmatically)
i got to know how to do it. When i tried implementing the same the method that contained this TestNG object which configures the test run was not invoked first.
So how do i tell TestNG engine to invoke this method first that contains my TestNG object when running the tests. (btw i'm using maven, so please let me i can tweak into pom.xml)
Thanks,
Prasanna
I have hit a road block when trying to run TestNG progrmmatically.
Earlier i had a testng.xml which i specifed in the pom.xml, and i had registered listeners and method-selectors. With this implementation the entry point used to be ITestMethodSelector.includeMethod
Now i wanted to replace this testng.xml with TestNG object and configuring my test run, from the documentation (http://testng.org/documentation-main.html#running-testng-programmatically)
i got to know how to do it. When i tried implementing the same the method that contained this TestNG object which configures the test run was not invoked first.
So how do i tell TestNG engine to invoke this method first that contains my TestNG object when running the tests. (btw i'm using maven, so please let me i can tweak into pom.xml)
Thanks,
Prasanna
prasanna bhat
Mar 30, 2010, 2:18:53 AM3/30/10
to testng-users
Any pointers will be of great help.
Thanks,
Prasanna
On Mar 29, 9:14 pm, prasanna bhat <prasannabha...@gmail.com> wrote:
> Hi,
>
> I have hit a road block when trying to run TestNG progrmmatically.
>
> Earlier i had a testng.xml which i specifed in the pom.xml, and i had
> registered listeners and method-selectors. With this implementation the
> entry point used to be ITestMethodSelector.includeMethod
>
> Now i wanted to replace this testng.xml with TestNG object and configuring
> my test run, from the documentation (http://testng.org/documentation-main.html#running-testng-programmatic...)
Kartik Kumar
Mar 30, 2010, 2:33:32 AM3/30/10
to testng...@googlegroups.com
If you are running TestNG with Maven, you should take a look at this
http://maven.apache.org/plugins/maven-surefire-plugin/testng.html
http://maven.apache.org/plugins/maven-surefire-plugin/testng.html
--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to testng...@googlegroups.com.
To unsubscribe from this group, send email to testng-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.
prasanna bhat
Mar 30, 2010, 3:50:22 AM3/30/10
to testng...@googlegroups.com
Hi Karthik,
Thanks for a quick reply. I visited the site pointed by you, but couldn't find anything particular related to this issue.
To be more clear from my side, i don't want to use testng.xml in my test run and the entire configuration which was earlier accomplished using testng.xml has to be done in a test/configuration method through TestNG object using the api's provided.
Using the below testng.xml, the TestNG runs only the test methods present in TestDependencyExample class (which is the expected/correct behaviour)
<?xml version="1.0" encoding="UTF-8"?>
<suite name="TmpSuite" verbose="1" >
<test name="TmpTest">
<classes>
<class name="org.automation.bo.TestDependencyExample" />
</classes>
</test>
</suite>
However when i tried doing the same using TestNG Object, things are not working fine.
public class TestNGObjectExample {
@BeforeSuite(alwaysRun=true)
public static void setUpEnv(){
TestNG myTestNG=new TestNG();
List<XmlSuite> testSuites=new ArrayList<XmlSuite>();
XmlSuite suite=new XmlSuite();
suite.setName("TmpSuite");
XmlTest test=new XmlTest(suite);
test.setName("TmpTest");
List<XmlClass> testClasses=new ArrayList<XmlClass>();
XmlClass myClass=new XmlClass("org.automation.bo.TestDependencyExample");
testClasses.add(myClass);
test.setXmlClasses(testClasses);
testSuites.add(suite);
myTestNG.setXmlSuites(testSuites);
myTestNG.run();
}
}
I'm executing the test with mvn test command.
Output :
It executes the methods in TestDependencyExample as expected but the test run doesn't stop there, it continues to execute the test methods in other classes in the same package (org.automation.bo)
Please let me know if my approach is right or wrong.
Thanks,
-Prasanna
Thanks for a quick reply. I visited the site pointed by you, but couldn't find anything particular related to this issue.
To be more clear from my side, i don't want to use testng.xml in my test run and the entire configuration which was earlier accomplished using testng.xml has to be done in a test/configuration method through TestNG object using the api's provided.
Using the below testng.xml, the TestNG runs only the test methods present in TestDependencyExample class (which is the expected/correct behaviour)
<?xml version="1.0" encoding="UTF-8"?>
<suite name="TmpSuite" verbose="1" >
<test name="TmpTest">
<classes>
<class name="org.automation.bo.TestDependencyExample" />
</classes>
</test>
</suite>
However when i tried doing the same using TestNG Object, things are not working fine.
public class TestNGObjectExample {
@BeforeSuite(alwaysRun=true)
public static void setUpEnv(){
TestNG myTestNG=new TestNG();
List<XmlSuite> testSuites=new ArrayList<XmlSuite>();
XmlSuite suite=new XmlSuite();
suite.setName("TmpSuite");
XmlTest test=new XmlTest(suite);
test.setName("TmpTest");
List<XmlClass> testClasses=new ArrayList<XmlClass>();
XmlClass myClass=new XmlClass("org.automation.bo.TestDependencyExample");
testClasses.add(myClass);
test.setXmlClasses(testClasses);
testSuites.add(suite);
myTestNG.setXmlSuites(testSuites);
myTestNG.run();
}
}
I'm executing the test with mvn test command.
Output :
It executes the methods in TestDependencyExample as expected but the test run doesn't stop there, it continues to execute the test methods in other classes in the same package (org.automation.bo)
Please let me know if my approach is right or wrong.
Thanks,
-Prasanna
prasanna bhat
Mar 30, 2010, 6:26:53 AM3/30/10
to testng...@googlegroups.com
Hi,
I tried an alternate approach.
package org.automation.bo;
public class TestNGObjectExample {
@BeforeSuite(alwaysRun=true)
public static void setUpEnv(){
TestNG myTestNG=new TestNG();
TestListenerExample tle=new TestListenerExample();
myTestNG.addListener(tle);
myTestNG.addMethodSelector("deshaw.automation.bo.TestNGMethodSelector", 2);
myTestNG.setTestClasses( new Class[] {
TestDependencyExample.class,
TestLazyDataProviderExample.class,
TestNGExample.class,
});
myTestNG.run();
}
}
/*TestNGMethodSelecor class */
package org.automation.bo;
public class TestNGMethodSelector implements IMethodSelector{
public boolean includeMethod(IMethodSelectorContext context,
ITestNGMethod method, boolean isTestMethod) {
boolean isIncluded=false;
if(!isTestMethod)
isIncluded=true;
else{
/*method present in TestDependencyExample.class */
if(method.getMethod().getName().equals("dependency"))
isIncluded=true;
/*method present in TestDependencyExample.class */
else if(method.getMethod().getName().equals("testDependsMethod"))
isIncluded=true;
/*method present in TestNGExample.class */
else if(method.getMethod().getName().equals("testMethod1"))
isIncluded=true;
else
isIncluded=false;
}
context.setStopped(true);
return isIncluded;
}
public void setTestMethods(List<ITestNGMethod> testMethods) {
}
}
In this approach also, TestNG engine first runs the methods selected in the method selector. After executing these test methods, TestNG again runs all the tests present in the classes in org.automation.bo package.
For more clarity i'm providing the output also:
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running TestSuite
Entry point
[Parser] Running:
Command line suite
/*
* these are the methods selected in the method -selectors
*
Executing method dependency
Executing method testDependsMethod
Executing test method 1
-------------------------------------------
===============================================
Command line suite
Total tests run: 3, Failures: 0, Skips: 0
===============================================
/*
* TestNG runs the entire test suite again
*/
[XmlMethodSelector] CLASSNAME:deshaw.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
[XmlMethodSelector] CLASSNAME:deshaw.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
[XmlMethodSelector] CLASSNAME:deshaw.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
[XmlMethodSelector] CLASSNAME:deshaw.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
[XmlMethodSelector] CLASSNAME:deshaw.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
Executing method dependency
Executing method testDependsMethod
Executing testIterator example
Executing test method 2
Executing test method 1
Executing test method 3
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.891 sec
Any inputs on how to resolve this issue or any alternatives will be much appreciated.
Thanks,
Prasanna
I tried an alternate approach.
package org.automation.bo;
public class TestNGObjectExample {
@BeforeSuite(alwaysRun=true)
public static void setUpEnv(){
TestNG myTestNG=new TestNG();
myTestNG.addListener(tle);
myTestNG.addMethodSelector("deshaw.automation.bo.TestNGMethodSelector", 2);
myTestNG.setTestClasses( new Class[] {
TestDependencyExample.class,
TestLazyDataProviderExample.class,
TestNGExample.class,
});
myTestNG.run();
}
}
/*TestNGMethodSelecor class */
package org.automation.bo;
public class TestNGMethodSelector implements IMethodSelector{
public boolean includeMethod(IMethodSelectorContext context,
ITestNGMethod method, boolean isTestMethod) {
boolean isIncluded=false;
if(!isTestMethod)
isIncluded=true;
else{
/*method present in TestDependencyExample.class */
if(method.getMethod().getName().equals("dependency"))
isIncluded=true;
/*method present in TestDependencyExample.class */
else if(method.getMethod().getName().equals("testDependsMethod"))
isIncluded=true;
/*method present in TestNGExample.class */
else if(method.getMethod().getName().equals("testMethod1"))
isIncluded=true;
else
isIncluded=false;
}
context.setStopped(true);
return isIncluded;
}
public void setTestMethods(List<ITestNGMethod> testMethods) {
}
}
In this approach also, TestNG engine first runs the methods selected in the method selector. After executing these test methods, TestNG again runs all the tests present in the classes in org.automation.bo package.
For more clarity i'm providing the output also:
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running TestSuite
Entry point
[Parser] Running:
Command line suite
/*
* these are the methods selected in the method -selectors
*
Executing method dependency
Executing method testDependsMethod
Executing test method 1
-------------------------------------------
===============================================
Command line suite
Total tests run: 3, Failures: 0, Skips: 0
===============================================
/*
* TestNG runs the entire test suite again
*/
[XmlMethodSelector] CLASSNAME:deshaw.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
[XmlMethodSelector] CLASSNAME:deshaw.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
[XmlMethodSelector] CLASSNAME:deshaw.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
[XmlMethodSelector] CLASSNAME:deshaw.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
[XmlMethodSelector] CLASSNAME:deshaw.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
Executing method dependency
Executing method testDependsMethod
Executing testIterator example
Executing test method 2
Executing test method 1
Executing test method 3
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.891 sec
Any inputs on how to resolve this issue or any alternatives will be much appreciated.
Thanks,
Prasanna
Cédric Beust ♔
Mar 30, 2010, 8:18:42 AM3/30/10
to testng...@googlegroups.com
Hi Prasanna,
Your code looks correct, it should definitely produce the same result as your testng.xml file.
I notice that you are running inside TestNG itself, so in effect, you have one TestNG running inside another TestNG. This is absolutely fine (that's how I write all the TestNG tests) but maybe this is causing some confusion.
Can you try to run your code below "standalone" (e.g. as a main(String[]) method) and see if you get the same result? If you do, run it in verbose mode (myTestNG.setVerbose(10)) and include the output so we can see what TestNG is doing exactly...
Thanks!
--
Cédric
Cédric
prasanna bhat
Mar 30, 2010, 9:43:09 AM3/30/10
to testng...@googlegroups.com
>>Can you try to run
your code below "standalone" (e.g. as a main(String[]) method) and see
if you get the same result? If you do, run it in verbose mode
(myTestNG.setVerbose(10)) and include the output so we can see what
TestNG is doing exactly...
I have been using maven to run the tests. I found that to run it from command-line i must specify atleast one testng.xml file. so do you want me to run it by creating a testng.xml and then provide it as a parameter while running test suite from command line?
please let me know if my interpretation of standalone execution was wrong?
Thanks,
Prasanna
2010/3/30 Cédric Beust ♔ <cbe...@google.com>
Cédric Beust ♔
Mar 30, 2010, 10:31:20 AM3/30/10
to testng...@googlegroups.com
Just put your code in a main() method and run that class with `java`.
--
Cédric
prasanna bhat
Mar 30, 2010, 11:38:26 AM3/30/10
to testng...@googlegroups.com
Thanks Cedric for the clarification.
I executed the code, dumping the output below:
Entry point
[Parser] Running:
Command line suite
[RunInfo] Adding method selector: org.testng.internal.XmlMethodSelector@b61fd1 priority: 10
[RunInfo] Adding method selector: org.automation.bo.TestNGMethodSelector@c8f6f8 priority: 2
[RunInfo] Adding method selector: org.automation.bo.TestNGMethodSelector@1ce2dd4 priority: 2
[RunInfo] Adding method selector: org.automation.bo.TestNGMethodSelector@122cdb6 priority: 2
[TestClass] Creating TestClass for [ClassImpl org.automation.bo.TestLazyDataProviderExample]
[TestClass] Adding method org.automation.bo.TestLazyDataProviderExample.testIterator(org.automation.bo.ExampleClass) on TestClass class org.automation.bo.TestLazyData
ProviderExample
[TestClass] Creating TestClass for [ClassImpl org.automation.bo.TestDependencyExample]
[TestClass] Adding method org.automation.bo.TestDependencyExample.dependency() on TestClass class org.automation.bo.TestDependencyExample
[TestClass] Adding method org.automation.bo.TestDependencyExample.testDependsMethod() on TestClass class org.automation.bo.TestDependencyExample
[TestClass] Creating TestClass for [ClassImpl org.automation.bo.TestNGExample]
Method public static java.lang.Object[][] org.automation.bo.TestNGExample.getData() has a @Test annotation but also a return value: ignoring it.
[TestClass] Adding method org.automation.bo.TestNGExample.testMethod3(org.testng.ITestContext) on TestClass class org.automation.bo.TestNGExample
[TestClass] Adding method org.automation.bo.TestNGExample.testDataProvider(java.lang.String, java.lang.String) on TestClass class org.automation.bo.TestNGExample
[TestClass] Adding method org.automation.bo.TestNGExample.testMethod1(org.testng.ITestContext) on TestClass class org.automation.bo.TestNGExample
[TestClass] Adding method org.automation.bo.TestNGExample.testMethod2(org.testng.ITestContext) on TestClass class org.automation.bo.TestNGExample
[SuiteRunner] Created 1 TestRunners
[TestRunner] Running test Command line test on 3 classes, included groups:[] excluded groups:[]
[TestClass]
======
TESTCLASS: org.automation.bo.TestLazyDataProviderExample
[TestClass] Test : org.automation.bo.TestLazyDataProviderExample.testIterator(org.automation.bo.ExampleClass)
[TestClass]
======
[TestClass]
======
TESTCLASS: org.automation.bo.TestDependencyExample
[TestClass] BeforeClass : org.automation.bo.TestDependencyExample.setUp()
[TestClass] Test : org.automation.bo.TestDependencyExample.dependency()
[TestClass] Test : org.automation.bo.TestDependencyExample.testDependsMethod()
[TestClass] AfterClass : org.automation.bo.TestDependencyExample.tearDown()
[TestClass]
======
[TestClass]
======
TESTCLASS: org.automation.bo.TestNGExample
[TestClass] BeforeClass : org.automation.bo.TestNGExample.setUpClass()
[TestClass] BeforeMethod: org.automation.bo.TestNGExample.setUpData(java.lang.reflect.Method)
[TestClass] Test : org.automation.bo.TestNGExample.testMethod3(org.testng.ITestContext)
[TestClass] Test : org.automation.bo.TestNGExample.testDataProvider(java.lang.String, java.lang.String)
[TestClass] Test : org.automation.bo.TestNGExample.testMethod1(org.testng.ITestContext)
[TestClass] Test : org.automation.bo.TestNGExample.testMethod2(org.testng.ITestContext)
[TestClass] AfterClass : org.automation.bo.TestNGExample.tearDown()
[TestClass]
======
[TestRunner] Found 2 applicable methods
[TestRunner] WILL BE RUN SEQUENTIALLY:
[TestRunner] org.automation.bo.TestDependencyExample.dependency()
[TestRunner] org.automation.bo.TestDependencyExample.testDependsMethod()
[TestRunner] ====
[TestRunner] ===
[TestRunner] WILL BE RUN SEQUENTIALLY:
0 <-- [Worker thread:1 org.automation.bo.TestDependencyExample.dependency()]
[TestRunner] WILL BE RUN IN RANDOM ORDER:
[TestRunner] org.automation.bo.TestNGExample.testMethod1(org.testng.ITestContext)
[TestRunner] on instances
[TestRunner] org.automation.bo.TestNGExample@1db7df8
[TestRunner] ===
[Invoker 2208288] Keeping method org.automation.bo.TestDependencyExample.setUp() for class [TestClass class org.automation.bo.TestDependencyExample]
[Invoker 2208288] Invoking @BeforeClass org.automation.bo.TestDependencyExample.setUp()
[Invoker 2208288] Invoking org.automation.bo.TestDependencyExample.dependency
Executing method dependency
[Invoker 2208288] Invoking org.automation.bo.TestDependencyExample.testDependsMethod
Executing method testDependsMethod
[Invoker 2208288] Keeping method org.automation.bo.TestDependencyExample.tearDown() for class [TestClass class org.automation.bo.TestDependencyExample]
[Invoker 2208288] Invoking @AfterClass org.automation.bo.TestDependencyExample.tearDown()
[Invoker 2208288] Keeping method org.automation.bo.TestNGExample.setUpClass() for class [TestClass class org.automation.bo.TestNGExample]
[Invoker 2208288] Invoking @BeforeClass org.automation.bo.TestNGExample.setUpClass()
[Invoker 2208288] Keeping method org.automation.bo.TestNGExample.setUpData(java.lang.reflect.Method) for class [TestClass class org.automation.bo.TestNGExample]
[Invoker 2208288] Keeping method org.automation.bo.TestNGExample.setUpData(java.lang.reflect.Method) for class [TestClass class org.automation.bo.TestNGExample]
[Invoker 2208288] Invoking @BeforeMethod org.automation.bo.TestNGExample.setUpData(java.lang.reflect.Method)
[Invoker 2208288] Invoking org.automation.bo.TestNGExample.testMethod1
Executing test method 1
[Invoker 2208288] Keeping method org.automation.bo.TestNGExample.tearDown() for class [TestClass class org.automation.bo.TestNGExample]
[Invoker 2208288] Invoking @AfterClass org.automation.bo.TestNGExample.tearDown()
*********** INVOKED METHODS
org.automation.bo.TestDependencyExample.setUp() 29839159
org.automation.bo.TestDependencyExample.dependency() 29839159
org.automation.bo.TestDependencyExample.testDependsMethod() 29839159
org.automation.bo.TestDependencyExample.tearDown() 29839159
org.automation.bo.TestNGExample.setUpClass() 31161848
org.automation.bo.TestNGExample.testMethod1(org.testng.ITestContext)org.testng.TestRunner@a4e2e3 31161848
org.automation.bo.TestNGExample.tearDown() 31161848
***********
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\Command line test.html
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\Command line test.xml
PASSED: dependency
PASSED: testDependsMethod
PASSED: testMethod1(org.testng.TestRunner@a4e2e3)
===============================================
Command line test
Tests run: 3, Failures: 0, Skips: 0
===============================================
===============================================
Command line suite
Total tests run: 3, Failures: 0, Skips: 0
===============================================
[org.testng.internal.PoolService] Shutting down poolservice org.testng.internal.PoolService@7bb290 terminated:false
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\toc.html
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\Command line test.properties
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\index.html
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\main.html
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\groups.html
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\methods.html
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\methods-alphabetical.html
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\classes.html
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\reporter-output.html
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\methods-not-run.html
[XmlMethodSelector] CLASSNAME:org.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
[XmlMethodSelector] CLASSNAME:org.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
[XmlMethodSelector] CLASSNAME:org.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\testng.xml.html
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\index.html
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\testng-results.xml
it seems like TestNG is running only the configured/selected methods. But the discrepancy was observed when run with Maven.
I executed the code, dumping the output below:
Entry point
[Parser] Running:
Command line suite
[RunInfo] Adding method selector: org.automation.bo.TestNGMethodSelector@c8f6f8 priority: 2
[RunInfo] Adding method selector: org.automation.bo.TestNGMethodSelector@1ce2dd4 priority: 2
[RunInfo] Adding method selector: org.automation.bo.TestNGMethodSelector@122cdb6 priority: 2
[TestClass] Creating TestClass for [ClassImpl org.automation.bo.TestLazyDataProviderExample]
[TestClass] Adding method org.automation.bo.TestLazyDataProviderExample.testIterator(org.automation.bo.ExampleClass) on TestClass class org.automation.bo.TestLazyData
ProviderExample
[TestClass] Creating TestClass for [ClassImpl org.automation.bo.TestDependencyExample]
[TestClass] Adding method org.automation.bo.TestDependencyExample.dependency() on TestClass class org.automation.bo.TestDependencyExample
[TestClass] Adding method org.automation.bo.TestDependencyExample.testDependsMethod() on TestClass class org.automation.bo.TestDependencyExample
[TestClass] Creating TestClass for [ClassImpl org.automation.bo.TestNGExample]
Method public static java.lang.Object[][] org.automation.bo.TestNGExample.getData() has a @Test annotation but also a return value: ignoring it.
[TestClass] Adding method org.automation.bo.TestNGExample.testMethod3(org.testng.ITestContext) on TestClass class org.automation.bo.TestNGExample
[TestClass] Adding method org.automation.bo.TestNGExample.testDataProvider(java.lang.String, java.lang.String) on TestClass class org.automation.bo.TestNGExample
[TestClass] Adding method org.automation.bo.TestNGExample.testMethod1(org.testng.ITestContext) on TestClass class org.automation.bo.TestNGExample
[TestClass] Adding method org.automation.bo.TestNGExample.testMethod2(org.testng.ITestContext) on TestClass class org.automation.bo.TestNGExample
[SuiteRunner] Created 1 TestRunners
[TestRunner] Running test Command line test on 3 classes, included groups:[] excluded groups:[]
[TestClass]
======
TESTCLASS: org.automation.bo.TestLazyDataProviderExample
[TestClass] Test : org.automation.bo.TestLazyDataProviderExample.testIterator(org.automation.bo.ExampleClass)
[TestClass]
======
[TestClass]
======
TESTCLASS: org.automation.bo.TestDependencyExample
[TestClass] BeforeClass : org.automation.bo.TestDependencyExample.setUp()
[TestClass] Test : org.automation.bo.TestDependencyExample.dependency()
[TestClass] Test : org.automation.bo.TestDependencyExample.testDependsMethod()
[TestClass] AfterClass : org.automation.bo.TestDependencyExample.tearDown()
[TestClass]
======
[TestClass]
======
TESTCLASS: org.automation.bo.TestNGExample
[TestClass] BeforeClass : org.automation.bo.TestNGExample.setUpClass()
[TestClass] BeforeMethod: org.automation.bo.TestNGExample.setUpData(java.lang.reflect.Method)
[TestClass] Test : org.automation.bo.TestNGExample.testMethod3(org.testng.ITestContext)
[TestClass] Test : org.automation.bo.TestNGExample.testDataProvider(java.lang.String, java.lang.String)
[TestClass] Test : org.automation.bo.TestNGExample.testMethod1(org.testng.ITestContext)
[TestClass] Test : org.automation.bo.TestNGExample.testMethod2(org.testng.ITestContext)
[TestClass] AfterClass : org.automation.bo.TestNGExample.tearDown()
[TestClass]
======
[TestRunner] Found 2 applicable methods
[TestRunner] WILL BE RUN SEQUENTIALLY:
[TestRunner] org.automation.bo.TestDependencyExample.dependency()
[TestRunner] org.automation.bo.TestDependencyExample.testDependsMethod()
[TestRunner] ====
[TestRunner] ===
[TestRunner] WILL BE RUN SEQUENTIALLY:
0 <-- [Worker thread:1 org.automation.bo.TestDependencyExample.dependency()]
[TestRunner] WILL BE RUN IN RANDOM ORDER:
[TestRunner] org.automation.bo.TestNGExample.testMethod1(org.testng.ITestContext)
[TestRunner] on instances
[TestRunner] org.automation.bo.TestNGExample@1db7df8
[TestRunner] ===
[Invoker 2208288] Keeping method org.automation.bo.TestDependencyExample.setUp() for class [TestClass class org.automation.bo.TestDependencyExample]
[Invoker 2208288] Invoking @BeforeClass org.automation.bo.TestDependencyExample.setUp()
[Invoker 2208288] Invoking org.automation.bo.TestDependencyExample.dependency
Executing method dependency
[Invoker 2208288] Invoking org.automation.bo.TestDependencyExample.testDependsMethod
Executing method testDependsMethod
[Invoker 2208288] Keeping method org.automation.bo.TestDependencyExample.tearDown() for class [TestClass class org.automation.bo.TestDependencyExample]
[Invoker 2208288] Invoking @AfterClass org.automation.bo.TestDependencyExample.tearDown()
[Invoker 2208288] Keeping method org.automation.bo.TestNGExample.setUpClass() for class [TestClass class org.automation.bo.TestNGExample]
[Invoker 2208288] Invoking @BeforeClass org.automation.bo.TestNGExample.setUpClass()
[Invoker 2208288] Keeping method org.automation.bo.TestNGExample.setUpData(java.lang.reflect.Method) for class [TestClass class org.automation.bo.TestNGExample]
[Invoker 2208288] Keeping method org.automation.bo.TestNGExample.setUpData(java.lang.reflect.Method) for class [TestClass class org.automation.bo.TestNGExample]
[Invoker 2208288] Invoking @BeforeMethod org.automation.bo.TestNGExample.setUpData(java.lang.reflect.Method)
[Invoker 2208288] Invoking org.automation.bo.TestNGExample.testMethod1
Executing test method 1
[Invoker 2208288] Keeping method org.automation.bo.TestNGExample.tearDown() for class [TestClass class org.automation.bo.TestNGExample]
[Invoker 2208288] Invoking @AfterClass org.automation.bo.TestNGExample.tearDown()
*********** INVOKED METHODS
org.automation.bo.TestDependencyExample.setUp() 29839159
org.automation.bo.TestDependencyExample.dependency() 29839159
org.automation.bo.TestDependencyExample.testDependsMethod() 29839159
org.automation.bo.TestDependencyExample.tearDown() 29839159
org.automation.bo.TestNGExample.setUpClass() 31161848
org.automation.bo.TestNGExample.testMethod1(org.testng.ITestContext)org.testng.TestRunner@a4e2e3 31161848
org.automation.bo.TestNGExample.tearDown() 31161848
***********
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\Command line test.html
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\Command line test.xml
PASSED: dependency
PASSED: testDependsMethod
PASSED: testMethod1(org.testng.TestRunner@a4e2e3)
===============================================
Command line test
Tests run: 3, Failures: 0, Skips: 0
===============================================
===============================================
Command line suite
Total tests run: 3, Failures: 0, Skips: 0
===============================================
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\toc.html
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\Command line test.properties
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\index.html
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\main.html
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\groups.html
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\methods.html
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\methods-alphabetical.html
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\classes.html
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\reporter-output.html
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\methods-not-run.html
[XmlMethodSelector] CLASSNAME:org.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
[XmlMethodSelector] CLASSNAME:org.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
[XmlMethodSelector] CLASSNAME:org.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\Command line suite\testng.xml.html
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\index.html
Creating H:\My_WorkSpace\TestNG_Maven\target\test-classes\test-output\testng-results.xml
it seems like TestNG is running only the configured/selected methods. But the discrepancy was observed when run with Maven.
Cédric Beust ♔
Mar 30, 2010, 11:41:17 AM3/30/10
to testng...@googlegroups.com
So you're saying that running from the command line works but running from Maven executes these extra test classes that you never mentioned anywhere?
This would point in the direction of a bug in the Surefire plug-in...
--
Cédric
prasanna bhat
Mar 30, 2010, 12:01:33 PM3/30/10
to testng...@googlegroups.com
I executed the same code (please see update 5) using mvn test command, dumping the output here:
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Building Unnamed - deshaw.automation:TestNG_Maven:jar:0.0.1-SNAPSHOT
[INFO] task-segment: [test]
[INFO] ------------------------------------------------------------------------
[INFO] [resources:resources {execution: default-resources}]
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources, i.e. buil
[INFO] Copying 0 resource
[INFO] [compiler:compile {execution: default-compile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [resources:testResources {execution: default-testResources}]
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources, i.e. buil
[INFO] Copying 1 resource
[INFO] [compiler:testCompile {execution: default-testCompile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [surefire:test {execution: default-test}]
[INFO] Surefire report directory: H:\My_WorkSpace\TestNG_Maven\target\surefire-reports
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running TestSuite
* methods that were selected by method-selectors which was configured using TestNG object
*/
* Test methods executed again
*/
[XmlMethodSelector] CLASSNAME:deshaw.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
[XmlMethodSelector] CLASSNAME:deshaw.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
[XmlMethodSelector] CLASSNAME:deshaw.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
[XmlMethodSelector] CLASSNAME:deshaw.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
[XmlMethodSelector] CLASSNAME:deshaw.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
Executing method dependency
Executing method testDependsMethod
Executing testIterator example
Executing test method 2
Executing test method 3
Executing test method 1
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.891 sec
Results :
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3 seconds
[INFO] Finished at: Tue Mar 30 21:16:44 IST 2010
[INFO] Final Memory: 7M/13M
Please note: i have tried following two approaches and the output i same as above when executed from maven.
/*
* TestNG object placed in @Before Suite
*/
myTestNG.addListener(tle);
myTestNG.addMethodSelector("org.automation.bo.TestNGMethodSelector", 2);
TestNG myTestNG=new TestNG();
TestListenerExample tle=new TestListenerExample();
myTestNG.addListener(tle);
>>So you're saying that running from the command line works but running from Maven executes these extra test classes that you never mentioned anywhere?
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Building Unnamed - deshaw.automation:TestNG_Maven:jar:0.0.1-SNAPSHOT
[INFO] task-segment: [test]
[INFO] ------------------------------------------------------------------------
[INFO] [resources:resources {execution: default-resources}]
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources, i.e. buil
[INFO] Copying 0 resource
[INFO] [compiler:compile {execution: default-compile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [resources:testResources {execution: default-testResources}]
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources, i.e. buil
[INFO] Copying 1 resource
[INFO] [compiler:testCompile {execution: default-testCompile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [surefire:test {execution: default-test}]
[INFO] Surefire report directory: H:\My_WorkSpace\TestNG_Maven\target\surefire-reports
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running TestSuite
Entry point
[Parser] Running:
Command line suite
/*[Parser] Running:
Command line suite
* methods that were selected by method-selectors which was configured using TestNG object
*/
Executing method dependency
Executing method testDependsMethod
Executing test method 1
Executing method testDependsMethod
Executing test method 1
===============================================
Command line suite
Total tests run: 3, Failures: 0, Skips: 0
===============================================
/*Command line suite
Total tests run: 3, Failures: 0, Skips: 0
===============================================
* Test methods executed again
*/
[XmlMethodSelector] CLASSNAME:deshaw.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
[XmlMethodSelector] CLASSNAME:deshaw.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
[XmlMethodSelector] CLASSNAME:deshaw.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
[XmlMethodSelector] CLASSNAME:deshaw.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
[XmlMethodSelector] CLASSNAME:deshaw.automation.bo.TestNGMethodSelector
[XmlMethodSelector] SETTING PRIORITY:2
Executing method dependency
Executing method testDependsMethod
Executing testIterator example
Executing test method 2
Executing test method 1
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.891 sec
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3 seconds
[INFO] Finished at: Tue Mar 30 21:16:44 IST 2010
[INFO] Final Memory: 7M/13M
Please note: i have tried following two approaches and the output i same as above when executed from maven.
/*
* TestNG object placed in @Before Suite
*/
public class TestNGObjectExample {
@BeforeSuite(alwaysRun=true)
public static void setUpEnv(){
TestNG myTestNG=new TestNG();
TestListenerExample tle=new TestListenerExample();@BeforeSuite(alwaysRun=true)
public static void setUpEnv(){
TestNG myTestNG=new TestNG();
myTestNG.addListener(tle);
myTestNG.addMethodSelector("org.automation.bo.TestNGMethodSelector", 2);
myTestNG.setTestClasses( new Class[] {
TestDependencyExample.class,
TestLazyDataProviderExample.class,
TestNGExample.class,
});
myTestNG.run();
}
}
/*
TestDependencyExample.class,
TestLazyDataProviderExample.class,
TestNGExample.class,
});
myTestNG.run();
}
}
/*
* TestNG object placed in main method
*/
public class TestNGObjectExample {
public static void main(){
*/
public class TestNGObjectExample {
public static void main(){
TestNG myTestNG=new TestNG();
TestListenerExample tle=new TestListenerExample();
myTestNG.addListener(tle);
myTestNG.addMethodSelector("org.automation.bo.TestNGMethodSelector", 2);
myTestNG.setTestClasses( new Class[] {
TestDependencyExample.class,
TestLazyDataProviderExample.class,
TestNGExample.class,
});
myTestNG.run();
}
}
TestDependencyExample.class,
TestLazyDataProviderExample.class,
TestNGExample.class,
});
myTestNG.run();
}
}
Maven POM.xml for my project:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.automation</groupId>
<artifactId>TestNG_Maven</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<dependencies>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>5.8</version>
<scope>test</scope>
<classifier>jdk15</classifier>
</dependency>
</dependencies>
<build>
<defaultGoal>install</defaultGoal>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.4.2</version>
<configuration>
</configuration>
</plugin>
</plugins>
</build>
</project>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.automation</groupId>
<artifactId>TestNG_Maven</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<dependencies>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>5.8</version>
<scope>test</scope>
<classifier>jdk15</classifier>
</dependency>
</dependencies>
<build>
<defaultGoal>install</defaultGoal>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.4.2</version>
<configuration>
</configuration>
</plugin>
</plugins>
</build>
</project>
>>So you're saying that running from the command line works but running from Maven executes these extra test classes that you never mentioned anywhere?
This would point in the direction of a bug in the Surefire plug-in...
I would request you to reproduce the same and then arrive at a conclusion.
Thanks for all your valuable inputs.
Regards
- PrasannaThanks for all your valuable inputs.
Regards
prasanna bhat
Mar 31, 2010, 2:24:09 AM3/31/10
to testng...@googlegroups.com
Hi Cedric,
Please let me know your observation.
Thanks for your time.
-Prasanna
Please let me know your observation.
Thanks for your time.
-Prasanna
Reply all
Reply to author
Forward
0 new messages