TestNG+Maven Surefire Plugin Multiple Methods Execution with Command Line --- Not Working

[ranganath chenna]

Hi All,

 Please help/guide me to solve the issue.

 I wanted to Execute testng Classes which are having one/multiple methods in each class. To do this I had tried following:

 Created a Sample Maven+TestNG Project with 2 Test Classes: [Project attached]

 As maven-surefireplugin[2.18] says : mvn -Dtest=Test1#Test1Method1 test....One Method is executed.

                                                        mvn -Dtest=Test1#Test1Method1+Test1Method2 test....Result is None of the tests were executed. 

 

 One More Query:

  I created a suite.xml and placed in our shared machine[Windows is Operating System] tried running the same suite.xml using maven command line in jenkins [Jenkins is hosted on linux/unix machine] with the following command : mvn test -DsuiteXmlFile=ipAddressOfServer/Folder/suite.xml...Execution is smooth and tests were executed successfully ===> This will be success if we dont specify surefire plugin inpor POM File.

But if we specify the surefire plugin in POM, try running the same in jenkins is failed....Dont know why ? Please help regarding.

Thanks,

Rangnath C.

[SUBRAMANYESWARA RAO BHAVIRISETTY]

Hi Rangnath,

  I am not sure if running multiple methods is supported. I am seeing the following in maven surefire plugin documentation.

As of Surefire 2.12.1, you can select multiple methods (JUnit 4.x only at this time; patches welcome!):

mvn -Dtest=TestCircle#testOne+testTwo test

I guess you can probably try using regex.

mvn -Dtest=TestCircle#test* test

Regarding the jenkins error related to suitexmlfiles, can you please attach the error log you are seeing on the console? This would help in debugging the issue easily.

~Subramanyam

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.
To post to this group, send email to testng...@googlegroups.com.
Visit this group at http://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--

Subramanyam

[ranganath chenna]

Hi,

 thanks for your reply...Please find the console logs of jenkins here.

Success : Without surefire plugin in POM

Failure : With Surefire Plugin in POM

Regards,

Ranganath.

--
You received this message because you are subscribed to a topic in the Google Groups "testng-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/testng-users/Gd3CTMr-3ag/unsubscribe.
To unsubscribe from this group and all its topics, send an email to testng-users...@googlegroups.com.

To post to this group, send email to testng...@googlegroups.com.
Visit this group at http://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--

Thanks,
Ranganath.

[SUBRAMANYESWARA RAO BHAVIRISETTY]

From the logs it's clear.

Dec 25, 2014 8:38:00 PM org.apache.maven.cli.event.ExecutionEventLogger mojoStarted
INFO: --- maven-surefire-plugin:2.10:test (default-test) @ mytestng ---
[INFO] Surefire report directory: /x/hudson/workspace/devJob/target/surefire-reports
org.apache.maven.surefire.booter.SurefireExecutionException: Suite file /x/hudson/workspace/devJob/ipAddress/Shared-Files/suite.xml is not a valid file; nested exception is org.apache.maven.surefire.testset.TestSetFailedException: Suite file /x/hudson/workspace/devJob/ipAddress/Shared-Files/suite.xml is not a valid file
org.apache.maven.surefire.testset.TestSetFailedException: Suite file /x/hudson/workspace/devJob/ipAddress/Shared-Files/suite.xml is not a valid file
    at org.apache.maven.surefire.testng.TestNGXmlTestSuite.locateTestSets(TestNGXmlTestSuite.java:131)
    at org.apache.maven.surefire.Surefire.createSuiteFromDefinition(Surefire.java:194)
    at org.apache.maven.surefire.Surefire.run(Surefire.java:150)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:350)
    at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1021)
[ERROR] There are test failures.

So basically it's not able to find your suite.xml. Can you try with below approach?

This is how I use it in my tests.

<plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.12</version>
                <configuration>
                    <suiteXmlFiles>
                        <suiteXmlFile>src/test/resources/testng.xml</suiteXmlFile>
                    </suiteXmlFiles>
                    <configuration>
                              <skipTests>${skipTests}</skipTests>
                    </configuration>

                    <properties>
                        <property>
                            <name>usedefaultlisteners</name>
                            <value>false</value>
                        </property>
                        <property>
                            <name>listener</name>
                            <value>org.uncommons.reportng.HTMLReporter,org.uncommons.reportng.JUnitXMLReporter</value>
                        </property>
                    </properties>
                </configuration>
            </plugin>
        </plugins>
        <testResources>
            <testResource>
                <directory>src/test/java/input</directory>
                <filtering>true</filtering>
            </testResource>
            <testResource>
                <directory>src/test/java/testcases</directory>
                <filtering>true</filtering>
            </testResource>
            <testResource>
                <directory>src/test/resources</directory>
                <filtering>true</filtering>
            </testResource>
            <testResource>
                <directory>src/test/java/config</directory>
                <filtering>true</filtering>
            </testResource>
            <testResource>
                <directory>src/test/java/test_suites</directory>
                <filtering>true</filtering>
            </testResource>
            <testResource>
                <directory>src/test/java</directory>
                <filtering>true</filtering>
            </testResource>
       </testResources>

Sample testng.xml file:

[bsrao@thosecharge tests]$ cat src/test/resources/testng.xml
<!DOCTYPE suite SYSTEM "http://beust.com/testng/testng-1.0.dtd" >

<suite name="Build Verification Test" configfailurepolicy="continue">
        <suite-files>
           <suite-file
                        path="src/test/java/testcases/storage_server/BVT_URL_parser_storage_server.xml" />
        </suite-files>
</suite>

So by default, maven-surefireplugin tries to look into testng.xml in src/test/resources.

Also please look into this post: https://groups.google.com/forum/#!topic/selenium-users/iP7Y-7Ph4tI which will be helpful.

[Krishnan Mahadevan]

Ranganath,

AFAIK, Maven by default relies on the surefire plugin to run your tests. The super pom in the maven world has this entry in it 

http://books.sonatype.com/mvnref-book/reference/pom-relationships-sect-pom.html#pom-relationships-sect-super-pom

You can confirm this yourself by opening up a pom file which doesnt have the surefire plugin explictily added, switch to dependency hierarchy view .

Coming back to your failures, can you please run the scenario wherein your tests are failing ? with the -X (stands for debug) flag enabled and share the logs ?

You say that you were able to run the tests from the Jenkins box directly via the commandline.

Does it mean that you are using your own instance of Jenkins setup or you are using something which is a common instance in your org [ I hope you are able to get what I am hinting at ]

Please feel free to share more context around your problem.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

[Krishnan Mahadevan]

To add to what I called out earlier, 

Here's what is going wrong in your case :

Your project has 1 or more .java classes which contain "Test" in its name. If the pom file doesnt contain any surefire plugin, as I called out earlier, the surefire plugin entry from the sure pom file kicks in. That entry is configured to apply a pattern matching which looks for the below in your project to identify test classes that can be run

includes2=**/*TestCase.java, includes1=**/*Test.java, includes0=**/Test*.java

This is what is causing your test classes to be executed when you dont have surefire plugin entry at all in your pom file, and NOT because your suite xml file from the network drive was accessible.

From your logs, I am guessing that your Jenkins is a Linux box. For you to be able to access a network drive, in the linux world, i think you would first need to mount the network drive before you can access it.

Currently what surefire plugin is doing here is, that it is trying to look for a relative directory with your network path as its name and trying to locate the suite file in it (which is NOT what you are looking for )

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/