dependsOnGroups question...

[Jeff]

Using Maven surefire 2.20.1 and TestNG 6.14.2

I have a Maven project with test classes all defined in the same test package.  Most of the test methods have the following similar class-level annotation:

@Test(

        suiteName = "Rating Server Acceptance Tests",

        testName = "Application Data",

        groups = {"rating-server", "functional"},

        threadPoolSize = 20

)

Note the "groups" have both the "rating-server" group shared by all test classes in this module, and "functional" shared by all but one of the test classes.

The other test class must be run by itself and am trying to do this with group dependencies.  I added this class-level @Test annotation to this class:

@Test(

        suiteName = "Rating Server Acceptance Tests",

        testName = "Destructive Tests (run in isolation)",

        groups = {"rating-server", "destructive"},

        dependsOnGroups = { "functional" },  //Remove to test alone.

        singleThreaded = true

)

After all other "functional" tests run, I wanted this to run on it's own.  When I do a simple 'mvn test' run, I get this error:

Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.20.1:test (default-test) on project webpulse-qa-ratingserver: There are test failures.

DependencyMap::Method "Test_RatingServer_Destructive.testRatingServer_CategoryNotZeroOnDRTRTimeout()[pri:0, instance:com.bluecoat.webpulse.qa.ratingserver.Test_RatingServer_Destructive@4b0b64cc]" depends on nonexistent group "functional"

org.apache.maven.surefire.booter.SurefireBooterForkException: There was an error in the forked process

DependencyMap::Method "Test_RatingServer_Destructive.testRatingServer_CategoryNotZeroOnDRTRTimeout()[pri:0, instance:com.bluecoat.webpulse.qa.ratingserver.Test_RatingServer_Destructive@4b0b64cc]" depends on nonexistent group "functional"

at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:673)

But I have defined the functional groups and they all ran just fine.

What am I missing?  Do I have to explicitly run with a groups directive?

[Krishnan Mahadevan]

Jeff,

 

I was able to simulate this using TestNG 6.14.3 with the below sample

 

import org.testng.annotations.Test;

public class ExampleClass {

  @Test

  public void parent() {
    System.err.println("parent");
  }

  @Test(dependsOnGroups = "parent")
  public void dependent() {
    System.err.println("child");
  }
}

org.testng.TestNGException: 

DependencyMap::Method "ExampleClass.dependent()[pri:0, instance:com.rationaleemotions.googleforums.groupdeps.jeff.ExampleClass@6b9651f3]" depends on nonexistent group "parent"

 

I was running the test from the IDE (IntelliJ) and behavior was the same using maven as well.

 

Here the moment I added the group attribute “parent” to the method parent(), the error was gone

 

This though sounds counter-intuitive [ why is dependency based on groups even being evaluated, when I am not running using groups], has a practical explanation.

 

TestNG at the point of executing a Test method, doesn’t know if the user is running everything or some sort of filtering is being applied [ was only a method chosen, or an entire class or perhaps a package, or did the user apply a groups filter on any of these first level filters].

 

So TestNG will try to satisfy whatever hard dependencies were specified via “dependsOnMethods” or “dependsOnGroups”.

 

To fix this issue, you would need to use a suite xml wherein you filter by groups properly, when you are running one or more @Test methods, that have the attribute “dependsOnGroups” defined for them.

 

Hope that clarifies.

 

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.
To post to this group, send email to testng...@googlegroups.com.
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

[Jeff]

As always, thanks for the response.  

Since I have the groups/dependsOnGroups at the class-level, the scenario you showed seemed slightly different than mine, so I setup a simple test project (code snippets below) similar to yours with a test class belonging to 'group1' and a 2nd class belonging to 'group2' but depending on 'group1' and it definitely worked fine in the simple test scenario.

In my actual code, I would also have expected it to work similarly, but it doesn't so on a whim I added all the annotation attributes.

I then discovered that if I include the 'testName' attribute to either of the class-level annotations, the dependency check fails.  The code below fails.  Comment out 'testName' from one or both classes and it runs fine.  

Why would a test name affect group dependency calculation?  

## Output - With 'testName' defined

There was an error in the forked process

DependencyMap::Method "TestGroup2.test2()[pri:0, instance:debug.TestGroup2@21bcffb5]" depends on nonexistent group "group1"

org.apache.maven.surefire.booter.SurefireBooterForkException: There was an error in the forked process

DependencyMap::Method "TestGroup2.test2()[pri:0, instance:debug.TestGroup2@21bcffb5]" depends on nonexistent group "group1"

## Output -  'testName' commented out

-------------------------------------------------------

 T E S T S

-------------------------------------------------------

Running TestSuite

debug.TestGroup1_Class1

debug.TestGroup2

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.866 s - in TestSuite

Results:

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0

##### Source Code #####

## Class 1

@Test(

        suiteName = "Test Group Dependencies",

        testName = "TestGroup1_Class1",  //<--CAUSES GROUP DEPENDENCY CALCULATION TO FAIL

        groups = { "Test", "group1" }

)

public class TestGroup1_Class1 {

    

    public TestGroup1_Class1() {

    }

    

    @Test

    public void test1() {

        System.out.println(this.getClass().getName());

    }

}

## Class 2

@Test (

        suiteName = "Test Group Dependencies",

        testName = "TestGroup2", //<--CAUSES GROUP DEPENDENCY CALCULATION TO FAIL

        groups = {"Test", "group2"},

        dependsOnGroups = { "group1" },

        singleThreaded = true

)

public class TestGroup2 {

    

    public TestGroup2() {

    }

    @Test

    public void test2() {

        System.out.println(this.getClass().getName());

    }

}

To unsubscribe from this group and stop receiving emails from it, send an email to testng-users+unsubscribe@googlegroups.com.

To post to this group, send email to testng...@googlegroups.com.
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to testng-users+unsubscribe@googlegroups.com.

[Krishnan Mahadevan]

Jeff,

I am not sure I quite understand the problem. Can you please help share the following:

 

1. Sample class (I guess I can use whatever you shared already)
2. TestNG version being used [ Please use 6.14.3]
3. How are the tests being run? If there’s a suite xml involved, please share.

 

Here’s what I tried:

 

package com.rationaleemotions.googleforums.groupdeps.jeff;

 

import org.testng.annotations.Test;

@Test(
  suiteName = "Test Group Dependencies",
  testName = "TestGroup1_Class1", // <--CAUSES GROUP DEPENDENCY CALCULATION TO FAIL
  groups = {"Test", "group1"}
)
public class TestGroup1_Class1 {

  public TestGroup1_Class1() {}

  @Test
  public void test1() {
    System.out.println(this.getClass().getName());
  }
}

 

-----------------------------------------------------------------------------------------------------------------------------------------------------------

 

package com.rationaleemotions.googleforums.groupdeps.jeff;

 

import org.testng.annotations.Test;

@Test(
  suiteName = "Test Group Dependencies"

,
  testName = "TestGroup2", // <--CAUSES GROUP DEPENDENCY CALCULATION TO FAIL
  groups = {"Test", "group2"},
  dependsOnGroups = {"group1"},
  singleThreaded = true

)
public class TestGroup2 {

  public TestGroup2() {}

  @Test
  public void test2() {
    System.out.println(this.getClass().getName());
  }
}

 

-----------------------------------------------------------------------------------------------------------------------------------------------------------

 

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="336_Suite-MySuite-InVM" parallel="false" verbose="2">
    <test name="336_test_1">
        <groups>
            <run>
                <include name="group1"/>
                <include name="group2"/>
            </run>
        </groups>
        <packages>
            <package name="com.rationaleemotions.googleforums.groupdeps.jeff"/>
        </packages>
    </test>
</suite>

 

 

Output

...

... TestNG 6.14.3 by Cédric Beust (ced...@beust.com)

...

com.rationaleemotions.googleforums.groupdeps.jeff.TestGroup1_Class1

com.rationaleemotions.googleforums.groupdeps.jeff.TestGroup2

 

===============================================

336_Suite-MySuite-InVM

Total tests run: 2, Failures: 0, Skips: 0

===============================================

 

 

Process finished with exit code 0

To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.

To post to this group, send email to testng...@googlegroups.com.
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "testng-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.

To post to this group, send email to testng...@googlegroups.com.
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.

[Jeff]

Going to attempt to send you the project ZIP file directly.  

My environment is:

Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T09:41:47-07:00)

Maven home: C:\Tools\apache-maven-3.3.9\bin\..

Java version: 1.8.0_131, vendor: Oracle Corporation

Java home: C:\Program Files\Java\jdk1.8.0_131\jre

Default locale: en_US, platform encoding: Cp1252

OS name: "windows 10", version: "10.0", arch: "amd64", family: "dos"

Using TestNG 6.14.3.  I run from command-line using mvn.  I do NOT use a testng.xml suite file.

Command:  'mvn clean test'

FULL OUTPUT (with error):

C:\Projects\testng-test>mvn clean test

[INFO] Scanning for projects...

[INFO]

[INFO] ------------------------------------------------------------------------

[INFO] Building testng-test 1.0-SNAPSHOT

[INFO] ------------------------------------------------------------------------

[INFO]

[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ testng-test ---

[INFO] Deleting C:\Projects\testng-test\target

[INFO]

[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ testng-test ---

[INFO] Using 'UTF-8' encoding to copy filtered resources.

[INFO] Copying 0 resource

[INFO]

[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ testng-test ---

[INFO] Nothing to compile - all classes are up to date

[INFO]

[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ testng-test ---

[INFO] Using 'UTF-8' encoding to copy filtered resources.

[INFO] skip non existing resourceDirectory C:\Projects\testng-test\src\test\resources

[INFO]

[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ testng-test ---

[INFO] Changes detected - recompiling the module!

[INFO] Compiling 2 source files to C:\Projects\testng-test\target\test-classes

[INFO]

[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ testng-test ---

[INFO]

[INFO] -------------------------------------------------------

[INFO]  T E S T S

[INFO] -------------------------------------------------------

[INFO] Running TestSuite

debug.TestGroup1_Class1

[INFO]

[INFO] Results:

[INFO]

[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO]

[INFO] ------------------------------------------------------------------------

[INFO] BUILD FAILURE

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 3.916 s

[INFO] Finished at: 2018-06-06T15:48:04-06:00

[INFO] Final Memory: 15M/206M

[INFO] ------------------------------------------------------------------------

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.21.0:test (default-test) on project testng-test: There are test failures.

[ERROR]

[ERROR] Please refer to C:\Projects\testng-test\target\surefire-reports for the individual test results.

[ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.

[ERROR] There was an error in the forked process

[ERROR]

[ERROR] DependencyMap::Method "TestGroup2.test2()[pri:0, instance:debug.TestGroup2@60f82f98]" depends on nonexistent group "group1"

[ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: There was an error in the forked process

[ERROR]

[ERROR] DependencyMap::Method "TestGroup2.test2()[pri:0, instance:debug.TestGroup2@60f82f98]" depends on nonexistent group "group1"

[ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:658)

[ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:533)

[ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:278)

[ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:244)

[ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1149)

[ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:978)

[ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:854)

[ERROR] at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)

[ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)

[ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)

[ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)

[ERROR] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)

[ERROR] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)

[ERROR] at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)

[ERROR] at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)

[ERROR] at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)

[ERROR] at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)

[ERROR] at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)

[ERROR] at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)

[ERROR] at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)

[ERROR] at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)

[ERROR] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

[ERROR] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

[ERROR] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

[ERROR] at java.lang.reflect.Method.invoke(Method.java:498)

[ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)

[ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)

[ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)

[ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)

[ERROR] -> [Help 1]

[ERROR]

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR]

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

To unsubscribe from this group and stop receiving emails from it, send an email to testng-users+unsubscribe@googlegroups.com.

To post to this group, send email to testng...@googlegroups.com.
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "testng-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to testng-users+unsubscribe@googlegroups.com.

To post to this group, send email to testng...@googlegroups.com.
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to testng-users+unsubscribe@googlegroups.com.

To post to this group, send email to testng...@googlegroups.com.
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to testng-users+unsubscribe@googlegroups.com.

[Krishnan Mahadevan]

Jeff,

 

My apologies for the delay in responding.

I finally got to the bottom of why this is happening.

 

For e.g., let’s consider the below from your sample

@Test (
        suiteName = "Test Group Dependencies"

,
        testName = "TestGroup2", //If set will cause failure
        groups = {"Test", "group2"},
        dependsOnGroups = { "group1" },
        singleThreaded = true
)
public class TestGroup2 {

 

When you set the attribute “testName” for the annotation @Test, here’s what you are telling TestNG:

 

Find a <test> whose name is “TestGroup2”:

• If you find one such <test> tag, please include the class “TestGroup2” and all the methods that are part of it, into that test tag.
• If you DON’T FIND any <test> tag with that name, create a new <test> tag dynamically set its name as “TestGroup2” and then include the class “TestGroup2” into this newly created test tag.

 

Quoting the Javadocs:

 

“The name of the test this test class should be placed in. This attribute is ignore if @Test is not at the class level.”

 

So when you run via “mvn test” wherein you haven’t configured the surefire plugin to use any testng suite xml, TestNG ends up creating 2 <test> tags wherein each class gets added into one <test> tag. Eventually this causes the test in one <test> tag to have a dependency on the test in the other <test> tag which causes the error that you are experiencing.

 

When you don’t use the “testName” attribute, TestNG ends up using the default mechanism to figure out the test name and so, there’s only 1 <test> tag in which both the classes are added and so the dependency resolution works fine.

 

So if you are planning to run your tests (which contains hard dependencies) and you plan to run them via surefire plugin without using any testng suite xml file, you should perhaps refrain from using the “testName” attribute for the @Test annotation when using it at the class level.

 

If you would like to use it, then you should be creating TestNG suite xml file and referring to that via your surefire plugin.

 

Hope that helps!

To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.

To post to this group, send email to testng...@googlegroups.com.
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "testng-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.

To post to this group, send email to testng...@googlegroups.com.
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.

To post to this group, send email to testng...@googlegroups.com.
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.

To post to this group, send email to testng...@googlegroups.com.
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.

[Jeff]

If I understand correctly, groups are defined as part of a "test" and limited in scope to only those tests/methods that share the same test name.  This seems to be almost like a group of test methods.  

And is this unique to maven sure-fire or is this something that would happen when running TestNG directly?

Thanks,

Jeff

[Krishnan Mahadevan]

Jeff,

From what I understand looking at the code, the behavior should be the same irrespective of whether surefire is used or not.