Connaot run tests when group-by-instances and preserve-order used together

[Pawel Kowalski]

Hi Cedric,

Since new  Gradle 1-milestone-7 has been released, which now supports TestNG >6.2, I decided to upgrade test projects to use TestNG 6.3.1, but I was surprised that none of my suites could be execuded. I always got exception:
 
org.testng.TestNGException:
No free nodes found in:[DynamicGraph
...

    at org.testng.TestRunner.privateRun(TestRunner.java:752)
    at org.testng.TestRunner.run(TestRunner.java:613)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
...

I have spent some time figuring out what is going on and I've discovered that is caused by using group-by-instance and preserve-order set to true (preserve-order default is true,) at the same time. When the mentioned options are set to true  and there are at least two classes with tests to run , than it is very likely that TestNG will build dependency graph with cycles even if tests don't have any explicit dependency. The built graph will have dependency on classes to satisfy order and then dependency on instances to guarantee grouping. The more classes is the probability is higher. In my real projects it was impossible to run anything anytime.  I was able to reproduced the problem with only two classes (it is not so easy ) but with three classes the probability is almost 100%.

Below is my "almost always reproducible" example:

//  1st class
package test;
import org.testng.annotations.Test;

public class X {
    @Test
    public void testA() {}
    @Test
    public void testB() {}
}

//  2nd class
package test;
import org.testng.annotations.Test;

public class Y {
    @Test
    public void testA() {}
    @Test
    public void testB() {}
}


//  3rd class
package test;
import org.testng.annotations.Test;

public class X {
    @Test
    public void testA() {}
    @Test
    public void testB() {}
}

Run with  following suite:

<suite name="Suite1" verbose="1" group-by-instances="true">
  <test name="Test" preserve-order="true">
    <packages>
       <package name="test.*" />
    </packages>
  </test>
</suite>

An almost every time get:


org.testng.TestNGException:
No free nodes found in:[DynamicGraph
  Ready:[X.testA()[pri:0, instance:test.tmp.x.X@38b5dac4], X.testB()[pri:0, instance:test.tmp.x.X@38b5dac4], Y.testA()[pri:0, instance:test.tmp.x.Y@69945ce], Y.testB()[pri:0, instance:test.tmp.x.Y@69945ce], Z.testA()[pri:0, instance:test.tmp.x.Z@266bade9], Z.testB()[pri:0, instance:test.tmp.x.Z@266bade9]]
  Running:[]
  Finished:[]
  Edges:
     X.testB()[pri:0, instance:test.tmp.x.X@38b5dac4]
        Y.testA()[pri:0, instance:test.tmp.x.Y@69945ce]
        Y.testB()[pri:0, instance:test.tmp.x.Y@69945ce]
        Z.testA()[pri:0, instance:test.tmp.x.Z@266bade9]
        Z.testB()[pri:0, instance:test.tmp.x.Z@266bade9]
     Y.testA()[pri:0, instance:test.tmp.x.Y@69945ce]
        Z.testA()[pri:0, instance:test.tmp.x.Z@266bade9]
        Z.testB()[pri:0, instance:test.tmp.x.Z@266bade9]
        X.testA()[pri:0, instance:test.tmp.x.X@38b5dac4]
        X.testB()[pri:0, instance:test.tmp.x.X@38b5dac4]
     X.testA()[pri:0, instance:test.tmp.x.X@38b5dac4]
        Y.testA()[pri:0, instance:test.tmp.x.Y@69945ce]
        Y.testB()[pri:0, instance:test.tmp.x.Y@69945ce]
        Z.testA()[pri:0, instance:test.tmp.x.Z@266bade9]
        Z.testB()[pri:0, instance:test.tmp.x.Z@266bade9]
     Y.testB()[pri:0, instance:test.tmp.x.Y@69945ce]
        Z.testA()[pri:0, instance:test.tmp.x.Z@266bade9]
        Z.testB()[pri:0, instance:test.tmp.x.Z@266bade9]
        X.testA()[pri:0, instance:test.tmp.x.X@38b5dac4]
        X.testB()[pri:0, instance:test.tmp.x.X@38b5dac4]
]
    at org.testng.TestRunner.privateRun(TestRunner.java:752)
    at org.testng.TestRunner.run(TestRunner.java:613)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
    at org.testng.SuiteRunner.run(SuiteRunner.java:240)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:87)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1142)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1067)
    at org.testng.TestNG.run(TestNG.java:979)
    at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:109)
    at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:202)
    at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:173)

Regards,
Paweł Kowalski

[Pawel Kowalski]

Small update
In my  example 3rd class should be  of course Z (or any different than X and Y) not X. 

--
Pozdrawiam,
Paweł Kowalski