How to run multiple suites with preserve-order using <suite-files> tag

[Maniganda perumal]

Hi All,

For my need, I want to execute multiple suite files. So for easy just created a suite file and mentioned all the other suite files in the <suite-files> tag. 

<suite name="Merge Test Suite"  preserve-order="true">

<suite-files>

                <suite-file path="error2.xml" /> 

                <suite-file path="error3.xml" /> 

<suite-file path="testng_test.xml" /> 

<suite-file path="test-output/testng-failed.xml" />

</suite-files>

</suite> 

I want to execute the suites in the order as I mentioned. So for that I tried to use preserve-order attribute on the suite tag. But it is not worked as expected. It executes the suite files in random order. 

Is there any possible solution/ workaround to achieve it(preserve-order). 

Any help would be appreciated. TIA.

[吴亭]

Firstly, based on the testng dtd, you cannot put preserve-order in the suite tag.

Secondly, I cannot reproduce your problem, everything works for me. try see my log:

Which version of testng are u using now? I use latest version of testng.

[TestNG] Running:
  C:\Users\etigwuu\github\testng\src\main\java\gg\error2.xml

beforeMethod()
HAHAH
afterMethod()

===============================================
DemoSuite3
Total tests run: 1, Failures: 0, Skips: 0
===============================================

[TestNG] Running:
  C:\Users\etigwuu\github\testng\src\main\java\gg\error3.xml

beforeMethod()
HAHAH
afterMethod()

===============================================
DemoSuite2
Total tests run: 1, Failures: 0, Skips: 0
===============================================

[TestNG] Running:
  C:\Users\etigwuu\github\testng\src\main\java\gg\testng_test.xml

beforeMethod()
HAHAH
afterMethod()

===============================================
DemoSuite
Total tests run: 1, Failures: 0, Skips: 0
===============================================

[TestNG] Running:
  C:\Users\etigwuu\github\testng\src\main\java\gg\testng-failed.xml

beforeMethod()
HAHAH
afterMethod()

===============================================
DemoSuite5
Total tests run: 1, Failures: 0, Skips: 0
===============================================

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.
To post to this group, send email to testng...@googlegroups.com.
Visit this group at http://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

[Maniganda perumal]

Thanks for reply Tim. I used testng-8. Upgraded to testng-9. Works fine.

I had one more questions here.

<suite name = "master">

<suite-files>

<suite-file path = "testOne.xml"/>

<suite-file path = "test-output/testOne/testng-failed.xml"/> <!-- failed test suite of the above suite (testOne.xml)) --->

</suite-files>

</suite>

What am trying to achieve here is, I want to run a suite file first and then i want to execute the testng-failed.xml file of the executed one(here testOne.xml).

When i execute the above suite file. It executes the testOne.xml file and then immediately it executes the testng-failed.xml file. Because of this short time gap it always picks up the failed results of the previous execution.

I just want to add some time gap between the suite files. Is it possible to do something like that in testng? Or Is there any possible way/workaround to achieve the same?

[吴亭]

Hi,

I dont think you can do it like this way.

TestNG will parse the suite files before the whole suite starts, so whatever how you sleep or pause, it will run the previous failed result.

If you want to auto rerun previous failed cases,

one way could be:
java -classpath testng.jar;%CLASSPATH% org.testng.TestNG testng.xml
java -classpath testng.jar;%CLASSPATH% org.testng.TestNG testng-failed.xml

This will keep correct and failed in different jvm.

Another way:

You can have a TestNG listener which implements ITestSuiteListener, and IExecutionListener, after all of suite are finished, inside onExecutionFinish method, you can use TestNG.main(XXX) to invoke your failed suite here (One thing need to be mentioned, due to failed report is not generated during this time, so you can just paste a small part (Check IFailedCaseReporter()) of testng code and generate yourself :))