Can we set dependency between <test> tag in testng.xml

[garvitag...@gmail.com]

Hi,

 I have 3 <test> tags in testng.xml. Is there a way i can set dependency between test tags like :

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">

<suite name="Suite" configfailurepolicy="continue">

    <listeners>

        <listener class-name="ipd.listeners.implementation.TestListener"></listener>

    </listeners>

    <test name="Project Setup" parallel="false">

        <classes>

            <class name="ipd.administration.clients.CreateClient" />

            <class name="ipd.administration.projects.CreateProject" />

          </classes>

    </test>

    <test name="Charting" parallel="classes" thread-count="3">

        <classes>

            <class name="ipd.administration.advancedSearch.Search" />

            <class name="ipd.administration.advancedSearch.NumberSearch" />

        </classes>

    </test>

    <test name="New Records Page" parallel="classes" thread-count="3">

        <classes>

            <class name="ipd.administration.newRecordsPage.Search" />

        </classes>

    </test>

</suite>

Test Charting only starts if test Project setup passes else it gets skipped. I can not set dependency in @Test as it disrupts order of parallel execution. 

Thanks !

[⇜Krishnan Mahadevan⇝]

No you cannot establish dependencies between <test> tags.

Dependencies can only be established with a <test> tag.

Also since the xml syntax doesnt have a way wherein you can express these dependencies, the only round about way of doing this would be the following:

1. Create a text file wherein you express this dependencies.

2. Use a JVM argument to pass in the name of this file to your execution.

3. Build a TestNG listener (implementing ITestListener) wherein within its onStart(ITestContext ctx) you check the file from (1) to see if there are any dependencies and if there are, then check for the overall ITestContext result (by checking if the failed tests size is zero) and only then proceed else, set an attribute to the current ITestContext indicating TestNG should fail everything.

4. Now build an IInvokedMethodListener which would cause TestNG to skip.

All of this is only theoretical. Am not sure how far would this work. But you can give this a try.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribblings @ http://rationaleemotions.wordpress.com/

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.
To post to this group, send email to testng...@googlegroups.com.
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

[garvitag...@gmail.com]

While exploring i came across this https://groups.google.com/forum/#!topic/testng-users/tArXKOPHjmk.

A way to define dependency between suites. I thought to try as it will do my purpose and i had to do minimal changes. I added one suite file as child of other. But its not working.

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">

<suite name="suite of suites" >

    <suite-files>

        <suite-file path="testng.xml" >        

        <suite-file path="my.xml" />

        </suite-file>

    </suite-files>

   

</suite>

I added my.xml as child of testng.xml so my.xml should not start if testng.xml fails. But it did not work this way. 

I will try the way around you have described. Thanks !

[Krishnan Mahadevan]

To the best of my knowledge, TestNG doesn’t let you define dependencies amongst suites in the way you describe.

 

Re-iterating what I said before:

Dependencies can exist only within a <test> tag (because that’s the small level of grouping in TestNG)

 

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

[garvita girotra]

Hi Krishnan,

As i have 3 <test> in testng.xml tag :

<test name = "Project Setup">

<test name = "Reporting">

<test name = "New Dashboard">

Requirement was : If any test in project setup failed, Reporting should not continue. 

I achieved by following implementation in listener:

1. Set flag if any test is failed in Project setup test in OnTestFailure

2. Check flag, if true skip in OnStart().

Boolean isProjectSetupFailed = false;

  @Override

    public void onTestFailure(ITestResult result) {

        getTestLog(result, "FAIL", ANSI_RED);                                            // It gives me log like method complete name, status etc.

       

        if (result.getTestContext().getName().equalsIgnoreCase("Project Setup")) {

            projectSetupFailed = true;

        }

    }

 @Override

    public void onStart(ITestContext context) {

        if (projectSetupFailed) {

                throw new SkipException(

                    "One or more methods failed in set up project suite");

        }

    }

Issue : Basic requirement is achieved, Next test does not start. But i encountered 2 issues :

1. Once <test> is skipped, i get 

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0     (though test run in project setup test.)

2. OnStart is not called for 3rd <test> New Dashboard. It just skips everything. Is this expected functionality that if once OnStart has skip statement , next <test> will not be executed ?

Thanks !!

[⇜Krishnan Mahadevan⇝]

How about you creating a simple standalone project that uses the latest released version of TestNG (7.0.0-beta1), which we can use to reproduce the problem? That way we can get better idea on what is going wrong.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribblings @ http://rationaleemotions.wordpress.com/

[garvitag...@gmail.com]

Hi,

Here is sample project https://github.com/reactJestuser/mavenproject.

I tried with latest version of maven-complier and testng. 

1. There are 2 <test> in  testng.xml. I have set condition in listner onStart() method taht if any of test fails in <test1>, <test2> should not start and throw exception.

2. It always shows  Tests run: 0, Failures: 0, Errors: 0 even if one of test passes in <test1>.

Thanks !!

[Krishnan Mahadevan]

The issue was within your listener. You shouldn’t be throwing an exception on your onStart(ITestContext) method. You should instead be doing that from within onStart(ITestResult).

 

Here’s how your fixed listener looks like:

 

package sample;

import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestResult;
import org.testng.SkipException;

public class TestListener implements ITestListener {

  private static final String ANSI_GREEN = "\u001B[32m";
  private static final String ANSI_YELLOW = "\u001B[33m";
  private static final String ANSI_RED = "\u001B[31m";
  private static final String ANSI_RESET = "\u001B[0m";


  @Override
  public void onTestStart(ITestResult result) {
    Object value = result.getTestContext().getSuite().getAttribute("skipFurther");
    boolean projectSetupFailed = value != null;
    if (projectSetupFailed) {
      throw new SkipException("Simulating a failure");
    }
  }

  @Override
  public void onTestSuccess(ITestResult result) {
    getTestLog(result, "PASS", ANSI_GREEN);
  }

  @Override
  public void onTestFailure(ITestResult result) {
    getTestLog(result, "FAIL", ANSI_RED);
    if (result.getTestContext().getName().equalsIgnoreCase(
        "Test1")) {
      result.getTestContext().getSuite().setAttribute("skipFurther", true);
    }
  }

  @Override
  public void onTestSkipped(ITestResult result) {
    getTestLog(result, "SKIP", ANSI_YELLOW);
  }

  @Override
  public void onTestFailedButWithinSuccessPercentage(ITestResult result) {
  }

  @Override
  public void onStart(ITestContext context) {

  }

  @Override
  public void onFinish(ITestContext context) {

  }

  private void getTestLog(ITestResult result, String testStatus, String statusColor) {

    float timetakenToExecute = (result.getEndMillis() - result
        .getStartMillis()) / 1000.0f;
    StringBuilder log = new StringBuilder();
    String methodname = result.getTestClass().getName() + "." + result
        .getMethod().getMethodName();
    String status = statusColor + testStatus + ANSI_RESET;
    log.append(methodname);

    // Add 1st parameter name in case of data providers
    if (result.getParameters().length > 1) {
      for (Object eachObject : result.getParameters()) {
        String parameter = eachObject.toString();
        String[] parameterName = parameter.split("_");
        log.append(".").append(parameterName[0]);
      }
    }
    log.append(" ...  ");
    log.append(status);
    log.append(" ");
    log.append("(").append(timetakenToExecute).append("s)");
    System.out.println(log);
  }

}

 

 

Output:

 

[INFO] --- maven-surefire-plugin:2.20:test (default-test) @ ipdashboardtests ---

[INFO]

[INFO] -------------------------------------------------------

[INFO]  T E S T S

[INFO] -------------------------------------------------------

[INFO] Running TestSuite

...

... TestNG 7.0.0-beta3 by Cédric Beust (ced...@beust.com)

...

 

Class1 method

sample.Class1.method1 ...  PASS (0.01s)

Class2 method

sample.Class2.method2 ...  FAIL (0.0s)

sample.Class3.method3 ...  SKIP (0.0s)

[ERROR] Tests run: 3, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 0.466 s <<< FAILURE! - in TestSuite

[ERROR] method2(sample.Class2)  Time elapsed: 0.007 s  <<< FAILURE!

java.lang.AssertionError: null

    at sample.Class2.method2(Class2.java:11)

 

[INFO]

[INFO] Results:

[INFO]

[ERROR] Failures:

[ERROR]   Class2.method2:11 null

[INFO]

[ERROR] Tests run: 3, Failures: 1, Errors: 0, Skipped: 1

[INFO]

[INFO] ------------------------------------------------------------------------

[INFO] BUILD FAILURE

[INFO] ------------------------------------------------------------------------

[INFO] Total time:  2.979 s

[INFO] Finished at: 2019-03-01T16:03:07+05:30

[INFO] ------------------------------------------------------------------------

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.20:test (default-test) on project ipdashboardtests: There are test failures.

[ERROR]

[ERROR] Please refer to /Users/krmahadevan/temp/mavenproject/target/surefire-reports for the individual test results.

[ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.

[ERROR] -> [Help 1]

[ERROR]

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR]

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

 

 

 

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.com/

[Gagandeep Singh]

Hi Krishnana,

The listener you have provided  is not doing what @garvitag asked for. He asked for the following:- As i have 3 <test> in testng.xml tag :

<test name = "Project Setup">

<test name = "Reporting">

<test name = "New Dashboard">

Requirement was : If any test in 'Project Setup' failed, Reporting should not continue.

He achieved this by implementing a listener. I have the same problem, I want to achieve the same kind of functionality as Suite level dependency is not supported in TestNG yet. So, I implemented his listener and its working fine upto some extent for the following XML File:- 

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name ="Suite Runner File" verbose="4" >

        <listeners>
            <listener class-name ="it.jira.seleniumtests.suiteutils.SuiteFailListener" />
        </listeners>

    <test name ="Project Setup" verbose ="2" >
        <classes>
            <class name ="it.jira.seleniumtests.smoketests.EasySsoEnableTest"/>
                        <class name ="it.org.techtime.jira.easysso.seleniumtests.smoketests.LicenseVerificationTest"/>
<!--                        <class name ="it.jira.seleniumtests.smoketests.AddOnsPanelTests"/>-->
        </classes>
    </test>

    <test name ="Reporting" verbose ="2" >
       
            <class name ="it.jira.seleniumtests.linksvalidation.AdminScreenTests"/>
        </classes>
    </test>

</suite>

The only problem is, it's not displaying the results in a right way as its supposed to show when a test fails. Secondly, if the last test in the "Project Setup" passes then it will start the "Reporting" test, which is not the requirement. It will be highly appreciated if you can provide a work around for these issues.

 

The listener you have provided is will stop the execution of the next test method/class in the same <test> tag i.e "Project Setup", which is not what is required.   

[Krishnan Mahadevan]

Gagan,

 

What is your point that you are trying to make? Can you please elaborate?

 

If you are saying that my solution didn’t work, that’s perfectly fine. But beyond that, I would like to understand what is it that you are stating?

 

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.com/

 

From: <testng...@googlegroups.com> on behalf of Gagandeep Singh <singh.d...@gmail.com>
Reply-To: <testng...@googlegroups.com>
Date: Friday, July 26, 2019 at 1:57 AM
To: testng-users <testng...@googlegroups.com>
Subject: Re: [testng-users] Can we set dependency between <test> tag in testng.xml

 

Hi Krishnana,

--

You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/testng-users/65a1d1bd-c419-452b-8dce-ab3c7a3d60e7%40googlegroups.com.