Handling configuration failures in Data providers and sending that status back programmatically

[Shreejit Nair]

Hi,

I noticed that if there is a failure in @DataProviders all the tests just get skipped.

I basically want to send back a status programmatically if the DataProvider fails.

This is what happens now:

1. if @DataProvider fails my tests are skipped.

TestngObject.status = 0 // This is still zero indicating no failures

TestngObject.hasSkip() // This is also false

I want to programmatically find out if there is a configuration failure due to DataProvider not working correctly and send that status back to my the running TestNG object.

[Shreejit Nair]

I am basically wanting a way to highlight and trap configuration failures (all dependencies before calling the test framework access routes/networks/test data setup) vs actual testing failures

some configuration failure happened - trap that and send the status back which would be different to testngobject.getStatus() // which only looks at @Test failures

[Shreejit Nair]

Any ideas on testing configuration failures in system other than just test failures.

I have a dataprovider that checksforsomeconfiguration grabs some data before pushing it to the test methods.

I want to find a way to trap this send this back and someway get a handle on it:

testng.hasconfigurationFailure // something on these lines.

Please let me know the best way to do it.

On Friday, July 15, 2016 at 1:44:39 PM UTC-5, Shreejit Nair wrote:

[⇜Krishnan Mahadevan⇝]

Shreejit,

lets start from the first. Can you please help provide a sample which we can run (please provide a full working sample) to simulate the problem you are experiencing ?

Also please let us know what is your expectation (showing example output from your sample code would go a long way in terms of helping us understand your issue)

From whatever little I understood about your problem statement, I have created a sample which simulates a data provider powered test failure and then prints out the list of tests that failed.

public class DPSample {
    public static void main(String[] args) {
        TestNG testNG = new TestNG();
        testNG.setTestClasses(new Class[] {SimpleTestClass.class});
        testNG.setVerbose(2);
        IReport report = new IReport();
        testNG.addListener(report);
        testNG.run();
        Set<ITestResult> failedTests = report.getFailedResults();
        for (ITestResult failedTest : failedTests) {
            String className = failedTest.getInstance().getClass().getName();
            String methodname = failedTest.getMethod().getMethodName() + "()";
            String parameters = Arrays.toString(failedTest.getParameters());
            System.err.println("***" + className + methodname + parameters + " failed.");
        }

    }

    public static class SimpleTestClass {
        @Test (dataProvider = "dp")
        public void hello(int a) {
            if (a == 0) {
                throw new IllegalArgumentException("Zeroes are not accepted.");
            }
            System.err.println("Hello " + a);
        }

        @DataProvider (name = "dp")
        public Object[][] getData() {
            return new Object[][] {
                {1}, {0}, {2}
            };
        }
    }


    public static class IReport implements IReporter {
        private Set<ITestResult> failedResults = new HashSet<>();

        @Override
        public void generateReport(List<XmlSuite> xmlSuites, List<ISuite> suites, String outputDirectory) {
            for (ISuite suite : suites) {
                Map<String, ISuiteResult> suiteResults = suite.getResults();
                for (ISuiteResult result : suiteResults.values()) {
                    failedResults.addAll(result.getTestContext().getFailedTests().getAllResults());
                }
            }
        }

        public Set<ITestResult> getFailedResults() {
            return failedResults;
        }
    }
}

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.
To post to this group, send email to testng...@googlegroups.com.
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.