dataprovider and test name

[Idan Shay]

Hi,

I am running a testNg using data provider.

I want to set the test name using the parameter given in the data provider.

If the test name is test1 and it run 3 times with data 1000,2000,3000 I want the test name to be :

test1_1000, test1_2000, test1_3000.

I define variable testName and set it in @BeforeMethod and return in in getTestName (implement ITest) but in report all tests are with name test1(3000).

I am running the tests using maven from jenkins.

Is there other way to change the test name ?

Thanks

[Julien Herr]

Hi,

It is supposed to work. Check the following test: https://github.com/cbeust/testng/blob/master/src/test/java/test/name/NameTest.java#L120-L137

Julien

[⇜Krishnan Mahadevan⇝]

Julien,

I gave this a try to check if something had changed in TestNG recently. It doesn't seem to work when it comes to trying to see the results in a built-in reporter.

If I had a test class as below :

public class DataProviderSample implements ITest {
    private static ThreadLocal<String> testNames = new ThreadLocal<String>();

    @Test (dataProvider = "dp")
    public void testMethod(int number, String text) {
        String txt = "Thread Id [" + Thread.currentThread().getId() + "] with value (" + number + "," + text + ")";
        testNames.set(txt);
    }

    @DataProvider (name = "dp", parallel = true)
    public Object[][] getData() {
        return new Object[][] {
            {1, "Poetry"},
            {2, "Novels"},
            {3, "TextBooks"}
        };
    }

    @Override
    public String getTestName() {
        return testNames.get();
    }
}

and a suite file as below 

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Class1Suite" parallel="methods" verbose="2">
    <test name="SampleTest">
        <classes>
            <class name="organized.chaos.forums.DataProviderSample"/>
        </classes>
    </test>
</suite> 

I initially trigger a NullPointerException because my getTestName() implementation attempts at invoking a ThreadLocal value, but since the Reporters are perhaps running in a different thread, they don't have any value set as a testname within testNames ThreadLocal variable. [ only way to get past the problem is to define a initialValue() implementation for the ThreadLocal variable and provide with a default value.

private static ThreadLocal<String> testNames = new ThreadLocal<String>(){
    @Override
    protected String initialValue() {
        return "Unknown";
    }
};

I believe that for one to be able to achieve this requirement from the OP, one would need to build a custom reporter which would perhaps extract an attribute named "testName" from a method's ITestResult object and then construct the test report. One could maybe build an implementation of org.testng.IInvokedMethodListener listener and within its beforeInvocation() one could set this attribute for a test method.

Atleast that was the only way I saw in which one can do this.

Please let me know if I missed something anywhere.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.
To post to this group, send email to testng...@googlegroups.com.
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

[Julien Herr]

You're right. There is no test with reporter and names, and it seems to not work.

Could you propose a new test case that shows the problem?

[⇜Krishnan Mahadevan⇝]

Sure Julien. Let me see if I can create one over the weekend and share it.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

[Idan Shay]

Hi,

I just want to know that I am not missing something - Shouldn't it be part of the data provider feature to be able simply report the test name with it's parameter ?

I thought on create list of test name in the @BeforeMethod and than return the correct one according the index of the call to getTestName.as a solution.

In my example it will (just an example) :

String [] testInstance = {"test1_1000","test1_2000","test1_3000"} ; // It actually will be created in @BeforeMethod

int i = 0;

public String getTestName() {

   return testInstance[i++ % 3];

[⇜Krishnan Mahadevan⇝]

@Idan,

No, I don't think that's an attribute of the data provider. The provider feature in its entirety (In My Opinion) should be just confined to being able to support executing the same test method but with different parameters. 

What you are expecting is supposed to be an attribute of the reporting mechanism. Since TestNG provides for all the means to customize the reporting in any way you want, I think you can get this done on your own [ by following the approach that I had called out in my earlier post.. that's just one of the ways. Am sure others have pretty neat and innovative ways of doing it as well ]

Julien,

Here's a sample test that can be executed to recreate the problem 

public class Example {
    public static TestNG create() {
        TestNG result = new TestNG();
        result.setUseDefaultListeners(false);
        result.setVerbose(0);
        return result;
    }

    public static TestNG create(Class<?>... testClasses) {
        TestNG result = create();
        result.setTestClasses(testClasses);
        return result;
    }

    @Test
    public void complexITestTest() {
        TestNG tng = create(ITestSample.class);
        tng.setDataProviderThreadCount(2);
        TestListenerAdapter adapter = new TestListenerAdapter();
        tng.addListener(adapter);
        LocalReporterListenerAdaptor reporter = new LocalReporterListenerAdaptor();
        tng.addListener(reporter);
        ExecutorService service = Executors.newFixedThreadPool(1);
        service.submit(new Runnable() {
            @Override
            public void run() {
                tng.run();
            }
        });

        Assert.assertTrue(adapter.getFailedTests().isEmpty());
        Assert.assertTrue(adapter.getSkippedTests().isEmpty());
        Assert.assertEquals(adapter.getPassedTests().size(), 5);
        List<String> testNames = new ArrayList<>(Arrays.asList("test1", "test2", "test3", "test4", "test5"));
        for (ITestResult testResult : reporter.getResults()) {
            Assert.assertTrue(testNames.remove(testResult.getName()),
                "Duplicate test names " + getNames(adapter.getPassedTests()));
        }
        Assert.assertEquals(testNames, Collections.emptyList());
    }

    private static List<String> getNames(List<ITestResult> results) {
        List<String> names = new ArrayList<>(results.size());
        for (ITestResult result : results) {
            names.add(result.getName());
        }
        return names;
    }

    public static class LocalReporterListenerAdaptor implements IReporter {
        private Set<ITestResult> results;

        @Override
        public void generateReport(List<XmlSuite> xmlSuites, List<ISuite> suites, String outputDirectory) {
            ITestContext testContext = suites.get(0).getResults().values().iterator().next().getTestContext();
            IResultMap resultMap = testContext.getPassedTests();
            results = resultMap.getAllResults();
        }

        public Set<ITestResult> getResults() {
            return results;
        }
    }

    public static class ITestSample implements ITest {
        public ThreadLocal<String> testName = new ThreadLocal<>();

        @DataProvider (name = "dp", parallel = true)

        public Object[][] getTests() {
            return new Object[][] {new Object[] {"test1"},
                new Object[] {"test2"},
                new Object[] {"test3"},
                new Object[] {"test4"},
                new Object[] {"test5"}};

        }

        @Test (dataProvider = "dp")

        public void run(String testName) {
            Assert.assertEquals(testName, this.testName.get());
        }

        @BeforeMethod
        public void init(Object[] testArgs) {
            testName.set((String) testArgs[0]);
        }

        @AfterMethod
        public void tearDown(Object[] testArgs) {
            Assert.assertEquals((String) testArgs[0], this.testName.get());
        }

        @Override
        public String getTestName() {
            return testName.get();
        }
    }

}

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/