Reporting and DataProvider

[Tina Li]

Hi there,

I'm trying to write a custom reporter to gather all of my test results and logging information. I started to use TestNG's Reporter class for logging. I then wrote my own listener, which implements ITestListener, to write a report file as my tests are executed. I'm finding that when I have a simple test that takes no parameter, I am able to do the following and create a report:

public void onTestSuccess(ITestResult tr) {
List<String> outputList = Reporter.getOutput(tr);

// ... write the content in outputList to file
}

In this case, I am able to get the results for each of the executed tests and organize them to different files.

However, when I add a @DataProvider annotation to my test and make it take a parameter, I can't use the Reporter.getOutput(ITestResult) method anymore. When I run with the same listener as listed above, I get no output back.

Is there something that I'm missing? How is each test result mapped to its logging output?

Any advice is appreciated!

Thanks,

Tina Li
---------------------------------------------------------------------
Posted via Jive Forums
http://forums.opensymphony.com/thread.jspa?threadID=43866&messageID=88328#88328

[Cédric Beust ♔]

Hi Tina,

I was not able to reproduce the behavior you describe.  I wrote a self-contained test that I include below, and I get the expected result:

[ReporterTest] ON SUCCESS, OUTPUT:[IN THE REPORTER: param1]
[ReporterTest] ON SUCCESS, OUTPUT:[IN THE REPORTER: param2]

===============================================
Command line suite
Total tests run: 2, Failures: 0, Skips: 0
===============================================

Can you modify this example in order to reproduce the bug you are describing?

--
Cédric


package test.tmp;

import org.testng.ITestListener;
import org.testng.ITestResult;
import org.testng.Reporter ;
import org.testng.TestListenerAdapter;
import org.testng.TestNG;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class ReporterTest {
 
  @DataProvider(name = "dp")
  public Object[][] createParameters() {
    return new Object[][] {
        new Object[] { "param1"},
        new Object[] {"param2"}
    };
  }

  @Test(dataProvider = "dp")
  public void report(String p) {
    Reporter.log("IN THE REPORTER: " + p);
  }
 
  public static void main(String[] argv) {
    TestNG tng = new TestNG();
    tng.setTestClasses(new Class[] { ReporterTest.class });
   
    ITestListener listener = new TestListenerAdapter() {
      @Override
      public void onTestSuccess(ITestResult tr) {
        super.onTestSuccess(tr);
        ppp("ON SUCCESS, OUTPUT:" + Reporter.getOutput(tr));
      }
    };
    tng.addListener(listener);
    tng.run();
  }

  private static void ppp(String s) {
    System.out.println("[ReporterTest] " + s);
  }
}

[Tina Li]

I just found out that getting the reports from a testresult if a timeout attribute is added to my Test annotation. If I remove the timeout attribute, I seem to be able to retrieve the report output for my each test result.

Tina

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=43866&messageID=88336#88336

[Cédric Beust ♔]

On 9/21/06, Tina Li <testng...@opensymphony.com> wrote:

I just found out that getting the reports from a testresult if a timeout attribute is added to my Test annotation.  If I remove the timeout attribute, I seem to be able to retrieve the report output for my each test result.

Oh, indeed!

That's a weird bug, I'm investigating it.

--
Cédric

[Tina Li]

Hi Cedric,

Thanks for your fast reply!

It's definitely something to do with the timeOut attribute. All I did was add a timeOut attribute to the test annotation and set that to 160000. When I run the test, I don't see any output. Below is what I've modified and the corresponding output.

------------------------------------------------------------------------------
package test.tmp;

public class ReporterTest {

-----------------------------------------------------------------------------------
Output:

[ReporterTest] ON SUCCESS, OUTPUT:[]
[ReporterTest] ON SUCCESS, OUTPUT:[]

===============================================
Command line suite
Total tests run: 2, Failures: 0, Skips: 0
===============================================

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=43866&messageID=88342#88342

[Cédric Beust ♔]

Hi Tina,

It's now fixed.  You can download the fixed distribution here.

Please confirm that it works for you.

--
Cedric

[Tina Li]

Cool, it works! Thanks!

Tina

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=43866&messageID=88353#88353