Remove retried test methods from report

[stefan...@gmail.com]

Hello everybody!

I wrote a RetryAnalyzer for my testng test cases based on this article:
http://stackoverflow.com/questions/7803691/how-to-optimize-testng-and-seleniums-tests

Now I am facing the problem of having duplicate entries for retried tests in reports. So when a failed test is being retried and is ends successful there are two entries in the report saying that it passed and failed one time. But what I need is the result to be overridden. Such tests should then be only marked as successful, This seems to be a problem for some others too, but a search through the forum didn't bring the conclusion.
The only thing I found was Cedric giving a hint to "patch the SuiteReporter" but I don't have a clue where to start or what to do exactly. Could you be more specific on how to do this?

It would be very helpful, because right now complete builds are being handled as failed, although all tests passed at least on second try.

Basic info:
* testng version 6.8
* using reportNG and default HTML reports
* using a build server: Jenkins

Best regards

Stefan

[John Papadakis]

Hi this could work for you: 

Your RetryAnalyzer should implement ITestListener

If the invocation count is 2 and one result is failed and one passed or the two of them are failed  then remove one result of the test method before reporting the results.

public void onFinish(ITestContext context) {

for(int i=0;i<context.getAllTestMethods().length;i++){

if(context.getAllTestMethods()[i].getCurrentInvocationCount()==2){

if (context.getFailedTests().getResults(context.getAllTestMethods()[i]).size() == 2 || context.getPassedTests().getResults(context.getAllTestMethods()[i]).size() == 1){

context.getFailedTests().removeResult(context.getAllTestMethods()[i]);

[stefan...@gmail.com]

Hi again,

First of all thanks for the reply!

I do have a TestListener and I added an onFinish method, which now removes duplicates from my generated reports. Thanks for that!

The only problem left is that the builds are still marked as unsuccessful, because retried failed methods are still recognized as failed by the system.
Any ideas on how to tell testng that there where no failures when a retried test succeeds?



Best regards,

Stefan

[Shu]

Never mind , I override the default test report and have it working correctly now .

On Friday, March 8, 2013 5:12:07 PM UTC-8, Shu wrote:

Hi,

I have a similiar issue that the HTML report generated by TestNG get dup for my retried test case on detail page.
 I have implemented the TestListener and added the de-dup code in onFinish method.

 Iterator<ITestResult> failedTestCases = testContext.getFailedTests().getAllResults().iterator();
        while (failedTestCases.hasNext())
        {
            ITestResult failedTestCase = failedTestCases.next();
            ITestNGMethod method = failedTestCase.getMethod();           
            if ( testContext.getFailedTests().getResults(method).size() > 1)
            {
                logger.debug("failed test case remove as dup:" + failedTestCase.getTestClass().toString());
                failedTestCases.remove();                 
            }
            else
            {
               
                if (testContext.getPassedTests().getResults(method).size() > 0)
                {
                    logger.debug("failed test case remove as pass retry:" + failedTestCase.getTestClass().toString());
                    failedTestCases.remove();
                }                         
            }           
        }
After the change, I can see the failed test count on the index.html page is correct , but if  I click to see details, it still shows all failed test cases with all retried

Any idea?

[kamalakar chary]

Hi

Could please let me how to override the default test report??

Regards
KV

[Indrani biswas]

Can you let us know how to override the test method to remove duplicate test after retry

[Govindaram PS]

I am able successful remove all the failed results from TestNG which are passed in second attempt.

I am running the test through maven. The problem here is maven fail safe plugin is making the build fail.

Maven fail safe plugin holds its own test results. 

How to update maven fail safe's results programatically?

[Owen Fletcher]

"

The only problem left is that the builds are still marked as unsuccessful, because retried failed methods are still recognized as failed by the system.

Any ideas on how to tell testng that there where no failures when a retried test succeeds?"  I would also be interested in a resolution to this problem.

[Indrani biswas]

Hi Govindaram,

Could you please share the code on how you have achieved to remove duplicate test results from reports. Will be really helpful.

Thanks,

--Indrani

[Indrani biswas]

Hi fletch,

Could you please share the code on how you have achieved to remove duplicate test results from reports. Will be really helpful.

Thanks,

--Indrani

[Martino Turturiello]

Hi all,

can someone share its working solution?

[Automation Enthusiast]

Shu,

I know this is now a #+ years old thread. Can you post your solution here since looks like a bunch of us would like to do something similar but unable to find the exact solution.

My TestNG results are showing accurately and are excluding any retried tests that PASS, but the build is still being marked unsuccessful and in Jenkins, the default test results are also being displayed with all the retries and failures being shown. Is there a way to turn off the default test results if one is using the TestNG results plugin ?

Thanks!

[aswin hari rao]

Hi,

Any solution to this, I am able to remove duplicate failed test cases from the report,testng report looks fine after the retries, but the maven build is always marked failed even if retry is successful.

Can any one help with solution

[nagandla lakshmi narayana]

May I know which version of extent report you are using currently ?

Recently I have given fix for the problem which we reported and please make sure below validations which you done from your side 

1. Just implement Retry analyzer as usual, Make sure remove skipped test using RetryAnalyzer (hope it will try to fix the issue from TestNG side)

2. Use extentReport.removeTest(test); method to remove logs from current ExtentTest object.  It will remove logs from current ExtentTestinstance. Then flush test results into extentreport level. 

3. Don't add child node to parent node category at start test level. Just add when you get desired result either pass instance < = RetryMaxCount or Fail == RetryMaxCount at end test case level.  

Note:  Make sure where we exactly get current instance of ExtentTest, which hold current test logs. 

Hope it may help you guys, let me know if you guys face any problems 

Thanks,

Lakshmi

[Ashwini Pattanayak]

Hi Lakshmi,

Right now I am struggling with remove retry failed test cases from extent report. Can you please give the code snippet please?

[Anoop Singh]

Hi Group,

How can we generate the extent report for re-run scenarios so that Test case is marked only once in the final report ? Please provide your input if you are able to achieve this using ExtenReport ?

 

[Banh Duy]

Vào 21:31:43 UTC+7 Thứ Năm, ngày 07 tháng 5 năm 2020, Anoop Singh đã viết:

Hi Group,

How can we generate the extent report for re-run scenarios so that Test case is marked only once in the final report ? Please provide your input if you are able to achieve this using ExtenReport ?

 

ExtentReports extentReports = ...;

ExtentTest currentExtentTest= ...;

...

@AfterMethod(alwaysRun = true)

public void afterMethod(Method method, ITestResult testResult) {

    if (testResult.getStatus() == 3) {  // 3 = skipped

        extentReports.removeTest(currentExtentTest);

    }

}