Wanted to rerun the failed testcases before generation of report

[kamalakar vadla]

Suppose i have two xml suites with 100 Testmethods .

10 testcases failed in the first run and report generated ? but before generating the report i want to rerun the failed testcases again generate the report.

Can anybody suggest how can i achieve that .? 

As per knoweldge i need to modify TestNG.class? 

 
Kamalakar
SocialTwist

[Cédric Beust ♔]

How about just running testng-failed.xml?

--
Cédric

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to testng...@googlegroups.com.
To unsubscribe from this group, send email to testng-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
Cédric

[kamalakar vadla]

I can rerun the testng-failed.xml but the prob is i have integrated these tests to Hudson CI so when u rerun the reports accumulated will be Over written by rerun . 

My Requirement :

I have integrated TestNG with Selenium Tests and i have 200 test cases those will run every hour and these selenium tests are very unstable due network timeout becoz most of my tests call outside api's.

So, in the first run out 200 testcases let say 3 failed then i wanted to rerun once and cross that these are real failures .

Solution 1:

I can rerun using testng-failed.xml but the problem is it will mess up hudson report(build stability as it override the first run report)??

Solution 2: 

I was looking for the code where i can rerun before generating the report??

Suggest me best solution for this.

-- Kamalakar

2010/6/29 Cédric Beust ♔ <ced...@beust.com>

--
Kamalakar.Vadla
Pramati Technologies Pvt Ltd
Begumpet

[Cédric Beust ♔]

Why don't you run the second time with a different output directory?  This way, your reports won't get overridden.

--
Cédric

[kamalakar vadla]

I am not sure how hudson will treat this rerun?? My only concern if Hudson should treat this as build stable if at all after rerun if all tests passed.

Lets try and see if it works for me or not?

Any thanks for quick reply 

Kamalakar

Socialtwist

2010/7/1 Cédric Beust ♔ <ced...@beust.com>

[kamalakar vadla]

• I ran 10 Testcases and in that  8 passed 2 failed  and so hudson treat as build unstable and sends a mail. This send a false alaram
• Rerunning testing-failed.xml will not serve my purpose .
• Why i want to Rerun before the report generate is 
• Most of my testcases communicate with the outside application and there so many issues like network lag etc will effect testcase result.I want to make sure once again by rerunning the tests and send the mail to the team( These tests will run very frequently)
• So suggest me best solution for this problem?

Kamalakar

SocialTwist

[Cédric Beust ♔]

Have you looked at IRetryAnalyzer?

--
Cédric

[Tomás Pollak]

You can also watch this issue:
http://jira.opensymphony.com/browse/TESTNG-399

2010/8/10 Cédric Beust ♔ <ced...@beust.com>

[kamalakar vadla]

Hi All,

I was looking to into IRetryAnalyzer.

Following issues i am facing:

• With the below implemation my testmethod runs only 3 times eventhough i have given as max count as 6
• Other issue actually i am running only one method why it is showing as failed 3 methods???
• Suppose i have one testmethod and it rerun how many times it should give only latest run result. -- How we can achieve this?
• If possible can you send i sample code
  

     
Output:
---------------------------------------------------------------------------------------------------------
Parser] [WARN] Unknown value of attribute 'parallel' at suite level: ''.
[Parser] Running:
  E:\selenium World\sel+TestNG\testAutomation\temp-testng-customsuite.xml

in method1
mainError in testMethod1 with status 2 Retrying 0 times
mainError in testMethod1 with status 2 Retrying 1 times
in method1
mainError in testMethod1 with status 2 Retrying 2 times
mainError in testMethod1 with status 2 Retrying 3 times
in method1
mainError in testMethod1 with status 2 Retrying 4 times
mainError in testMethod1 with status 2 Retrying 5 times
in method1
FAILED: testMethod1
FAILED: testMethod1
FAILED: testMethod1

===============================================
    myPackage.Testing
    Tests run: 3, Failures: 3, Skips: 0
===============================================


===============================================
test+TestNG
Total tests run: 3, Failures: 3, Skips: 0
===============================================

[org.testng.internal.PoolService] Shutting down poolservice org.testng.internal.PoolService@18a7efd terminated:false
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------


Analyzer.java
-------------------------------------------------------------------------------------------
public class Analyzer extends RetryAnalyzerCount {
    private int count = 0;
    private final int MAX_COUNT = 6;
   
    public Analyzer() {
        setCount(MAX_COUNT);
    }

    @Override
    public boolean retryMethod(ITestResult result) {
        if (count < MAX_COUNT) {
            System.out.println(Thread.currentThread().getName()+"Error in " + result.getName() + " with status "
                    + result.getStatus() + " Retrying " + count + " times");
            count += 1;
            return true;
        }
        return false;
    }
   
------------------------------------------------------------------------------------

Test.java
---------------------------------------------------------------------------------------------------
public class Testing {


    @Test(retryAnalyzer=Analyzer.class)
    public void testMethod1() {
          System.out.println("in method1 " );
          Assert.fail();
    }
 
}

-------------------------------------------------------------------------------------------------


kamalakar
SocialTwist

[Cédric Beust ♔]

Kamalakar,

Please try the latest version (5.13.1), I fixed a few bugs in that area recently.

--
Cédric

[kamalakar vadla]

I tried with testing-5.13.1.jar but no luck same issues i am facing.

Could you please help me to solve this problem.
For details of my code refer below


-- VK

2010/8/12 Cédric Beust ♔ <ced...@beust.com>

[kamalakar vadla]

Hi Cédric,

Could you please help me to solve this issue.

VK
SocialTwist

[Jeegar]

We are facing similar issues but have figured out a hacky work-around.
You can override the retry method and set the test result to SKIP.
Then, TestNG doesn't mark the previous test as FAILED (if the last one
fails, it does mark it as FAILED which is correct).

Regarding the count, you gotta specify the number as 6 since it calls
retry method twice per retry.

But it would be nice to have the maxRetryCount feature without the
above mentioned hassle. If you feel so, please vote on the below
issue :
http://jira.opensymphony.com/browse/TESTNG-427

Jeegar Maru

On Aug 16, 9:39 am, kamalakar vadla <vkchary...@gmail.com> wrote:
> Hi Cédric,
>
> Could you please help me to solve this issue.
>
> VK
> SocialTwist
>

> On Fri, Aug 13, 2010 at 10:23 AM, kamalakar vadla <vkchary...@gmail.com>wrote:
>
> > I tried with testing-5.13.1.jar but no luck same issues i am facing.
>
> > Could you please help me to solve this problem.
> > For details of my code refer below
>
> > -- VK
>
> > 2010/8/12 Cédric Beust ♔ <ced...@beust.com>
>
> > Kamalakar,
>
> >> Please try the latest version (5.13.1), I fixed a few bugs in that area
> >> recently.
>
> >> --
> >> Cédric
>

> >> On Thu, Aug 12, 2010 at 6:17 AM, kamalakar vadla <vkchary...@gmail.com>wrote:
>
> >>> Hi All,
>
> >>> I was looking to into IRetryAnalyzer.
>
> >>> Following issues i am facing:
>

> >>>    - With the below implemation my testmethod runs only 3 times

> >>>    eventhough i have given as max count as 6

> >>>    - Other issue actually i am running only one method why it is showing
> >>>    as failed 3 methods???
> >>>       - Suppose i have one testmethod and it rerun how many times it

> >>>       should give only latest run result. -- How we can achieve this?

> >>>    - If possible can you send i sample code
>
> >>> *Output:*

> >>> *Analyzer.java*

> >>> On Tue, Aug 10, 2010 at 9:39 PM, Tomás Pollak <tpollak...@gmail.com>wrote:
>
> >>>> You can also watch this issue:
> >>>>http://jira.opensymphony.com/browse/TESTNG-399
>
> >>>> 2010/8/10 Cédric Beust ♔ <ced...@beust.com>
>
> >>>> Have you looked at IRetryAnalyzer?
>
> >>>>> --
> >>>>> Cédric
>

> >>>>> On Tue, Aug 10, 2010 at 7:04 AM, kamalakar vadla <vkchary...@gmail.com
> >>>>> > wrote:
>
> >>>>>>    - I ran 10 Testcases and in that  8 passed 2 failed  and so hudson

> >>>>>>    treat as build unstable and sends a mail. This send a false alaram

> >>>>>>    - Rerunning testing-failed.xml will not serve my purpose .
> >>>>>>    - Why i want to Rerun before the report generate is
> >>>>>>       - Most of my testcases communicate with the outside application

> >>>>>>       and there so many issues like network lag etc will effect testcase result.I
> >>>>>>       want to make sure once again by rerunning the tests and send the mail to the
> >>>>>>       team( These tests will run very frequently)

> >>>>>>       - So suggest me best solution for this problem?

>
> >>>>>> Kamalakar
> >>>>>> SocialTwist
>
> >>>>>> On Thu, Jul 1, 2010 at 10:36 AM, kamalakar vadla <
> >>>>>> vkchary...@gmail.com> wrote:
>
> >>>>>>> I am not sure how hudson will treat this rerun?? My only concern
> >>>>>>> if Hudson should treat this as build stable if at all after rerun if all
> >>>>>>> tests passed.
> >>>>>>> Lets try and see if it works for me or not?
>
> >>>>>>> Any thanks for quick reply
>
> >>>>>>> Kamalakar
> >>>>>>> Socialtwist
>
> >>>>>>> 2010/7/1 Cédric Beust ♔ <ced...@beust.com>
>
> >>>>>>> Why don't you run the second time with a different output directory?
> >>>>>>>> This way, your reports won't get overridden.
>
> >>>>>>>> --
> >>>>>>>> Cédric
>
> >>>>>>>> On Wed, Jun 30, 2010 at 9:49 PM, kamalakar vadla <
> >>>>>>>> vkchary...@gmail.com> wrote:
>
> >>>>>>>>> I can rerun the testng-failed.xml but the prob is i have
> >>>>>>>>> integrated these tests to Hudson CI so when u rerun the
> >>>>>>>>> reports accumulated will be Over written by rerun .

> >>>>>>>>> *My Requirement :*

> >>>>>>>>>>> testng-users...@googlegroups.com<testng-users%2Bunsu...@googlegroups.com>
> >>>>>>>>>>> .

> >>>>>>>>>>> For more options, visit this group at
> >>>>>>>>>>>http://groups.google.com/group/testng-users?hl=en.
>
> >>>>>>>>>> --
> >>>>>>>>>> Cédric
>
> >>>>>>>>>>  --
> >>>>>>>>>> You received this message because you are subscribed to the Google
> >>>>>>>>>> Groups "testng-users" group.
> >>>>>>>>>> To post to this group, send email to
> >>>>>>>>>> testng...@googlegroups.com.
> >>>>>>>>>> To unsubscribe from this group, send email to

> >>>>>>>>>> testng-users...@googlegroups.com<testng-users%2Bunsu...@googlegroups.com>
> >>>>>>>>>> .

> >>>>>>>>>> For more options, visit this group at
> >>>>>>>>>>http://groups.google.com/group/testng-users?hl=en.
>
> >>>>>>>>> --
> >>>>>>>>> Kamalakar.Vadla
> >>>>>>>>> Pramati Technologies Pvt Ltd
> >>>>>>>>> Begumpet
>
> >>>>>>>>>  --
> >>>>>>>>> You received this message because you are subscribed to the Google
> >>>>>>>>> Groups "testng-users" group.
> >>>>>>>>> To post to this group, send email to testng...@googlegroups.com
> >>>>>>>>> .

> >>>>>>>>> To unsubscribe from this group, send email to

> >>>>>>>>> testng-users...@googlegroups.com<testng-users%2Bunsu...@googlegroups.com>
> >>>>>>>>> .

> >>>>>>>>> For more options, visit this group at
> >>>>>>>>>http://groups.google.com/group/testng-users?hl=en.
>
> >>>>>>>> --
> >>>>>>>> Cédric
>
> >>>>>>>>  --
> >>>>>>>> You received this message because you are subscribed to the Google
> >>>>>>>> Groups "testng-users" group.
> >>>>>>>> To post to this group, send email to testng...@googlegroups.com.
> >>>>>>>> To unsubscribe from this group, send email to

> >>>>>>>> testng-users...@googlegroups.com<testng-users%2Bunsu...@googlegroups.com>
> >>>>>>>> .

> >>>>>>>> For more options, visit this group at
>

> ...
>
> read more »

[Cédric Beust ♔]

On Thu, Oct 7, 2010 at 12:06 PM, Jeegar <jeegar...@gmail.com> wrote:

Regarding the count, you gotta specify the number as 6 since it calls
retry method twice per retry.

I fixed this a little while ago, are you using the latest TestNG?


--
Cédric

[sabf]

can someone show me an example for that:

"We are facing similar issues but have figured out a hacky work-around. 
You can override the retry method and set the test result to SKIP.  "

Thanks,

sabf

[saikrishna]

HI All

Iam using TestNG 6.3.jar.

I have implemented Rerun by writing retry listener and RetryAnalyzer.

RetryListener.java

import java.lang.reflect.Constructor;

import java.lang.reflect.Method;

import org.testng.IAnnotationTransformer;

import org.testng.IRetryAnalyzer;

import org.testng.annotations.ITestAnnotation;

public class RetryListener implements IAnnotationTransformer {

@Override

public void transform(ITestAnnotation arg0, Class arg1, Constructor arg2,

Method arg3) {

IRetryAnalyzer retry = arg0.getRetryAnalyzer();

if (retry == null) {

arg0.setRetryAnalyzer(RetryAnalyzer.class);

}

}

}

RetryAnalyser.java

public class RetryAnalyzer implements IRetryAnalyzer {

private int count = 0;

private int maxCount = 2;

protected Logger log;

public RetryAnalyzer() {

setCount(maxCount);

}

@Override

public boolean retry(ITestResult result) {

if (!result.isSuccess()) {

if (count < maxCount) {

count++;

Reporter.log(Thread.currentThread().getName() + "Error in "

+ result.getName() + " with status "

+ result.getStatus() + " Retrying " + count + " times");

return true;

}

}

return false;

}

public void setCount(int count) {

maxCount = count;

}

}

I was able to rerun the failed testcase, but the in the report when the failed testcase is rerun for second time and it is passed, the passed testcase count is showing as Two , where as it should show One  . can some one clarify me on this?

Thanks

Saikrishna

[Krishnan Mahadevan]

SaiKrishna,

I would suggest that you please upgrade to TestNG 6.8 and retry this scenario.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

--

You received this message because you are subscribed to the Google Groups "testng-users" group.

To view this discussion on the web visit https://groups.google.com/d/msg/testng-users/-/h-kb_aDx5-AJ.

To post to this group, send email to testng...@googlegroups.com.

To unsubscribe from this group, send email to testng-users...@googlegroups.com.

[martino turturiello]

Hi sai krishna,

I would like to try this as well.. can you please give me some info on how to do it?

Cheers

2012/11/26 Krishnan Mahadevan <krishnan.ma...@gmail.com>

--
-----------------------------------------------------
Chi non ha apprezzato il suo maestro né la sua lezione, un giorno sarà forse colto, ma non sarà mai saggio.


Numero telefonico 09711835500
Skype: turturiellomartino

[saikrishna]

Hi Krishnan

I have tried out with testNG6.8.jar, I have a scenario where when my
testmethod fails, i should retry twice and then it should generate
report

the following is the testmethod i have ran

public void testMethod1() {
System.out.println("in method1 " );

if(count<2)
{
count++;
Assert.fail();
}
System.out.println("test passed");
}

so when my count is 2 my testcase passed, but in the report it is
showing testmethods passed three instead of one.


Thanks
Saikrishna
On 26 Nov, 17:15, martino turturiello <turturiellomart...@gmail.com>
wrote:
> Hi sai krishna <https://plus.google.com/u/0/101336265920473187050?prsrc=4>,

>
> I would like to try this as well.. can you please give me some info on how
> to do it?
>
> Cheers
>

> 2012/11/26 Krishnan Mahadevan <krishnan.mahadevan1...@gmail.com>

>
>
>
>
>
>
>
>
>
> > SaiKrishna,
> > I would suggest that you please upgrade to TestNG 6.8 and retry this
> > scenario.
>
> > Thanks & Regards
> > Krishnan Mahadevan
>
> > "All the desirable things in life are either illegal, expensive, fattening
> > or in love with someone else!"
>

[Сергей Охотник]

hi, guys, so had anyone found the way to rerun failed tests without writing first failures to the report?

[Martinho QA]

From what I have read and seen it appears that there still is not a supported way to rerun failed test without including them in the final report.

I am trying the hack mentioned above of setting the test to SKIP.

I have looked at one discussion where they were reprocessing the results, however, that looked painful.

So, let's ask TestNG folks - any chance you are planning on providing an approved way to say "only count the test once regardless of  number of retries"?

[Gangadhar Perineedi]

I hope following might be helpful to rerun failed test before generating report.

static int reTryCount=0;

@Test

public void test(){

    try{      
        test1(); // Actual test method

    }catch(AssertionError e){

        reTryCount++;

        if(reTryCount < 5){   // 5 is MaxReTryCount.

            test1();
        }