[request] Test retry

[Tomás Pollak]

Hi Cedric,

I recently have been faced with a situation where I could have used a global "retry" parameter.
I understand that the @Test annotation can have a IRetryAnalyzer parameter, but I think its use is too complicated for what the normal use case is, forcing the clients to implement that interface and declaring it for all the tests.

It would ease the configuration a lot if we could pass a retryCount="2" to the suite xml file and/or to the maven plugin configuration.

Would this be difficult to implement?
Would other users in this mailing list also like this feature? (just reply with +1)

Cheers,
Tomás

[Roman Hiden]

+1
That would be a nice feature

[charris]

I would love this.

But I would need it to work correctly with DataProviders and Factories (ie. each instance of a TC would need to get X retries) and to only should a failure if all retries failed. I would like it to show TCs that failed then passed later as skips, or a new status of retried so I can look at them later, but mostly, I want to be able to see all passes if they passed in the end.

[shankar KC]

I added this functionality to our test framework. No need to add annotation to each method/class. I read a property and based on it i will add retryAnalyzer annotation to tests.

@Override

public void transform(ITestAnnotation annotation,

Class testClass,

Constructor testConstructor,

Method testMethod) {

if ((annotation.getRetryAnalyzer() == null)

&& (RetryAnalyzer.getMaxRerun() > 0)) {

logDebug(

tcID,

"Adding RetryAnalyzer.class to method "

+ testMethod.getName()

+ " as test suite properties has rerun count = "

+ RetryAnalyzer.getMaxRerun()+ " To enable rerun of failed test case");

annotation.setRetryAnalyzer(RetryAnalyzer.class);

}

}

static public int getMaxRerun() {

Integer rerunCount = 0;

String property = EnvironmentSpecificProperty.getProperty(

xxxxxxxx.base.common.Constants.NUMBER_OF_RERUNS,

tcID);

if (!StringUtils.isEmpty(property))

rerunCount = Integer.valueOf(property);

return rerunCount;

}

Hope it helps. Its working fine in our test framework.

Tests just use it as below :- in property file they just add

rerunNum=1

By default no retry. If we see this we retry that many attempts user specified.

For the kicks we have added a configurable sleeptime as well before rerunning tests incase if user want to wait before rerunning tests(OPTIONAL).

like :-

#delay in secs

rerunDelay=1

Thanks

Shankar KC