Repeat On Failure

[Eric Rini]

Okay, so in the world of Web and UI integration testing, sometimes even the best written test cases fail. They need to go through UI automation frameworks, web browsers, complicated test distribution servers, run on different browsers on different operating systems. The bottom line is that there's a lot of points of failure for any given test and sometimes when they fail, it's best to run them again before jumping to any conclusions.

Floating around the internet there is a popular block of code that extends MBUnit to adapt the Repeat(n) attribute into a RepeatOnFailure(n) attribute to address this issue.

http://trishkhoo.com/2011/07/giving-tests-a-second-chance/

http://devnet.jetbrains.com/thread/304906

When we watch these tests run in Icarus, they report success if it succeeds the first time or on any successive attempt. However when we run the tests with Echo, if the first test fails, a failure is reported as output... even if successive runs are successful  This means we are always searching through Gallio reports whenever we get a failure on our continuous integration dashboard.

So the question is, can anyone familiar with MBUnit internals help us adapt the TestDecoratorPatternAttribute described in the above websites to correctly report failure to MBUnit only after all test retry attempts are exhausted.