Extending Gallio for Retry

[Michael Mattera]

I've spent the last couple days trying to extend Gallio to allow repeats for our GUI automation.  I can get the tests to repeat using the repeat attribute however I'm having a tough time swallowing the errors.

For example, If a test is set to retry twice before failing:

Run #1: Fails

Run #2: Passes

Overall result in Icarus is passing, but the overall result in Gallio.Echo is failing.  Icarus is producing the desired results.

Icarus is looking at the return outcome from IPatternScope.TestBuilder.TestInstanceActions.RunTestInstanceBodyChain and showing passing.  However, Gallio.Echo is ignoring this outcome.  To get around this issue I used a TestRunner extension that would modify the Statistics from the report so that it would show failures as inconclusive unless it's the final run of the retry, which is working well for me. 

But the Repeat attribute and the TestRunnerExtension can't talk so it's impossible for me to notify the extension when the final run is so I'm forced to hard code the number of runs in both areas.  I tried modifying the Test MetaData, and parameters, and adding Context MetaData but the PatternTestInstanceState is different from the State class inside the runner which makes it seemingly impossible to send values.

Is there something I'm missing or would it be possible to add some kind of functionality where I can keep some kind of state and make my use-case work?

Final Use-case: I would like to retry tests X number of times until they pass or until X is hit.  Any failures before X is hit are ignored if the test passes.  If the test does not pass after X times then cause a failure.

Thank you