Keep track of flaky tests with NUnit

[Carlos Manuel Trepeu Pupo]

Hi,

I am currently using VSTests in TeamCity sending service messages to log the results of my tests. I do it this way because I have created an app that is able to re-run the failed tests. If it is the first failure, then I generate a fake testname like "Failed-testname" to keep a record of the flaky tests. That information is displayed in TeamCity as an additional test of my suite. My current environment is very unstable and that is why I am re-running all the tests that failed the first time. I know that using NUnit I can re-run all the failed tests, but I would like to keep a history of the tests that failed the first time. In order to do that I was thinking to use something to save the current status inside my retry function. I was reviewing the NUnit code online but I didn't find the way to do it. 

Is there any 'clean' way to do it? 

Is there any other approach that I should follow?

Thanks for your help.

This is my current retry function:

public override TestResult Execute(TestExecutionContext context)

{

int count = _retryCount;

while (count-- > 0)

{

try

{

context.CurrentResult = innerCommand.Execute(context);

}

catch (Exception ex)

{

if (context.CurrentResult == null) context.CurrentResult = context.CurrentTest.MakeTestResult();
                        // If this is the first time that the test failed, I would like to generate a nunit xml report and save it with a fake name (Failed-"NameOfTheTest")

context.CurrentResult.RecordException(ex);

}

if (context.CurrentResult.ResultState != ResultState.Error

&& context.CurrentResult.ResultState != ResultState.Failure

&& context.CurrentResult.ResultState != ResultState.SetUpError

&& context.CurrentResult.ResultState != ResultState.SetUpFailure

&& context.CurrentResult.ResultState != ResultState.TearDownError

&& context.CurrentResult.ResultState != ResultState.ChildFailure)

break;

// Clear result for retry

if (count > 0)

{

context.CurrentResult = context.CurrentTest.MakeTestResult();

context.CurrentRepeatCount++; // increment Retry count for next iteration. will only happen if we are guaranteed another iteration

}

}

return context.CurrentResult;

}