Keep track of flaky tests with NUnit

12 views
Skip to first unread message

Carlos Manuel Trepeu Pupo

unread,
Jul 24, 2020, 4:30:39 PM7/24/20
to NUnit-Discuss
Hi,

I am currently using VSTests in TeamCity sending service messages to log the results of my tests. I do it this way because I have created an app that is able to re-run the failed tests. If it is the first failure, then I generate a fake testname like "Failed-testname" to keep a record of the flaky tests. That information is displayed in TeamCity as an additional test of my suite. My current environment is very unstable and that is why I am re-running all the tests that failed the first time. I know that using NUnit I can re-run all the failed tests, but I would like to keep a history of the tests that failed the first time. In order to do that I was thinking to use something to save the current status inside my retry function. I was reviewing the NUnit code online but I didn't find the way to do it. 

Is there any 'clean' way to do it? 
Is there any other approach that I should follow?

Thanks for your help.

This is my current retry function:
public override TestResult Execute(TestExecutionContext context)
{

int count = _retryCount;

while (count-- > 0)
{
try
{
context.CurrentResult = innerCommand.Execute(context);
}
catch (Exception ex)
{
if (context.CurrentResult == null) context.CurrentResult = context.CurrentTest.MakeTestResult();
                        // If this is the first time that the test failed, I would like to generate a nunit xml report and save it with a fake name (Failed-"NameOfTheTest")
context.CurrentResult.RecordException(ex);
}

if (context.CurrentResult.ResultState != ResultState.Error
&& context.CurrentResult.ResultState != ResultState.Failure
&& context.CurrentResult.ResultState != ResultState.SetUpError
&& context.CurrentResult.ResultState != ResultState.SetUpFailure
&& context.CurrentResult.ResultState != ResultState.TearDownError
&& context.CurrentResult.ResultState != ResultState.ChildFailure)
break;

// Clear result for retry
if (count > 0)
{
context.CurrentResult = context.CurrentTest.MakeTestResult();
context.CurrentRepeatCount++; // increment Retry count for next iteration. will only happen if we are guaranteed another iteration
}
}

return context.CurrentResult;
}


Reply all
Reply to author
Forward
0 new messages