Mocking exceptions rising inside an async method

[Xavier Hahn]

Hello,

It's not specifically a NSubstitute question, but I'm writing tests for methods that are asynchronous and wondering if I'm doing the things correctly and if my tests are actually meaningful.

Specifically, I am testing an ApiController which has methods that are marked async, for example:

        public class CarController : ApiController
        {

             private ICarService carService;

             [...]

             public async Task<IEnumerable<Car>> Get()
             {
                 return await this.carService.GetAllAsync();
             }

        }

For reference, my ICarService is defined in this way:

    
        public interface ICarService
        {
             Task<IEnumerable<Car>> GetAllAsync();

             [...]
        }

I have defined a substitute for my CarService that has a GetAllAsync method that returns all my cars from the database like so:

        var carService = Substitute.For<ICarService>();
        cars = new List<Car> { new Car() { Name = "Car 1", Id = 0, Description = "Car n1"},
                               new Car() { Name = "Car 2", Id = 1, Description = "Car n2"},
                               new Car() { Name = "Car 3", Id = 2, Description = "Car n3"},
                               new Car() { Name = "Car 4", Id = 3, Description = "Car n4"} };
        carService.GetAllAsync().Returns(Task.FromResult(cars.AsEnumerable()));
 
        var subject = new CarController(carService);

Now, I'm using MSPec for the testing. I'd like to have a test that checks what happens if, in the call to the GetAllAsync method of the service, an exception is raised.

        public class When_I_call_get_and_the_service_throws_an_exception         

        {             

            Establish context = () => carService

                                        .When(x => x.GetAllAsync())

                                        .Do(x => { throw new Exception(); });

            Because of = () => result = Catch.Exception(() => subject.Get().Await());

            It should_throw_an_exception = () => result.ShouldBeOfExactType<HttpResponseException>();

            static Exception result;

        }

This seems to be working as the result has the exception that I expected, but I'm wondering if the test I'm doing is really meaningful and if I'm testing a real scenario.

I'm a bit confused as to how exceptions operates on async methods and if it would ever happen that the call to GetAllAsync() throws an exception directly.

As a resume, am I stubbing the call to GetAllAsync() correctly?

Should I create a task that would mimic what would happen in case of an exception in the async method? If so, how?

Thanks!

Xavier

[Jake Ginnivan]

Hey Xavier,

The simplest way is/quick answer is:

Establish context = () => carService.GetAllAsync().Returns(Task.Run(()=> { throw new Exception(); });

For a bit of an explanation. 

The framework guidelines are to not throw exceptions synchronously from a method which returns a Task, unless they are preconditions (i.e ArgumentExceptions etc). 

Here is a quick example of the difference of throwing synchronously vs asynchronous inside the task:

public Task ThrowSynchronously() { throw new Exception(); return Task.FromResult(0); }

var result = ThrowSynchronously(); // BOOM on this line

await result;

public async Task ThrowAsynchronously() { throw new Exception(); }

var result = ThrowSynchronously(); 

await result; // BOOM on this line

In your example, what you have done will work either way because you are not passing tasks around, but if more complex examples you want to be doing the later (which you will be when using async/await). 

If you use the solution at the top of this post, when this line executes:

Because of = () => result = Catch.Exception(() => subject.Get().Await());

The .Await() extension (which is part of MSpec) will call `.Wait()` on the running task, when .Wait() is called it will synchronously wait, once the task has faulted the exception to be thrown out of the task, the Await() extension method will catch the AggregateException then throw the inner exception which will be the exception you originally threw.

Hope that helps. If you have any follow up questions, or I haven't explained it well just asked.

- Jake

[Xavier Hahn]

Hey Jake,

Thanks for the answer.

I had the feeling that what I was doing wasn't correctly testing the scenario, but now it's much more clear.

I'm still having issues with that though, now if I write the Returns the way you wrote it, I get yelled at by NSubstitute for a reason I don't quite get, telling me that he cannot return value of type "UnwrapPromise for ICarService.GetAllAsync" and that he expected the type to be Task.

Here's the full error from the failed test:

Test Name: should throw an exception

Test FullName: Controllers.API.CarControllerTests+When_I_call_get_and_the_service_throws_an_exception::should_throw_an_exception

Test Source: c:\Projects\Git\Web.Tests\Controllers\API\CarControllerTests.cs : line 62

Test Outcome: Failed

Test Duration: 0:00:00.0020001

Result Message: Can not return value of type UnwrapPromise`1 for ICarService.GetAllAsync (expected type Task`1).

Make sure you called Returns() after calling your substitute (for example: mySub.SomeMethod().Returns(value)),

and that you are not configuring other substitutes within Returns() (for example, avoid this: mySub.SomeMethod().Returns(ConfigOtherSub())).

If you substituted for a class rather than an interface, check that the call to your substitute was on a virtual/abstract member.

Return values cannot be configured for non-virtual/non-abstract members.

Correct use:

 mySub.SomeMethod().Returns(returnValue);

Potentially problematic use:

 mySub.SomeMethod().Returns(ConfigOtherSub());

Instead try:

 var returnValue = ConfigOtherSub();

 mySub.SomeMethod().Returns(returnValue);

Result StackTrace: 

at NSubstitute.Core.ConfigureCall.CheckResultIsCompatibleWithCall(IReturn valueToReturn, ICallSpecification spec)

   at NSubstitute.Core.ConfigureCall.SetResultForLastCall(IReturn valueToReturn, MatchArgs matchArgs)

   at NSubstitute.Core.CallRouter.LastCallShouldReturn(IReturn returnValue, MatchArgs matchArgs)

   at NSubstitute.Core.SubstitutionContext.LastCallShouldReturn(IReturn value, MatchArgs matchArgs)

   at NSubstitute.SubstituteExtensions.Returns[T](MatchArgs matchArgs, T returnThis, T[] returnThese)

   at NSubstitute.SubstituteExtensions.Returns[T](T value, T returnThis, T[] returnThese)

   at Controllers.API.CarControllerTests.When_I_call_get_and_the_service_throws_an_exception.<.ctor>b__10() in c:\Projects\Git\Controllers\API\CarControllerTests.cs:line 59

I have the feeling that he gets confused by the Task, maybe I should create a partial substitute for the Task?

[Jake Ginnivan]

Try extracting the task to a variable. What version of .NET are you using?

 

That is super odd, I just tried this in linqpad:

 

void Main()

{

    var foo = Substitute.For<IFoo>();

    foo.Foo().Returns(Task.Run(() => { throw new Exception(); }));

   

    foo.Foo().Wait();

}

 

public interface IFoo

{

    Task Foo();

}

 

And it all worked as expected with foo.Foo().Wait() throwing the exception. Maybe try fully qualifying Task by using `System.Threading.Task.Run(() => { throw new Exception(); });`

--
You received this message because you are subscribed to a topic in the Google Groups "NSubstitute" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nsubstitute/8qP8tzARtFI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nsubstitute...@googlegroups.com.
To post to this group, send email to nsubs...@googlegroups.com.
Visit this group at http://groups.google.com/group/nsubstitute.
For more options, visit https://groups.google.com/d/optout.

[Xavier Hahn]

I'm using the .Net Framework 4.5.

I'm starting to wonder if it's not something about the synchronization context used during unit testing. I read somewhere that there was something special about it.

I modified my test in this way:

            Establish context = () =>
            {
                crashService = Substitute.For<ICarService>();
                var test = Task.Run(() => { throw new Exception(); });
                crashService.GetAllAsync().Returns(test);
            };

And interestingly (but annoyingly :) ), when I run the test in debug, it throws an exception at the line where I create the Task. 

You can see that for a reason I don't quite get, the debugger goes into the "throw new Exception" and it crashes directly.

Then the exception seems to be caught by MSpec and that is why this weird error is thrown.

[Jake Ginnivan]

The exception will be thrown inside the task, and the debugger will break into the task.

 

            Establish context = () =>

            {

                crashService = Substitute.For<ICarService>();

                var test = Task.Run(() => { throw new Exception(); });

                var test2 = test; // Set a breakpoint here, after the debugger breaks, continue, then this breakpoint should be hit. This shows that the Task.Run() method did not throw and the exception has been trapped inside the task.

                crashService.GetAllAsync().Returns(test);

            };

 

Talking about sync context, I am not a fan about the way MSpec has done its async support. When you mix sync contexts, async/await and .Wait() in the same spot you are in a very dangerous place because you can really easily deadlock your tests. But that is a much larger discussion, I have a blog post on the subject: http://jake.ginnivan.net/blog/2014/01/10/on-async-and-sync-contexts/

[Xavier Hahn]

OK, I think I understand what's going on.

You see, the Task my method returns is of type Task<IEnumerable<Car>>, but the Task.Run(() => { throw new Exception(); }); returns a Task (with no return type). Therefore, the "Returns" method of the Substitute object tries to typecast the Task into Task<IEnumerable<Car>> and crashes because he can't.

At least that seems to be the issue.

<p class="MsoNormal" style="background: whit

...

[Jake Ginnivan]

Ah.. Yeah, that is my fault..

 

You need:

 

Task.Run<IEnumerable<Car>>(() => { throw new Exception(); });

--

[Xavier Hahn]

Still not quite there...

The compiler is complaining that the call to the method is ambiguous between two methods.

The only workaround I found is to create a method with this signature:

            private static IEnumerable<Car> ThrowsException()
            {
                throw new Exception();
            }

And the creation of the substitute will then be this way:

	   
var test = Task
.Run<IEnumerable<Protocol>>(() => ThrowsException());

	   crashService = Substitute.For<IProtocolService>().GetAllAsync().Return(test);

This is working correctly.

            <span style="font-size:10.0pt;font-family:Consolas;color:#2B91AF

...