Strict mocks

[SimoneB]

Hello, I read that NSubstitute does not support strict mocks, it's been pointed out here as well, but although I agree that usually strict mocks is not what you should use I cannot deny that they can be very useful in some scenarios. In my case I would like to adopt NSubstitute on a somewhat large codebase that requires some mocks to be strict, therefore this limitation effectively prevents me from switching.

I would be glad to take the opportunity to fork the project and implement it, but I didn't look at the code thoroughly yet and would like to know if the author thinks it is feasible and how much effort it would be roughly. Also, if the answer is yes some pointers to where to start looking would be appreciated.

Thanks

[David Tchepak]

Hi Simone,

Would it be possible to provide a few example scenarios where you are looking at using strict mocks? We have some planned extensions to the current Received() functionality for checking call ordering that may cover some cases where strict mocks have traditionally been used.

Regards,

David

--
You received this message because you are subscribed to the Google Groups "NSubstitute" group.
To view this discussion on the web visit https://groups.google.com/d/msg/nsubstitute/-/WUhGKxoDSRUJ.
To post to this group, send email to nsubs...@googlegroups.com.
To unsubscribe from this group, send email to nsubstitute...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/nsubstitute?hl=en.

[SimoneB]

Hi David, thanks for your reply. Unfortunately the main scenario I'm dealing with does not relate to call ordering (I have implemented some extensions for that already*), but it requires strict mocks to really check that no calls except for those explicitly expected have been made.

* I have come up with this syntax for checking call order

sut.Do(1);

sut.DoSomethingElse(2);

sut.ReceivedOrdered(s => s.Do(1), s => s.DoSomethingElse(2));

[David Tchepak]

Would it be possible to modify your existing extensions to have .ReceivedOnly(...) or similar to verify only those calls were received?

I'd be keen to see the code you used for the extensions if you're happy to share it. :)

--
You received this message because you are subscribed to the Google Groups "NSubstitute" group.

To view this discussion on the web visit https://groups.google.com/d/msg/nsubstitute/-/0gE7lj82eUoJ.

[SimoneB]

On Wednesday, November 30, 2011 1:07:10 PM UTC+1, David Tchepak wrote:

Would it be possible to modify your existing extensions to have .ReceivedOnly(...) or similar to verify only those calls were received?

It would work, yes, but it would require that each time we would have to specify both expectations (if any) and verifications. Instead, we would like write just eventual expectations and simply fail if anything other than that was called. Which is in fact the strict mock behavior.

 

I'd be keen to see the code you used for the extensions if you're happy to share it. :)

Sure, I don't think you'll find it very useful as it's just a straightforward implementation I've been using as a proof of concept without modifying the project's source code. CallData class is simply a container value type for method and arguments.

public static void ReceivedOrdered<T>(this T substitute, params Expression<Action<T>>[] expectedCalls) where T : class

{

var receivedCalls = substitute.ReceivedCalls()

     .Select(c => new CallData(c.GetMethodInfo(), c.GetArguments()))

     .Reverse()

     .ToArray();

var expectedCallData = new List<CallData>();

foreach (var expectedCall in expectedCalls)

{

var callExpr = expectedCall;

var call = callExpr.Compile();

substitute.When(call)

 .Do(c => expectedCallData.Add(new CallData(((MethodCallExpression)callExpr.Body).Method, c.Args())));

 

call(substitute);

}

ArrayAssert.AreEqual(expectedCallData.ToArray(), 

                             receivedCalls.Where(expectedCallData.Contains).ToArray());

}

[Simone Busoli]

So I plan to go ahead and try to play with the code. Any advice is appreciated btw.

To view this discussion on the web visit https://groups.google.com/d/msg/nsubstitute/-/fGqD6PzO5pUJ.

[SimoneB]

David, I probably found a first showstopper, that is why I was wondering whether strict mocks were really feasible in NSubstitute.

Assuming there existed a syntax to create strict mocks, say:

fake = Substitute.StrictFor<ICalculator>();

there is no way to get this statement to fail as it is, which I would expect with a strict mock:

fake.Add(1, 2);

The reason is that there is no enough contextual information at that point to figure out whether that is an invocation - which should throw as there is no corresponding expectation - or is a future expectation, if for instance followed by .Returns(1).

This is quite straightforward if you reason about the syntax, but it was not obvious to me until now.

[David Tchepak]

Hi Simone,

I was just writing an email to you about this but you beat me to it. :)

You would need to look at a syntax or convention where you say "now I'm defining the strict calls I want". I think this would make the API more confusing overall, so would recommend pushing this into a Contrib project to try out first. I'm happy to look at changes to make NSubstitute easier to extend to support a Contrib project (say, an event or callback you can hook into that gets fired whenever a method on a sub is called. This will still have the issue of distinguishing actual calls from calls made in the process of stubbing via Returns().)

I'm still not quite clear on the difference between .ReceivedOny(x, y, z) during the Assert phase vs. .Expect(x, y, z) during the Arrange phase. Can you please explain it to me? It sounds like it might be our best bet at the moment. Thanks for your patience with me on this. :)

Cheers,

David

--

You received this message because you are subscribed to the Google Groups "NSubstitute" group.

To view this discussion on the web visit https://groups.google.com/d/msg/nsubstitute/-/d8GHiLmJDIsJ.

[Simone Busoli]

Before I start a flame let me clarify that I'm not willing to argue whether strict mocks make sense or not here. Assuming they do, the point of strict mocks is that you don't specify the calls you don't want to receive, rather you only specify the calls that you want to receive, if any, and expect that an exception will be thrown if any unexpected call is made. I'm not suggesting a syntax like .Expect as it would deviate from the current syntax and would be confusing, but using expectations rather than verifications lets you for instance set return values, or if you don't expect to receive any calls during a specific scenario just avoid writing anything as the strict mock would take care of throwing in case any call is made.

Does this clear it up a bit?

[David Tchepak]

Hi Simone,

Don't worry, I'm not going to debate the merits of strict mocks. I'm just trying to understand what you are trying to achieve to see if it is something NSubstitute can help you with. :)

If I understand correctly your goal is to only specify calls you want to receive, and throw if other calls are received without having to mention the explicitly. 

Let me try and go through a few cases around this goal so we can start talking concrete examples. I'm going to stick with .Expect() for now as I'm unsure how else this could work. Please correct any incorrect assumptions or conclusions I've made.

* Strict mock with no calls

var sub = Substitute.StrictlyFor<IFoo>();

Behaviour: throws if any calls received. 

Alternative with existing syntax: Assert.AreEqual(0, sub.ReceivedCalls().Count()); //throws during Assert

Alternative with hook for contrib: sub.ThrowOnAnyCall(); //throws immediately if a call is made

* Strict mock expecting one call

sub.Expect().Bar();

Behaviour: throws if more than one call to Bar(), or if other calls are received. Also need a Verify() to check the expectation was met (unless you don't want that behaviour, in which case you might prefer an AllowCall() or similar)

Possible alternative with AAA style with same behaviour (but also includes verify step):

  sub.ReceivedOnly(s => s.Bar()); 

  (or Received.Only(() => sub.Bar());... just hypotheticals here)

* Strict mock with stubbed values

sub.Expect().Zap().Returns(3);

Behaviour: sets expectation and stubs value. Throws as per previous example.

Cons: Doesn't work with current Returns() syntax as previously discussed. Need to chain with Expect or some other mode toggle.

Alternative with AAA:

  sub.ReceivedOnly(s => s.Zap());

Con with AAA style: have to duplicate stub value in the Arrange and the expectation in the Assert.

* Strict with multiple calls:

sub.Expect().Bar();

sub.Expect().Zap().Returns(42);

Behaviour: throws if multiple calls to Bar()/Zap(), throws if calls to other methods

Alternative with AAA: sub.ReceivedOnly(s => s.Bar(), s => s.Zap());

Again, AAA duplicates the Returns() and the assertion around the expected call.

From your emails it sounds like I'm missing some understanding and/or scenarios. If you can provide an example or some pseudocode for what you want to achieve that would be a great help to me. Based on my current understanding the main difference between traditional strict mocks and asserting AAA-style is when the exception is thrown (on the call during Act vs. during Assert). I'm interested in cases where this difference is important.

Cheers,

David

--
You received this message because you are subscribed to the Google Groups "NSubstitute" group.

[SimoneB]

Hi David, thanks a lot for your detailed email. There is really not much more to it than I have said and I don't think you missed anything. 

One scenario where strict mocks can be useful is when you are writing embedded software (realtime, high performance, ...) and you absolutely need to make sure that nothing you didn't explicitly expect has happened. That said, I generally like the .Expect call as it would make strict mocks feasible, and also prefer the main examples that you gave over the alternatives.

What I instead don't like very much is that it somewhat deviates from the current syntax and can become confusing, because you have to remember that normal mocks don't need Expect while strict ones do (this could be notified with an Exception, btw).

As you notice there are some variants of strict mocks which would need to be addressed too:

- throwing for calls made but not expected

- throwing for calls expected but not made

So far my primary concern was with the first, but there would need to be some syntax to deal with the second either. I think I would stick with verifying at the end of the test that the expectation was met, thus conforming to how  it is implemented currently and avoid introducing another variant solely for strict mocks.

[Slav Ivanyuk]

As an alternative syntax to setup expectations, perhaps something like this could work?

 

using( var expect = Substitute.SetupExpectations() )

{

             sub.Bar().Times( 10 ).Returns( ... ); // expected 10 calls

}

 

Verify would work the same.

Until "expect" is disposed any calls to substitute setup expectations.

This doesn't break existing or modify existing syntax.

 

Not sure how feasible or complicated it would be to add support for something like this. Or if you guys even like this approach.

 

Slav Ivanyuk

--

You received this message because you are subscribed to the Google Groups "NSubstitute" group.

To view this discussion on the web visit https://groups.google.com/d/msg/nsubstitute/-/ODl4218vCW0J.

[SimoneB]

Hello Slav, I would stay away from this syntax as it is somewhat obsolete and superseded by the AAA syntax.

[David Tchepak]

Thanks for your ideas Simone and Slav.

I'll look into this further when I'm adding in support for ordered calls as it could potentially use similar infrastructure.

If you'd like to help out feel free to fork and spike out some ReceivedOnly-style ideas, or if you'd like to try adding .Expect() style expectations then add that as a contrib project and let me know if you need any changes in the NSub internal API to help.

Cheers,

David

On Fri, Dec 2, 2011 at 2:17 AM, SimoneB <simone...@gmail.com> wrote:

Hello Slav, I would stay away from this syntax as it is somewhat obsolete and superseded by the AAA syntax.

--

You received this message because you are subscribed to the Google Groups "NSubstitute" group.

To view this discussion on the web visit https://groups.google.com/d/msg/nsubstitute/-/qrZr6OK7S3EJ.

[Anthony Egerton]

How about a strict verification at the end. 

var sub = Substitute.For<IFoo>();

sub.Foo();

sub.Received(1).Foo();

sub.DidNotReceiveAnyOtherCalls();

Cheers,

Anthony

On 02/12/2011, at 2:17 AM, SimoneB <simone...@gmail.com> wrote:

Hello Slav, I would stay away from this syntax as it is somewhat obsolete and superseded by the AAA syntax.

--

You received this message because you are subscribed to the Google Groups "NSubstitute" group.

To view this discussion on the web visit https://groups.google.com/d/msg/nsubstitute/-/qrZr6OK7S3EJ.

[David Tchepak]

Would that include calls stubbed out via Returns()?

Or just calls you checked were Received()?

[Anthony Egerton]

Would that include calls stubbed out via Returns()?

Yes

Or just calls you checked were Received()?

No

Any calls that you use to setup the sub or check were received are expected calls. 

Alternate name

sub.DidNotReceiveAnyUnexpectedCalls();

Cheers,

Anthony

[Will Green]

All this talk about strict mocks makes we wonder: why are you not using
RhinoMocks *for this scenario*? It has very mature support for strict
mocks. Use that, and move on to the next problem to be solved.

Personally, I started using NSubstitute because it wasn't RhinoMocks; the
simple things are dead simple, and the lack of strict mocks helps me to not
write over-specified, brittle tests. When I truly need strict mocks (which,
for me, is exceedingly rare), I reach for RhinoMocks.

Just my 2 cents.

--

Will Green

[Abi]

There is nothing stopping us. But it is good to have smaller technology stack. Currently we use NSub for general mocking and Moles for unmockables. It is team choice to decide on the balance between what is needed and smaller technology stack.

Sent from my iPhone

[David Tchepak]

Hi Will,

I'm not keen to add anything that compromises our goal of keeping the simple stuff simple. :)

But I do like to understand any issues people have while testing to see if we can find nice ways to help with those cases.

As an aside, if people aren't keen on Rhino Mocks I believe Moq and FakeItEasy also support strict mocks.

Regards,

David

[Abi]

Thanks David,

Simplicity of NSub is a big factor for us choosing NSub.

Sent from my iPhone