Upgraded to 1.8 and .Returns throws ArgMatchers exception on RunAll

[Andy Gagne]

If i run the unit test on its own, the unit test will work just fine. If i trigger a Run Al/Debug Alll unit tests in visual studio 2013 or Run All/Debug All using my resharper tools, I will get a single exception

    [TestFixture]
    [Category("Unit")]
    public class FakeTests
    {


        public interface IFake
        {
            int Length { get; }
        }
        [Test]
        public void FakeTesting()
        {
            const int reservedLength = 5;
            var mockField = Substitute.For<IFake>();
            mockField.Length.Returns(reservedLength); //<-- It will crash here. It will crash on any type


        }
    }

It will throw

An exception of type 'NSubstitute.Exceptions.UnexpectedArgumentMatcherException' occurred in NSubstitute.dll but was not handled in user code

Additional information: Argument matchers (Arg.Is, Arg.Any) should only be used in place of member arguments. Do not use in a Returns() statement or anywhere else outside of a member call.

Correct use:

  sub.MyMethod(Arg.Any<string>()).Returns("hi")

I checked to see if maybe i had conflicting DLLs but they are all

{NSubstitute, Version=1.8.0.0, Culture=neutral, PublicKeyToken=92dd2e9066daa5ca}.

I tried it on a small project but couldnt duplicate it... I'm not 100% certain what to do next...

Thanks!

Andrew

[David Tchepak]

Hi Andrew,

This was a change introduced in 1.8, designed to detect incorrect argument matcher use earlier and prevent hard to detect problems later. Unfortunately this means transitioning to 1.8 means dealing with the pain of any existing problems.

This exception means there is an `Arg.Any` or `Arg.Is` used in a place where it shouldn't be.

For example:

Incorrect use: sub.Method().Returns(Arg.Any<string>())  // doesn't return "any string"; NSub thinks you are specifying a call

Correct use:   sub.Method(Arg.Any<string>()).Returns("hi")

Alternatively there could be an argument matcher used with a non-virtual method, which will cause NSubstitute not to recognise it.

The way I debug these problems is:

• Check the failing test and test setup and make sure there is no arg matcher misuse (non-virtuals,`Arg.xyz` matcher not used to specify a call)
• Try to reduce the scope. Does the problem occur if we run all tests in a fixture? Just two tests in the fixture? Or multiple fixtures?
• Examine the test output/logs to work out what order tests are running (hopefully in the reduced scope we've identified in the previous step). The problem is most likely in the test or test fixture that runs immediately prior to the failing test.

Most of the time I've found the problem is with an earlier test in the test fixture, or in the fixture that runs immediately before the failing one.

Hope this helps. (If you get really stuck, you can email me the failing fixture and the fixture that runs before it and I'll see if I can track it down. Directly to my personal email if it is code you can't share publicly)

Regards,

David

--
You received this message because you are subscribed to the Google Groups "NSubstitute" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nsubstitute...@googlegroups.com.
To post to this group, send email to nsubs...@googlegroups.com.
Visit this group at http://groups.google.com/group/nsubstitute.
For more options, visit https://groups.google.com/d/optout.

[Andy Gagne]

Hi David,

Thank you so much for the advice...

Here's the culprit that was causing issues. 

            var things= new List<IThing> { Arg.Any<IThing>() };

I changed it to 

            var things = new List<IThing> { Substitute.For<IThing>() };

 I followed your steps and I found the issue by doing following:

1. Run All Tests - 1 Red

2. Sort by projects - Tested each group till one group went 1 Red

3. Removed all the the tests except for that project and then sorted by namespace

4. I then started running in batches and eliminating groups until i was down to 2 namespaces... 

5. Once i was down to 2 namespaces, i dug around the other namespace that wasnt going Red. I poked around the tests for any Arg.Any and I found the one i referenced above. 

Conclusion:

Just to validate the work, i ran both Visual Studio's Test Runner along with Resharpers Test Runner to make sure i was out of the woods.

I checked-in my work and my CI Build ran all tests successfully...

Thank you again for the great advice. I absolutely enjoy developing with NSub. 

[David Tchepak]

Glad you got it sorted! Thanks for posting the steps.

Sorry it was such a pain, but hopefully you'll get this error immediately if an arg matcher goes astray again, so it should be much easier to find in future.

Cheers,

David

[Samuel Langlois]

I had this error, found the cause and fixed it, but it was frustrating because:

1. It happened at random. Sometimes no test would fail, sometimes only one of the tests would fail, but it was not always the same.

2. It doesn't tell where the problem is. I had to look at every Arg.* usage.

I suppose it failed at random because the order in which the tests was run could change (my test runner is ncrunch and can run tests in parallel). But it is annoying that some future test can fail even though the problem is in a previous test.

Couldn't there be any extra info attached to the exception, helping to identify the problematic line of code?

Thanks,

Sam

[David Tchepak]

Hi Sam,

Sorry for the frustration. Were you using version 1.8? (it has slightly better detection of these cases)

One of the problems with providing a better exception is we can't always identify the problematic line of code. If arg matchers are queued up with a non-virtual call, NSubstitute won't know about it until it receives a new call to a virtual method. This makes getting the location of the line of code, both programmatically and manually, quite tough.

[Samuel Langlois]

Yes, version 1.8.1.

Thanks for the answer, and for your great work on NSubstitute. We really appreciate its simplicity and excellent documentation.

Sam