TDD, test doubles and the SOLID principles.

[Math Newman]

This is an issue that's been plaguing me for quite a bit and I'm struggling to reach a definitive conclusion.

The company I'm currently working for uses mocking frameworks in their tests and it brought about a pretty serious consequence, there is mocking framework "set up" code splattered all over the test classes.

Not only does this make the tests incredibly hard to read, especially for people unfamiliar with the framework, but it seems to make the tests more fragile because since none of the mocking code is re-used, changes to the production code can break many tests for the same reason.

The conclusion I've reached is that by using mocking frameworks instead of creating separate classes for their test doubles, they are violating the SRP principle because the test class is now responsible for the test double's logic and it has to perform the actual tests with them.

Would I be correct in thinking that in almost all cases, test doubles should reside in separate classes and that our tests should only be responsible for using the ones they need in for their specific tests?

Or is there something obvious that I'm missing?

[Norbert Nemes]

That's what I've been doing. 

I don't particularly like mocking frameworks so I create my own test doubles because they are just so darn easy to create. And they are different classes (one class per stub, dummy, spy etc)

Regards,

Norbert

[Andrew Coulton]

I agree, ever since I started writing test-specific doubles I find that my specifications are much easier to read and much more maintainable. It's also much easier to use automated refactorings (rename, change signature, etc) and similar code tools because the class is just another plain old class - the domain language of mocking frameworks generally means that there are method calls and usages the IDE can't find.

I usually implement doubles as-required in the same namespace as the test class (putting the code at the bottom of the same file) and then extract them to a global test\mock namespace if/when there's a reason to reuse a particular double elsewhere. Wherever possible, I try to implement separate well-named doubles for specific behaviour rather than producing a generic "mock" implementation that does everything - though these may share some behaviour through inheritance from a base class when appropriate.

Some of the more opinionated BDD tools make this quite difficult, we were using PHPSpec for a while but I found the strong coupling with their mocking framework was making it very hard to produce clean specifications. Perhaps it's an indicator of bad design on our part, but generally I felt like even simple things became very verbose or brittle quite quickly. We're gradually moving back to PHPUnit for that reason.

The nice thing about using dedicated mock classes is that you can build a testcase that creates the subject with dummies for all collaborators, and then only swap out the ones that are relevant for a particular scenario which means all of those specifications are fully isolated from any other changes to the class, collaborators, etc. In this simple example, you can see that it's unlikely the existing code would ever need to change if the subject gains extra collaborators, which wouldn't always be true with a mocking framework - especially one like PHPSpec that injects the mocks into the specification method itself.

class SomethingTest {

  protected $logger;

  public function test_it_is_initialisable()

  {

    $this->assertInstanceOf('Something', $this->newSubject());

  }

  public function test_it_logs_something()

  {

    // Using state verification style tests with a simple implementation of the interface

    $this->logger = new MemoryStringLoggerStub;

    $this->newSubject()->doSomething();

    $this->assertEquals('something', $this->logger->getLog());

  }

  public function test_it_ignores_logger_exceptions()

  {

    $this->logger = new ThrowingLoggerStub;

    $this->newSubject()->doSomething();

  }

  public function test_it_logs_something_else()

  {

    // Using mockist-style messaging verification

    $this->logger = new SpyingLogger;

    $this->newSubject()->doSomething();

    $this->logger->shouldHaveReceivedLog('something');

  }

  public function newSubject() 

  {

     return new Something($this->logger);

  }

  public function setUp()

  {

    $this->logger = new DummyLogger;

  } 

}

Andrew

[Jakob Holderbaum]

Hi Math,

I'd suggest the same as all the previous writers have:

The usage of dedicated Mock Frameworks kind of misses the point!

I think that a unit test case should always inspect a specific part of
the overall behavior of the unit it is inspecting. In my experiences, a
lot of people confuse this with "a unit test case should always inspect
a specific method of the unit it is inspecting".

Using mocking frameworks moves the focus in testing to "inspecting
methods" whereas it should be "inspecting behavior".

If you on the other hand construct specific test doubles, you are likely
to express the actual intent of the test.

Consider the following examples:

```
UserRegistration user_registration = mock(UserRegistration.class)
when(user_registration.addAccount(user_profile)).thenThrow(new
InvalidUserProfile());

// inject user_registration into the unit ...
```

vs

```
UserRegistration user_registration = FailingUserRegistration.new

// inject user_registration into the unit ...
```

The first one screams IMPLEMENTATION all over the place and nearly hides
the fact that you want to test behavior in an invalid case.

The other implementation tries to hide all the particular details from
the test and instead provides a tailored (and probably reusable) double.

I hope I could communicate my main reason why I am not using mocking
frameworks anymore. I feel that my code bases have become less brittle
and more readable.

Cheers
Jakob

On 02/17/2015 02:16 PM, Math Newman wrote:
> This is an issue that's been plaguing me for quite a bit and I'm struggling
> to reach a definitive conclusion.
>
>
> The company I'm currently working for uses mocking frameworks in their
> tests and it brought about a pretty serious consequence, there is mocking
> framework "set up" code splattered all over the test classes.
>
> Not only does this make the tests incredibly hard to read, especially for
> people unfamiliar with the framework, but it seems to make the tests more
> fragile because since none of the mocking code is re-used, changes to the
> production code can break many tests for the same reason.
>
>
> The conclusion I've reached is that by using mocking frameworks instead of
> creating separate classes for their test doubles, they are violating the
> SRP principle because the test class is now responsible for the test

> double's logic *and* it has to perform the actual tests with them.

>
> Would I be correct in thinking that in almost all cases, test doubles
> should reside in separate classes and that our tests should only be
> responsible for using the ones they need in for their specific tests?
>
> Or is there something obvious that I'm missing?
>

--
Jakob Holderbaum, M.Sc.
Embedded Software Engineer

0176 637 297 71
http://jakob.io/
http://jakob.io/mentoring/
h...@jakob.io
@hldrbm

[Peter Boszormenyi]

Well, you can move the mock creation into a factory:

UserRegistration newFailingUserRegistration() {
UserRegistration registration = mock(UserRegistration.class);
when(registration.addAccount(user_profile)).thenThrow(new InvalidUserProfile());
return registration;
}

And then, in the test:

UserRegistration user_registration = newFailingUserRegistration();

[Jakob Holderbaum]

Yeah, that is true. :)

Then the usage of a mocking framework would be the implementation detail
of your double.

I also tried this approach a few times. And it is a totally reasonable
approach.

I did not want to say that mocking frameworks are evil. It is just not
such a great idea to use them directly in the test. And if they can be
used to simplify certain test double implementations, than they are a
good choice.

Cheers
Jakob