Mocks Suck

797 views
Skip to first unread message

Sageniuz

unread,
Dec 20, 2010, 12:33:25 PM12/20/10
to Growing Object-Oriented Software
Hi everybody,

because I'm a big fanboy of the book and I've learned so much by
reading it, I was a little bit shocked as I saw Brian Swan's
presentation title "Mocks Suck".

Here's the link to the presentation:
http://www.testingtv.com/2010/12/20/mocks-suck-and-what-to-do-about-it/

I'd really recommend watching this presentation and would love to
discuss all Brian's criticism about "Mockist TDD".

Has somebody already seen the presentation? What are your thoughts?

Steve Freeman

unread,
Dec 20, 2010, 12:44:04 PM12/20/10
to growing-object-o...@googlegroups.com
I worked through the slides a while ago and was as impressed as you'd expect me to be. As usual with these cases, he appears to be attacking us for stuff we don't do.

The underlying issue, which I only really understood while writing the book, is that there are different approaches to OO. Ours comes under what Ralph Johnson calls the "mystical" school of OO, which is that it's all about messages. If you don't follow this, then much of what we talk about doesn't work well.

That said, I would also expect the balance of practices to be different in a dynamic language where we can't rely on the language to keep interfaces in sync--although the original ideas came from a(nother) bunch of Smalltalkers.

S.

Matt Wynne

unread,
Dec 20, 2010, 1:12:42 PM12/20/10
to growing-object-o...@googlegroups.com
I know Brian, and in fact I recently did a 'head to head' session at ScotRUG where we solved the same problem in our different styles. Both recitals took about half an hour, links here: http://blog.mattwynne.net/2010/08/31/outside-in-vs-inside-out-comparing-tdd-approaches/

My approach is, I hope, much more akin to what you'd pick up from reading GOOS. I'd say a significant part of Brian's scepticism about mocks is mostly because he hasn't ever worked with anyone who knows how to use them properly. He certainly seemed to be intrigued by the idea from watching my session that you didn't need to keep scurrying off down a rabbit warren each time you needed a collaborator class.

If you watch the videos, you'll also see we ended up with quite different designs, even for such a simple problem.

cheers,
Matt

ma...@mattwynne.net
07974 430184

Uberto Barbini

unread,
Dec 20, 2010, 1:43:49 PM12/20/10
to growing-object-o...@googlegroups.com

Looking at the presentation, it reminds me of a classical dialect
techniques: make your opponent ideas idiotic and then have fun to
demolish them.
It's used by politician everywhere, I hope this is not the start of a
new trend in tech conference.

I think it's completely ok dislike mocks, it's also perfectly ok to
ignore them, but if you present a session about mocks and you
misrepresent them in this way either you do on purpose or you didn't
any preparation for the session.

Said that, I agree that sometimes it's difficult (at least for me) to
choose how much mocking and where cut the layers... but you can always
refactor later.

cheers

Uberto

Brian Swan

unread,
Dec 29, 2010, 12:39:28 PM12/29/10
to Growing Object-Oriented Software
Hi all,

I'd be happy to discuss why I dislike using mocks and prefer to use
stubs and spies in the cases where I want to verify that specific
messages were sent to collaborating objects.

I've been doing TDD for over 10 years and have seen numerous
situations where overusing mocks leads to brittle, hard to read test
cases. Having moved from Java to Ruby over that last couple of years
I'm seeing a lot of the problems I saw in Java code bases reappearing
in Ruby code bases. The goal of the presentation was to highlight
these problems, albeit by taking an extreme position.

I don't believe I particularly misrepresented mockist TDD in the
presentation but certainly emphasised the aspects that I dislike.

I completely agree with Steve's comment above that there are different
styles of OO (and TDD) and that if you don't follow the GOOS OO design
style much of the mock advice won't work well. What I find in practice
is that many teams don't recognise this and use mocks inappropriately.

By taking an extreme position in my presentation I hope that people
will discuss both views and as a result refine the advice on when to
use (or not use) mocks.

Regards

Brian

Uberto Barbini

unread,
Dec 29, 2010, 5:50:58 PM12/29/10
to growing-object-o...@googlegroups.com
Hi, Brian
I agree with pretty everything you wrote here, but I think you should
try to explain 2 things:

1. where exactly stubs and spies are better than mocks and why.
2. (more important) everything can be misused, so do you see something
wrong with the "correct" way to use mocks (let's take GOOS as
example).

> style much of the mock advice won't work well. What I find in practice
> is that many teams don't recognise this and use mocks inappropriately.

In my experience, most of teams don't use TDD at all. Most of the
misuses of mocks I saw they were on teams who wrote the test later or
in teams who were forced to use overcomplex architectures.

Said that, taking extreme positions is fine for me, but then what else
could you expect if not extreme reactions? ;)

cheers

Uberto

Steve Freeman

unread,
Dec 30, 2010, 4:23:01 AM12/30/10
to growing-object-o...@googlegroups.com
Thanks for engaging with the list. So...

"Hammers suck. They strip the threads from my screws."

Yes, a lack of skill will lead to brittle code and tests. And there are plenty of broken TDD codebases that don't use mocks. And even more broken codebases that don't use TDD. No technique can survive inadequately trained developers.

I only had the slides to go on, so I don't know if the words altered the position. Personally, I'm very, very tired of "mocks are bad because X", "Yes, that's why we never do X" arguments. There's a lot of misunderstanding in the world, much of it perpetrated by Big Names, which leads to exactly the sort of mess you've been seeing. If the teams that you were working with had been Listening to the Tests, rather than just typing furiously, then they wouldn't have gone down that hole.

It might have been more productive (if less noticeable) to have done a symptoms and cures talk. To me, taking an extreme anti-mocks position is getting a bit old.

S.

Steve Freeman

unread,
Dec 30, 2010, 4:32:18 AM12/30/10
to growing-object-o...@googlegroups.com
On 29 Dec 2010, at 18:39, Brian Swan wrote:
> I'd be happy to discuss why I dislike using mocks and prefer to use
> stubs and spies in the cases where I want to verify that specific
> messages were sent to collaborating objects.

personally, I don't mind that much whether interactions are specified before or after. I prefer before because that's what I'm used to and it looks a bit like a poor man's Design by Contract.

What does concern me is that all the interactions should be verified unless I turn them off explicitly. If the behaviour of the object under test changes, than I want my test to have changed too (preferably beforehand). If I need to default or ignore a significant proportion of the interactions then that's a clue that something's too big.

I would guess that there's an interesting project in the dynamic languages to write a tool that reads the expectations and cross-checks the signatures with candidate classes. In Smalltalk these could be defined as protocols. Ruby's less regular, so it would be harder to be consistent.

S.

Brian Swan

unread,
Dec 30, 2010, 12:31:26 PM12/30/10
to Growing Object-Oriented Software
Hi Uberto,

> 1. where exactly stubs and spies are better than mocks and why.

Better is obviously subjective but for me stubs and spies let me do
the same things you can do with mocks but without what I consider to
be the disadvantages of mocks.

I prefer that stubs are distinctly providing input and that spies are
checking indirect outputs. With mocks I find the distinction between
input and output less obvious (although I notice that JMock has added
'allowing' to add more distinction).

I also prefer that spies allow me to put asserts after the invocation
of the method being tested, I consider it to be much more readable and
again makes a clearer distinction between input and output.

> 2. (more important) everything can be misused, so do you see something
> wrong with the "correct" way to use mocks (let's take GOOS as
> example).

I don't see anything wrong with the GOOS 'correct' way of using mocks
but I think what people generally don't appreciate is the relationship
between using mocks and the GOOS design style. What I find is that if
your design style differs, or is constrained by using a framework,
then mocking can lead to the brittle, unreadable test problems that I
have often seen.

Brian

Brian Swan

unread,
Dec 30, 2010, 1:01:03 PM12/30/10
to Growing Object-Oriented Software
> If the behaviour of the object under test changes, than I want my test to have changed too (preferably beforehand).

I too would expect to change the test first if the behaviour of the
object under test changes. However the failure mode that most concerns
me with using mocks is when the behaviour doesn't change but a
refactoring of the object under test results in test failures because
the mock is effectively duplicating the implementation.

> If I need to default or ignore a significant proportion of the interactions then that's a clue that something's too big.

I agree that people don't "listen to the tests" enough. In response to
problem tests the options to consider also include changing the
testing style to a more stateful style.

Brian

J. B. Rainsberger

unread,
Dec 30, 2010, 1:58:38 PM12/30/10
to growing-object-o...@googlegroups.com
On Wed, Dec 29, 2010 at 12:39, Brian Swan <bgs...@gmail.com> wrote:

I'd be happy to discuss why I dislike using mocks and prefer to use
stubs and spies in the cases where I want to verify that specific
messages were sent to collaborating objects.

Spies and mocks do the same thing, so I don't understand this statement. Spies only differ from mocks in when you set the expectations: spies after the action, mocks before.

I've been doing TDD for over 10 years and have seen numerous
situations where overusing mocks leads to brittle, hard to read test
cases.

Overusing spies results in the same problems.
 
Having moved from Java to Ruby over that last couple of years
I'm seeing a lot of the problems I saw in Java code bases reappearing
in Ruby code bases. The goal of the presentation was to highlight
these problems, albeit by taking an extreme position.

I don't believe I particularly misrepresented mockist TDD in the
presentation but certainly emphasised the aspects that I dislike.

I completely agree with Steve's comment above that there are different
styles of OO (and TDD) and that if you don't follow the GOOS OO design
style much of the mock advice won't work well. What I find in practice
is that many teams don't recognise this and use mocks inappropriately.

I have found that using verifying test doubles (spies and mocks both) as opposed to non-verifying test doubles (stubs) creates positive pressure on your design by highlighting unhealthy dependencies. Verifying test doubles encourage me to do something about the unhealthy dependencies, whereas non-verifying test doubles help me tolerate them. When designing new elements, I prefer my tools to encourage me to fix design problems early, so that I don't let them linger and grow; when changing legacy code, I prefer my tools to let me decide which of the many existing design problems I want to address at the moment.

As a result, I posit two central inappropriate uses of verifying test doubles:

1. When trying to change code with existing, deep design problems,
2. When modeling queries, rather than commands

By taking an extreme position in my presentation I hope that people
will discuss both views and as a result refine the advice on when to
use (or not use) mocks.

How clear have you made that intent in your presentation? I haven't read it, so don't interpret that as a passive aggressive question, because I genuinely don't know.
-- 
J. B. (Joe) Rainsberger :: http://www.jbrains.ca :: http://blog.thecodewhisperer.com
Diaspar Software Services :: http://www.diasparsoftware.com
Author, JUnit Recipes
2005 Gordon Pask Award for contribution to Agile practice :: Agile 2010: Learn. Practice. Explore.

Brian Swan

unread,
Dec 30, 2010, 4:48:47 PM12/30/10
to Growing Object-Oriented Software
>
> Spies and mocks do the same thing, so I don't understand this statement.
> Spies only differ from mocks in when you set the expectations: spies after
> the action, mocks before.

Spies and mocks do the same thing but in different ways. I find where
the expectations are set to affect the readability of the test and
that asserting on a spy after the action to be more natural and
readable. In addition a mock will fail if the implementation invokes a
method on the collaborator that isn't specified in the test, whether
or not I care about checking it. With a spy I can make the choice of
what I want to assert in the test, making the test less brittle in the
face of refactoring the implementation.

>
> Overusing spies results in the same problems.
>

I find less so as a result of the asserts being limited to those
methods on the collaborator that I care about checking rather that
having to specify all (or explicitly ignore) the methods invoked by
the implementation on the mock.

> I have found that using verifying test doubles (spies and mocks both) as
> opposed to non-verifying test doubles (stubs) creates positive pressure on
> your design by highlighting unhealthy dependencies.

This is an aspect in the use of mocks that I genuinely don't "get",
the influence on the design that comes from using mocks. I've always
had an interest in different techniques for software design and would
like to understand this aspect more. What I'm not sure about is how
much mocks influence the design versus mocks are useful when testing
code written in a certain style.

>
> As a result, I posit two central inappropriate uses of verifying test
> doubles:
>
> 1. When trying to change code with existing, deep design problems,
> 2. When modeling queries, rather than commands
>

I like this advice. I remember something from Bertrand Meyers Object
Oriented Software Construction, his 'Advice to the advisors' about
giving 'always' or 'never' or qualified 'always' or 'never' advice.

What I'd like to find is the qualified advice on when to use mocks,
which lies somewhere between always and never.

> By taking an extreme position in my presentation I hope that people
>
> > will discuss both views and as a result refine the advice on when to
> > use (or not use) mocks.
>
> How clear have you made that intent in your presentation? I haven't read it,
> so don't interpret that as a passive aggressive question, because I
> genuinely don't know.
> --

It's not made clear in the presentation.

Brian

Nat Pryce

unread,
Dec 30, 2010, 7:58:10 PM12/30/10
to growing-object-o...@googlegroups.com
On 30 December 2010 21:48, Brian Swan <bgs...@gmail.com> wrote:
> I like this advice. I remember something from Bertrand Meyers Object
> Oriented Software Construction, his 'Advice to the advisors' about
> giving 'always' or 'never' or qualified 'always' or 'never' advice.
>
> What I'd like to find is the qualified advice on when to use mocks,
> which lies somewhere between always and never.

That's what we set out to do in the book.

When writing the book we were very careful to avoid any absolute
statements and to describe *our experience* of using a variety of TDD
tools (including mocks), and the context in which we found the tools
useful. We deliberately did not write in absolute terms ("mocks are
great" or "mocks suck", for example), because any tool is applicable
in some context. Outside that context any tool becomes less helpful
and, far enough away from the sweet spot, a hindrance. We wanted to
let the readers understand enough about the contexts we have
experienced that they could hopefully map them to their own contexts
and make informed decisions as to which tools are appropriate for
them. Anything else is, I think, a disservice to the audience and just
continues the parlous state of debate and education the industry in
which technologies are described almost like fundamentalist religious
beliefs instead of in terms of engineering trade-offs. No technology
is bad, it's all context dependent -- even Spring might be useful
somewhere, maybe*...

--Nat

* only kidding, Spring fans!

--
http://www.natpryce.com

Nat Pryce

unread,
Dec 30, 2010, 8:07:08 PM12/30/10
to growing-object-o...@googlegroups.com
On 30 December 2010 17:31, Brian Swan <bgs...@gmail.com> wrote:
> I prefer that stubs are distinctly providing input and that spies are
> checking indirect outputs. With mocks I find the distinction between
> input and output less obvious

That, I think, is the key difference in thinking. You're designing in
terms of inputs and outputs, mock objects (and jMock in particular)
aid designing in terms of state machines that communicate by message
passing protocols. In that model, a state transition is triggered by
an event (incoming message) and causes some observable effects
(outgoing messages). The transition rule -- trigger, effects, new
state -- is a single "chunk" of behaviour.

> (although I notice that JMock has added 'allowing' to add more distinction).

JMock has always had that feature, although it was called "stubs" in jMock 1.

--Nat

--
http://www.natpryce.com

Nat Pryce

unread,
Dec 30, 2010, 8:22:48 PM12/30/10
to growing-object-o...@googlegroups.com
On 29 December 2010 17:39, Brian Swan <bgs...@gmail.com> wrote:
> I don't believe I particularly misrepresented mockist TDD in the
> presentation but certainly emphasised the aspects that I dislike.

Sorry to repeat myself again (and again (and again ...))

THERE IS NO SUCH THING AS MOCKIST TDD!

There are different ways of designing how a system is organised into
modules and the interfaces between those modules. In one design, you
might split the system into modules that communicate by leaving data
in shared data structures. In another, you might split the system
into modules that communicate by synchronous message-passing. In
another, modules might communicate by lazily calculated streams, or by
async messaging between concurrent actors, or by content-based pub/sub
events, or...

Depending on how you design inter-module interfaces, you'll need
different tools to unit-test modules (whether TDD or not).

Mock Objects are designed for test-driving code that is modularised
into objects that communicate by "tell, don't ask" style message
passing. In that style, there is little visible state to assert about
because state is an implementation detail -- housekeeping used to
coordinate the message-passing protocols.

In other styles of code, procedural or functional, assertions on
queried state works fine. (In asynchronous code, they do not because
of race conditions).

Even in a system that uses Mock Objects appropriately in its
unit-tests, a significant part is purely functional in my experience,
and so will not be tested with mock objects.

Talking about Mockist TDD vs Statist TDD leads people to think in
terms of adopting a "TDD style" and rigidly following a process,
rather than understanding how to select from a wide array of tools
those that are appropriate for the job at hand.

Rant over (this time!).

--Nat
--
http://www.natpryce.com

Dale Emery

unread,
Dec 31, 2010, 12:28:32 AM12/31/10
to growing-object-o...@googlegroups.com
Hi Brian,

I find where the expectations are set to affect the readability of the test and that asserting on a spy after the action to be more natural and readable.

I too prefer to describe the desired results after the action that leads to the results. It nicely fits the way I think about responsibilities: some context exists, some stimulus or event occurs in that context, and some object has the obligation to produce one or more planned results in response.

Describing responsibilities in other orders always feels awkward to me, as in the old joke: If I had some bread I could make a ham sandwich, if I had some ham.

In addition a mock will fail if the implementation invokes a method on the collaborator that isn't specified in the test, whether or not I care about checking it.

I don't think there's anything about mocks that says they must by default fail when other-than-specified methods are called. I've used mocking frameworks that allowed other calls by default; in order to convince them to reject other calls, you had to put them into "strict" mode. So perhaps your complaint is about a particular mocking framework, and not about mocks per se.

This is an aspect in the use of mocks that I genuinely don't "get", the influence on the design that comes from using mocks.

I tend to think of the influence as being in the other direction: My design choices often create collaborations in which mocks are useful.

It's possible that the influence also sometimes flows the other way, and I'm not aware of it.

What I'd like to find is the qualified advice on when to use mocks, which lies somewhere between always and never.

My personal guideline is this: When I give an object the responsibility to send a message to another object, mocks offer a nice way to describe that responsibility.

Dale

--
Dale Emery
Consultant to software teams and leaders
Web: http://dhemery.com

Dale Emery

unread,
Dec 31, 2010, 12:30:30 AM12/31/10
to growing-object-o...@googlegroups.com
Hi Joe,

I have found that using verifying test doubles (spies and mocks both) as opposed to non verifying test doubles (stubs) creates positive pressure on your design by highlighting unhealthy dependencies.

Can you give an example or two?

Dale

Steve Freeman

unread,
Dec 31, 2010, 4:54:09 AM12/31/10
to growing-object-o...@googlegroups.com
On 30 Dec 2010, at 19:01, Brian Swan wrote:
>> If the behaviour of the object under test changes, than I want my test to have changed too (preferably beforehand).
>
> I too would expect to change the test first if the behaviour of the
> object under test changes. However the failure mode that most concerns
> me with using mocks is when the behaviour doesn't change but a
> refactoring of the object under test results in test failures because
> the mock is effectively duplicating the implementation.

Then that might a symptom of unbalanced relationships between the object and its collaborators, or that you should be using real objects in the test. One clue is if you're mocking out setters and getters, which suggests there isn't really an action in the collaboration.

>> If I need to default or ignore a significant proportion of the interactions then that's a clue that something's too big.
>
> I agree that people don't "listen to the tests" enough. In response to
> problem tests the options to consider also include changing the
> testing style to a more stateful style.

That might be an answer, or perhaps you're looking at the wrong interactions. I often find that tests that involve mocks are actually exercising a small cluster of objects: a small container and some helper and value objects. The focus should be on interactions that trigger changes in the environment.

S.

Steve Freeman

Winner of the Agile Alliance Gordon Pask award 2006
Book: http://www.growing-object-oriented-software.com

+44 (0) 797 179 4105
M3P Limited. http://www.m3p.co.uk
Registered office. 2 Church Street, Burnham, Bucks, SL1 7HZ.
Company registered in England & Wales. Number 03689627

Steve Freeman

unread,
Dec 31, 2010, 5:01:37 AM12/31/10
to growing-object-o...@googlegroups.com
On 30 Dec 2010, at 18:31, Brian Swan wrote:
> I prefer that stubs are distinctly providing input and that spies are
> checking indirect outputs. With mocks I find the distinction between
> input and output less obvious

That's a reasonable position. I prefer to think in terms of protocols between objects, so I like to keep the interactions together. Not a show stopper either way.

> (although I notice that JMock has added 'allowing' to add more distinction).

allowing has been there since the beginning. It was called stubbing in jMock 1, but Nat changed the terminology. I'm not sure where he took it from, but I know that it's used in Syntropy in event specifications. We've been banging on about "Stub queries, mock actions" for some years.

I think this also why Easymock has had such a pernicious effect, since it makes that distinction at the end of the line where it's hard to see.

S.

Uberto Barbini

unread,
Dec 31, 2010, 5:32:06 AM12/31/10
to growing-object-o...@googlegroups.com
>> 1. where exactly stubs and spies are better than mocks and why.
>
> Better is obviously subjective but for me stubs and spies let me do
> the same things you can do with mocks but without what I consider to
> be the disadvantages of mocks.

still not sure what you mean with "disadvantages of mocks".
Also there are differences between (for example) mockito and JMock in
practical use.

> I also prefer that spies allow me to put asserts after the invocation
> of the method being tested, I consider it to be much more readable and
> again makes a clearer distinction between input and output.

I need to see the code here to understand what you mean.
Generally speaking I'm not very concerned by the "distinction between
input and output", I'm interested in testing the behaviour as a whole.

> I don't see anything wrong with the GOOS 'correct' way of using mocks
> but I think what people generally don't appreciate is the relationship
> between using mocks and the GOOS design style. What I find is that if
> your design style differs, or is constrained by using a framework,
> then mocking can lead to the brittle, unreadable test problems that I
> have often seen.

As Steve put it: "No technique can survive inadequately trained developers."
Anyway my target is improving my team skills to go near the GOOS
design style model.
So I want for them to learn to use mocks in the correct way, nothing less. ;)

Cheers

Uberto

Esko Luontola

unread,
Dec 31, 2010, 6:56:13 AM12/31/10
to Growing Object-Oriented Software
On Dec 31, 12:32 pm, Uberto Barbini <ube...@ubiland.net> wrote:
> > Better is obviously subjective but for me stubs and spies let me do
> > the same things you can do with mocks but without what I consider to
> > be the disadvantages of mocks.
>
> still not sure what you mean with "disadvantages of mocks".
> Also there are differences between (for example) mockito and JMock in
> practical use.

Mockito creates spies, not mocks. That is what makes it different from
JMock.

http://code.google.com/p/mockito/wiki/FAQ

> > I also prefer that spies allow me to put asserts after the invocation
> > of the method being tested, I consider it to be much more readable and
> > again makes a clearer distinction between input and output.
>
> I need to see the code here to understand what you mean.
> Generally speaking I'm not very concerned by the "distinction between
> input and output", I'm interested in testing the behaviour as a whole.

Here is an example:
https://github.com/orfjackal/tdd-tetris-tutorial/blob/beyond/src/test/java/tetris/RemovingFullRowsTest.java

Uberto Barbini

unread,
Dec 31, 2010, 9:11:03 AM12/31/10
to growing-object-o...@googlegroups.com
On Fri, Dec 31, 2010 at 12:56 PM, Esko Luontola <esko.l...@gmail.com> wrote:
> On Dec 31, 12:32 pm, Uberto Barbini <ube...@ubiland.net> wrote:
>> > Better is obviously subjective but for me stubs and spies let me do
>> > the same things you can do with mocks but without what I consider to
>> > be the disadvantages of mocks.
>>
>> still not sure what you mean with "disadvantages of mocks".
>> Also there are differences between (for example) mockito and JMock in
>> practical use.
>
> Mockito creates spies, not mocks. That is what makes it different from
> JMock.
>
> http://code.google.com/p/mockito/wiki/FAQ

technically speaking yes, but given its name, all its method names I
though it was included in the "mocks suck".

So at the end it's all about JMocks vs. Mockito? how boring...

>> I need to see the code here to understand what you mean.
>> Generally speaking I'm not very concerned by the "distinction between
>> input and output", I'm interested in testing the behaviour as a whole.
>
> Here is an example:
> https://github.com/orfjackal/tdd-tetris-tutorial/blob/beyond/src/test/java/tetris/RemovingFullRowsTest.java

I don't see the point. Why using JMocks instead of mockito in that
class would result in less brittle tests and better design?

cheers

Uberto

Brian Swan

unread,
Dec 31, 2010, 9:40:08 AM12/31/10
to Growing Object-Oriented Software

On Dec 31, 12:58 am, Nat Pryce <nat.pr...@gmail.com> wrote:
> Outside that context any tool becomes less helpful
> and, far enough away from the sweet spot, a hindrance. We wanted to
> let the readers understand enough about the contexts we have
> experienced that they could hopefully map them to their own contexts
> and make informed decisions as to which tools are appropriate for
> them.

Unfortunately what I've seen in many of the teams I've worked with is
a much less nuanced approach to using mock objects than that presented
in the GOOS book, similar to the pattern explosion that is often seen
after reading the GoF book.

> Anything else is, I think, a disservice to the audience and just
> continues the parlous state of debate and education the industry in
> which technologies are described almost like fundamentalist religious
> beliefs instead of in terms of engineering trade-offs.

I agree, it's unfortunate that I had to use such a blunt instrument to
get people to notice and to have this discussion. I'd really like to
see more tools and techniques being upfront about their sweet spot/
trade offs, it's what first attracted me to both XP and Ruby on Rails.

Brian

Brian Swan

unread,
Dec 31, 2010, 10:09:56 AM12/31/10
to Growing Object-Oriented Software
On Dec 31, 1:07 am, Nat Pryce <nat.pr...@gmail.com> wrote:
>
> That, I think, is the key difference in thinking.  You're designing in
> terms of inputs and outputs, mock objects (and jMock in particular)
> aid designing in terms of state machines that communicate by message
> passing protocols.  In that model, a state transition is triggered by
> an event (incoming message) and causes some observable effects
> (outgoing messages).  The transition rule -- trigger, effects, new
> state  -- is a single "chunk" of behaviour.
>

I think you've hit the nail on the head here. I tend to think of
objects using design by contract so my tests follow a pattern of,
invoke a command and assert on queries. I'm rarely interested in
verifying the interactions between collaborators and when I need to I
find spies a better fit.

I'm not sure that the relationship between mock objects and design
style is well known. Would you say that using mock objects leads to
the design style you describe or that you favour that design style and
use mocks because they enable testing that design style?

Brian

Nat Pryce

unread,
Dec 31, 2010, 11:24:08 AM12/31/10
to growing-object-o...@googlegroups.com
On 31 December 2010 14:40, Brian Swan <bgs...@gmail.com> wrote:
> it's unfortunate that I had to use such a blunt instrument to
> get people to notice and to have this discussion.

You didn't. The discussion has been going on for over ten years. You
could have just joined in.

--Nat

--
http://www.natpryce.com

Brian Swan

unread,
Dec 31, 2010, 11:24:25 AM12/31/10
to Growing Object-Oriented Software


On Dec 31, 10:32 am, Uberto Barbini <ube...@ubiland.net> wrote:

>
> I need to see the code here to understand what you mean.
> Generally speaking I'm not very concerned by the "distinction between
> input and output", I'm interested in testing the behaviour as a whole.

Here is an example of a common use (abuse) of mocks from a Rails app
demonstrating what I see as the disadvantages of poor readability and
brittleness.

https://gist.github.com/761102

Perhaps instead of input and output I should have said setup and
asserts. So when I read a test I like to be able to clearly see what
is setup code and what are asserts and I find that spies make that
clearer. Here is another example that I think demonstrates that.

https://gist.github.com/761110

>
> Anyway my target is improving my team skills to go near the GOOS
> design style model.

Where I have seen problems is applying mocks without using the GOOS
design style.

Brian

Brian Swan

unread,
Dec 31, 2010, 11:32:09 AM12/31/10
to Growing Object-Oriented Software

On Dec 31, 2:11 pm, Uberto Barbini <ube...@ubiland.net> wrote:
>
> technically speaking yes, but given its name, all its method names I
> though it was included in the "mocks suck".

I actually used Mocktio in the presentation as an example of a test
spy framework as an alternative to mocking. The terminology of the
Mockito api is unfortunate.

>
> So at the end it's all about JMocks vs. Mockito? how boring...
>

Not at all, it's about the misuse of mocks and making people aware of
test spies as an alternative which may be more suitable depending on
your design style.

Brian

Steve Freeman

unread,
Dec 31, 2010, 12:03:31 PM12/31/10
to growing-object-o...@googlegroups.com
On 31 Dec 2010, at 17:32, Brian Swan wrote:
>> So at the end it's all about JMocks vs. Mockito? how boring...
>
> Not at all, it's about the misuse of mocks and making people aware of
> test spies as an alternative which may be more suitable depending on
> your design style.

I'm really struggling to see how a team that doesn't have the skills to write mocks properly will do better with spies. I can understand that they will get away with inadequate design for longer, but is that really a good thing?

S.

Uberto Barbini

unread,
Dec 31, 2010, 12:20:43 PM12/31/10
to growing-object-o...@googlegroups.com
On Fri, Dec 31, 2010 at 5:32 PM, Brian Swan <bgs...@gmail.com> wrote:
>
> On Dec 31, 2:11 pm, Uberto Barbini <ube...@ubiland.net> wrote:
>>
>> technically speaking yes, but given its name, all its method names I
>> though it was included in the "mocks suck".
>
> I actually used Mocktio in the presentation as an example of a test
> spy framework as an alternative to mocking. The terminology of the
> Mockito api is unfortunate.

Is it really? I'm pretty sure that 90% of programmers cannot tell the
difference.

I saw your examples, in the first example first case I think you're
also testing RoR, that could be a good thing or a bad thing depending
on the contest.
In the second example I really don't understand why the second case is
less brittle than the first one.

>
>>
>> So at the end it's all about JMocks vs. Mockito? how boring...
>>
>
> Not at all, it's about the misuse of mocks and making people aware of
> test spies as an alternative which may be more suitable depending on
> your design style.

More suitable => yes. More solid => no.
I can provide you with lots of tests using mockito in the wrong way,
but what's the point?


cheers

Uberto

Steve Freeman

unread,
Dec 31, 2010, 1:07:10 PM12/31/10
to growing-object-o...@googlegroups.com
On 31 Dec 2010, at 17:24, Brian Swan wrote:
> Here is an example of a common use (abuse) of mocks from a Rails app
> demonstrating what I see as the disadvantages of poor readability and
> brittleness.
>
> https://gist.github.com/761102

This one has a singleton and it's expecting on a query, rather than stubbing, which we don't recommend. Also, I might factor out the book params to show that a value is being passed through.

> Perhaps instead of input and output I should have said setup and
> asserts. So when I read a test I like to be able to clearly see what
> is setup code and what are asserts and I find that spies make that
> clearer. Here is another example that I think demonstrates that.
>
> https://gist.github.com/761110

Here the expectation is buried in the setup, it should come afterwards. One of the advantages of the jmock initialiser hack is that it makes the protocol stand out. And the problem with the spy version is I haven't made explicit that I'm ignoring other calls rather than I've forgotten about them.

S.

Steve Freeman

unread,
Dec 31, 2010, 1:15:00 PM12/31/10
to growing-object-o...@googlegroups.com
> I'm not sure that the relationship between mock objects and design
> style is well known.

That's because a bunch of well-known names mischaracterised the technique from the very beginning, and we were too busy to write it up properly (and too unimportant to get heard). Dave Astels was the only author who took the trouble to understand. The damage continues to propagate, especially in the .Net world. That's why I'm so tired of these arguments.

> Would you say that using mock objects leads to
> the design style you describe or that you favour that design style and
> use mocks because they enable testing that design style?

There's a postscript in the book, copied on the mockobject.com website, that describes how we got there. In the spirit of Nat's recent rant, mocks arise naturally from doing responsibility-driven OO. All these mock/not-mock arguments are completely missing the point. If you're not writing that kind of code, people, please don't give me a hard time.

S.

Brian Swan

unread,
Dec 31, 2010, 1:39:55 PM12/31/10
to Growing Object-Oriented Software

On Dec 31, 5:03 pm, Steve Freeman <st...@m3p.co.uk> wrote:
> I'm really struggling to see how a team that doesn't have the skills to write mocks properly will do better with spies.

My understanding is that using mocks "properly" is strongly related to
designing in the GOOS style. What I'm seeing is teams trying to use
mocks when the design style differs and having the problems I
presented. These teams are not necessarily aware of spies as an
alternative.

> I can understand that they will get away with inadequate design for longer, but is that really a good thing?

There are many ways to design a system, because a team is misusing
mocks does not mean that their design is inadequate. As Matt Wynne
posted earlier, he and I did an exercise based on this paper
http://blogs.agilefaqs.com/wp-content/uploads/2007/12/avatars-of-tdd.pdf
at ScotRUG to build the same app test first but using different
approaches. Matt used mocks, I didn't but I don't think my design was
"inadequate". The code is here for my solution https://github.com/bgswan/tddavatar2
and here for Matt's https://github.com/mattwynne/tddavatar/

What I think I'm getting at is that there is a relationship between
design style and use of mocks that most people aren't aware of. When
people try to apply mocks to a different type of design they have
problems. Basically what you and Nat have said earlier about using a
tool outside of it's sweet spot, just that people don't know what the
sweet spot is.

Brian

Brian Swan

unread,
Dec 31, 2010, 2:04:08 PM12/31/10
to Growing Object-Oriented Software

On Dec 31, 6:15 pm, Steve Freeman <st...@m3p.co.uk> wrote:
> The damage continues to propagate, especially in the .Net world. That's why I'm so tired of these arguments.

Also in the Ruby world

> If you're not writing that kind of code, people, please don't give me a hard time.

I think we're actually in total agreement here (except for me giving
you a hard time, sorry). I think it would be fair to say that my
presentation could be retitled 'Mocks suck, when used out of context',
without changing the content.

It seems the message of what the correct context is continues to be
missed.

Happy new year all.

Brian

Esko Luontola

unread,
Jan 1, 2011, 4:37:31 AM1/1/11
to Growing Object-Oriented Software
On Dec 31 2010, 4:11 pm, Uberto Barbini <ube...@ubiland.net> wrote:
> I don't see the point. Why using JMocks instead of mockito in that
> class would result in less brittle tests and better design?

Compare this spy version https://github.com/orfjackal/tdd-tetris-
tutorial/blob/beyond/src/test/java/tetris/RemovingFullRowsTest.java
with this mock version https://github.com/orfjackal/tdd-tetris-
tutorial/blob/beyond/src/test/java/tetris/easymock/
EasyMockExample_RemovingFullRowsTest.java

When using this style, setting the expectations before the action
confuses the tests - the expectations need to be called in the setup
instead of having a test for each feature. And in the last test there
is no easy way to say that the method was not called.

In addition to the style breakage, if one of those expectations fails,
then the tests for unrelated features break. This makes the tests
highly coupled because there is more than one reason why a test could
fail. As J.B. Rainsberger would say, they depend on more than one
piece of interesting behavior - they are integrated tests.

Steve Freeman

unread,
Jan 1, 2011, 5:12:39 AM1/1/11
to growing-object-o...@googlegroups.com
Agreed, in this format, there isn't a good place to put expectations. Using setup just hides the relationship.

In some frameworks, you can specify never(), which is the same as not specifying but makes the intention clear.

I think it's reasonable for a set of tests to break because an expectation fails, especially if you can have it pop up the debugger at the right time. Controlling for single failures at this granularity seems a bit too precise for me.

Again, what's missing from this is making explicit which unspecified calls can be ignored and which have just been forgotten about.

S.

Steve Freeman

unread,
Jan 1, 2011, 5:13:47 AM1/1/11