Mock Objects and Test Design/Clean Tests

[Hemal Varambhia]

Dear All,

Episode 23 on Mocking was interesting. My question is: aren't mock objects (spys especially) piercing the encapsulation barrier. Is it wise to use Mock Objects when the implementation of a method/class is volatile? I recall in the previous episode on Clean Tests saying that the principles of Object Oriented Design apply as much to tests as to production code. Aren't Spies violating the principle of encapsulation?

 

Happy New Year!

 

Yours sincerely,

Hemal

[Ron Jeffries]

Hemal,

Good question …

On Jan 1, 2014, at 8:48 AM, Hemal Varambhia <h.n.va...@gmail.com> wrote:

Episode 23 on Mocking was interesting. My question is: aren't mock objects (spys especially) piercing the encapsulation barrier. Is it wise to use Mock Objects when the implementation of a method/class is volatile? I recall in the previous episode on Clean Tests saying that the principles of Object Oriented Design apply as much to tests as to production code. Aren't Spies violating the principle of encapsulation?

I do not use mocks and spies much, and often think that the need for them is a sign that the code needs to be made more modular. And yes, they do violate encapsulation principles sometimes. Sometimes that’s the only way to get testing done, in which case, I’d hold my nose, get things working … and then see if I could figure out why the code needed to undergo exploratory surgery.

Ron Jeffries

www.XProgramming.com

There's no word for accountability in Finnish. 

Accountability is something that is left when responsibility has been subtracted. 

--Pasi Sahlberg

[Grzesiek Gałęzowski]

On Wednesday, January 1, 2014 2:48:58 PM UTC+1, Hemal Varambhia wrote:

My question is: aren't mock objects (spys especially) piercing the encapsulation barrier. Is it wise to use Mock Objects when the implementation of a method/class is volatile? I recall in the previous episode on Clean Tests saying that the principles of Object Oriented Design apply as much to tests as to production code. Aren't Spies violating the principle of encapsulation?

Hemal,

I think this topic will attract many different opinions and I just wanted to add that one of the "godfathers" of mocks' popularity, Nat Pryce, wrote on this topic some time ago. I understand that you're referring to episode 23 of Clean Code??? I don't know what are Uncle Bob's exact views on using mocks, but I understand them as a tool made with a specific design principles in mind, which Nat Pryce and Steve Freeman explain in their book on mocks in TDD.

In my opinion, mocks, when done right, enhance encapsulation. In my team, we converted code tested without mocks to mock-based and the code ended up less cluttered, more encapsulated and more "tell, don't ask''ey. But others may have different experiences.

Best regards,

grzesiek

[Curtis Cooley]

I completely agree. This topic has encouraged me to re-read Enodtesting with Mock Objects because I think we've misunderstood how to use them.

The current mocking frameworks, understanding and usage leads to encapsulation invading white box testing, but the claims in the paper are mocks lead to the opposite.

--
Curtis Cooley
curtis...@gmail.com
blog:http://ponderingobjectorienteddesign.blogspot.com
===============
Leadership is a potent combination of strategy and character. But if you must be without one, be without the strategy.
-- H. Norman Schwarzkopf

[Philip Schwarz]

Hello Hemal,

you wrote: " Aren't Spies violating the principle of encapsulation?"

I take it you are referring to the section starting @ 8':23'' of "Clean Code  →  Episode 23, Mocking Part 2" http://cleancoders.com/codecast/clean-code-episode-23-p2/show :

EINSTEIN: the Spy tests are coupled to the algorithm. So any change to the algorithm will break the spy tests. But on the other hand, the spot check tests, they depend only on the return values, and so a change to the algorithm will not break the spot check tests. And so the spot check tests are better.

FARMER: No Albert, you old fuzzhead, you have it backwards son, it is more important to know that the algorithm works: you can tolerate a little bit of coupling in return for that sure knowledge. 

ROBERT MARTIN: It's a dilemma isn't it? On the one hand, if you test the results of functions and ignore the implementation, then you can't know that the system will behave properly for every input. On the other hand, if you spy on the algorithm but ignore the output results then you can't refactor the algorithm.

EINSTEIN: We call this the uncertainty principle of TDD.

ROBERT MARTIN: And this leads us to the two different schools of thought of TDD: The Mockists and the Stateists.

SPOCK: The mockist school of TDD, sometimes known as the London School, puts an emphasis on the use of spies to verify the implementation of algorithms. They tolerate a higher coupling because of the increased assurance.

KIRK: The stateist school of TDD is sometimes also called the Chicago school, the Detroit school or the Cleveland school. It places a great emphasis on the values returned by functions and prefers to decouple the tests from the implementations of the algorithms they test.

ROBERT MARTIN: Mockism versus Stateism, Stateism versus Mockism: which is right? which is best?

SHERLOCK HOLMES: Neither and both, of course.

ROBERT MARTIN: Of course, it turns out that spying is very useful when you are testing things that cross the Dependency Inversion boundaries of the system. Remember our Login Architecture diagram? Mockist style tests are most useful when you are testing things that cross that red boundary and the reason for that is obvious: the stuff on the other side of the boundary is the stuff you want to mock out.  On the other hand, if you are not testing something that crosses a boundary then a stateist approach is probably better, because it drastically

reduces the coupling of the test to the implementation. And so of course it is a matter of balance and of knowing which tool to use for the appropriate situation.

FRANCISCAN MONK: Neither a Stateist nor a Mockist be.

Philip

[Philip Schwarz]

Grzesiek wrote: "Nat Pryce, wrote on this topic some time ago. "

Yes, State vs Interaction Based Testing is IMHO a must read for every reader of Martin Fowler's Mocks Aren't Stubs.

It doesn't look like Pryce has updated his post to reflect the following terminology changes Fowler made in his post in 2007:

Split the original distinction of state-based versus interaction-based testing into two: state versus behavior verification and classic versus mockist TDD...

Philip 

[Philip Schwarz]

In 2009 there was a Yahoo TDD Group "Classic/mockist" discussion (link to discussion is broken so see http://www.infoq.com/news/2009/01/classic-mockist-tdd for more details) in which Nat Pryce characterised the "mockist" vs. "stateist" distinction as pointless and nonsensical. Here are excerpts from two of his comments:

#1 I think the whole division of TDD practice into "mockist" vs "state based" is pointless, distracting and does a disservice to people trying to learn and use TDD.

Mock objects are just a tool. They are one of many tools that you need to use when doing TDD. Like any tool, they are designed to help solve a set of problems in a specific context. Outside that context they do not help and can be a hindrance. Hammering a screw into a wall will damage the screw-thread, but carpenters don't complain about "hammerist" carpentry!

So, instead of treating mock objects, or any other technique or tool, as an all-or-nothing "golden hammer", it's more helpful to understand where that tool fits into the broader picture of TDD practice.

#2 And that's my point about the nonsensical "mockist" vs. "stateist"

distinction. You can't use any tool exclusively. Each has a context

in which it is applicable and outside that context is no help or even

a hindrance.

I once helped a project who proudly showed me how much they were using

mock objects. They had mocked *everything*, including simple value

types like dates and currency amounts. It turned out they were hardly

testing anything at all and their tests were unreadable and

unmaintainable. They had learned that one either did "mockist" TDD or

"stateist" TDD, and so decided to do the "mockist" type. That's why

the whole meaningless distinction does people learning TDD such a big

disservice. Teach people to use their tools appropriately, not to be

mindless followers on one side or other of a non-existent schism.

Philip.  

On Thursday, 2 January 2014 07:36:05 UTC+1, Grzesiek Gałęzowski wrote:

[Philip Schwarz]

In Dec 2010 Brian Swan gave a presentation called "Mocks Suck" (http://www.testingtv.com/2010/12/20/mocks-suck-and-what-to-do-about-it/) and 

Claus Polanka started a GOOS Google group to discuss the presentation https://groups.google.com/forum/#!topic/growing-object-oriented-software/rwxCURI_3kM.

In the thread, Nat Pryce wrote again on the problems with the Mockist/Stateist distinction:

THERE IS NO SUCH THING AS MOCKIST TDD!

There are different ways of designing how a system is organised into

modules and the interfaces between those modules. In one design, you

might split the system into modules that communicate by leaving data

in shared data structures.  In another, you might split the system

into modules that communicate by synchronous message-passing.  In

another, modules might communicate by lazily calculated streams, or by

async messaging between concurrent actors, or by content-based pub/sub

events, or...

Depending on how you design inter-module interfaces, you'll need

different tools to unit-test modules (whether TDD or not).

Mock Objects are designed for test-driving code that is modularised

into objects that communicate by "tell, don't ask" style message

passing.  In that style, there is little visible state to assert about

because state is an implementation detail -- housekeeping used to

coordinate the message-passing protocols.

In other styles of code, procedural or functional, assertions on

queried state works fine.  (In asynchronous code, they do not because

of race conditions).

Even in a system that uses Mock Objects appropriately in its

unit-tests, a significant part is purely functional in my experience,

and so will not be tested with mock objects.

Talking about Mockist TDD vs Statist TDD leads people to think in

terms of adopting a "TDD style" and rigidly following a process,

rather than understanding how to select from a wide array of tools

those that are appropriate for the job at hand.

Rant over (this time!). 

 

Philip 

On Thursday, 2 January 2014 07:36:05 UTC+1, Grzesiek Gałęzowski wrote:

[Philip Schwarz]

In his latest book, Responsible Design for Android - Part 1: Dancing with the SDK, J.B. Rainsberger has the following very interesting section on the the nonsensical classicist/mockist divide:

I end up mixing the two predominant styles of design somewhat

haphazardly. In some places I strive to keep things separate, designing in what some people call

the “London school”, guided by the principles of Presenter-First Design that I first learned from

“Presenter First: Organizing Complex GUI Applications for Test-Driven Development” [1]. In other

places I throw everything into the same spot and use duplication and naming problems to guide my

refactoring, designing in what some people call the “Detroit school”, guided more directly by the

Four Elements of Simple Design [2]. We use the terms “London school” and “Detroit school” to recognise

the people from whom we’ve learned the underlying principles. The book Growing Object-Oriented

Software Guided by Tests, written by distinguished London programmers Steve Freeman and Nat

Pryce, illustrates the “London school” approach centered on strict application of the Dependency

Inversion and Interface Segregation Principles, which involves heavy and highly-effective use of

test doubles. Detroit-adjacent programmers Ron Jeffries and Chet Hendrickson (along with several

non-Detroit-adjacent programmers like Joshua Kerievsky, James Shore and Arlo Belshee) exemplify

the “Detroit school” approach which centers on a more intuitive and direct application of the Four

Elements of Simple Design, in which they avoid test doubles and use the resulting tension to guide

them towards designs that better reflect the Single Responsibility Principle. Many commentators

see this as schism, but I don’t. As you’ve seen in this book so far, I use and value both approaches,

although I don’t agree with some of my colleagues’ disdain for test doubles. I prefer not to make

this an issue of “to mock or not to mock”, since that takes focus away from where I think it belongs.

I distinguish the London and Detroit schools a different way: the Detroit school approach favors

those who like to build big things, then break them apart as they become unwieldy; but the London

school approach favors those who like to build little things, then put them together as they exhibit

properties of belonging together. While I see many authors² claiming that either approach objectively

outperforms the other, I can’t agree. As with most such debates, I find I have better results from

knowing and practising both approaches when they seem to fit the situation. Broadly speaking, I

use the London school approach when I feel confident in making one or two up-front high-level

design (“architecture”) decisions, such as “I will use MVC/MVP to design this part of the system” or

“I will model these interactions as publish/subscribe events instead of direct method invocations”;

I use the Detroit school approach when I don’t feel confident about making those decisions. The

London school approach feels more like methodical, safe, steady design, whereas the Detroit school

approach feels more exploratory or meandering (in the good sense, like wandering through a garden,

unworried about the path, trusting that I’ll get where I need to go). Some critics of the Detroit

school dislike the feeling of having to forget what they already know about design and re-discover

everything from first principles; some critics of the London school feel that test doubles lock their

design in and resist change. I see it another way: the Detroit school allows me to meander safely

when I have a general idea of the destination, but an unclear picture of the route; the London school

encourages me to build tiny things and put them together, and when I find that a piece needs to

change shape, I can throw that piece away and design a new one in its place without paying a

prohibitive cost for it. I think of the London school approach as a shortcut that I can take when I

think I recognise what I need to build and the Detroit school approach on which I can fall back when

I just don’t know what to do next. Thus, I benefit the most from practising both, feeling comfortable

with both, and even feeling comfortable refactoring from one style of design to the other. Please

don’t limit this question in your mind to “mocks suck” or “mocks rock”, no matter how pithy you

find those two slogans.

[1] http://atomicobject.com/files/PresenterFirstAgile2006.pdf

[2] http://www.jbrains.ca/permalink/the-four-elements-of-simple-design

Philip

On Thursday, 2 January 2014 07:36:05 UTC+1, Grzesiek Gałęzowski wrote:

[Philip Schwarz]

Although in his video Robert Martin says "Neither a Stateist nor a Mockist be.", when he compares the GOOS (http://www.growing-object-oriented-software.com/) outside in architectural approach with his inside out architectural approach (in this thread in the discussion forum for his videos) he says that "Inside out tests have a _lot_ fewer mocks.  Outside in tests are _dominated_ by mocks.":

The difference between my approach and the GOOS book is a common issue.  First, there _is_ a pretty big difference.  They start at the periphery of the system and work in.  I start in the center of the system and work out.  Oddly, that's about where the difference ends.  The "hexagonal architecture" that they promote is remarkably similar to the "Clean Architecture" that I promote.  

Why do these two techniques result in the same architecture?  Because our goals and emphases are the same, and we're both firing tracer bullets.  Both techniques are agile, both techniques recommend developing in tiny iterations, and both techniques push you end to end within the confines of an iteration.  Both wind up with acceptance tests running through the GUI at the end of the iteration.  And both want those acceptance tests written at the start of the iteration.  

Somehow people have gotten the idea that I don't think you should have tests at the GUI.  This is probably because I often stridently assert that you should not test business rules through the GUI, and people infer that to mean _no_ tests of the GUI.  OF COURSE you will test the GUI.  I just don't want you testing business rules through the GUI.  When write tests at the GUI, I want them to be GUI tests.  And yes, this means that the technique of writing BDD style cucumber tests through the GUI bothers me deeply.  I think the Rails community has gotten way off course with this approach.  

Whether you follow my approach, or the GOOS approach, you will still isolate the GUI and the Database from the business rules.  You will still create an architecture where the GUI and Database are plugins.  The only difference is whether, in the course of an iteration, you are moving from the outside in, or the inside out.  

While that difference in direction creates almost no difference in the structure of the application, it creates a _profound_ difference in the structure of the tests.  Inside out tests have a _lot_ fewer mocks.  Outside in tests are _dominated_ by mocks.  When you code from the inside out your tests depend on return values and state.  When you code from the outside in, your tests depend more on mocks that spy on the way your system work.  Inside out tests depend on results.  Outside in tests depend on mechanism and algorithm.  

Philip

[Philip Schwarz]

One of the Detroit-adjacent programmers mentioned by Rainsberger is James Shore.

Here is the section on mocks from Shore's book: The Art of Agile Development, published two years before GOOS:

Mock objects are a popular tool for isolating classes for unit testing. When using mock objects, your test substitutes its own object (the “mock object”) for an object that talks to the outside world. The mock object checks that it is called correctly and provides a pre-scripted response. In doing so, it avoids time-consuming communication to a database, network socket, or other outside entity.

Beware of mock objects. They add complexity and tie your test to the implementation of your code. When you’re tempted to use a mock object, ask yourself if there’s a way you could improve the design of your code so that a mock object isn’t necessary. Can you decouple your code from the external dependency more cleanly? Can you provide the data it needs—in the constructor, perhaps—rather than having it get the data itself?

Mock objects are a useful technique, and sometimes they’re the best way to test your code. Before you assume that a mock object is appropriate for your situation, however, take a second look at your design. You might have an opportunity for improvement.

And here @01':50''-2':50'' of episode 23 of his Let's Play TDD screencasts (http://www.jamesshore.com/Blog/Lets-Play/Episode-23.html) are Shore's thoughts on the mockist and classicist mindsets (as of September 2010):

"So I actually don't use mock object very often. I tend to avoid them. It is the classicist mindset, as opposed to the mockist mindset. some people like to go top-down using mocks to drive the implementation as they go. I don't claim to understand that approach very well, but it is very popular and I think the preeminent book describing that would be Steve Freeman and Nat Pryce's GOOS, I have heard many good things about that book though I haven't read it yet. So if you want to see more of the mockist approach, which is more of a top-down approach that stubs out each layer using mocks, check out his book. Steve Freeman and Nat Pryce were the ones of course who invented the mock object technique, so they really know their stuff. I have used mocks before, never really got it, I need to read their book to understand how they do it, but I prefer not to use mocks....I'm sure there will be an occasion when we actually do want to use them... "

I mentioned the above in https://groups.google.com/forum/#!topic/growing-object-oriented-software/GNS8bQ93yOo and Shore kindly intervened with his more recent thoughts on the classicist/mockist divide and on Rainsberger's piece:

 I don't think what I wrote six years ago in The Art of Agile Development is a good reflection of my opinion about test doubles today. I've learned a lot since then.

I thought that what Joe wrote about the "London school" and "Detroit school" was fairly insightful. It did a pretty good job of reflecting how I think about design: I'm focused on a few fundamental principles (simplicity, removing duplication, enhancing cohesion, and reducing coupling), and I'm constantly working to suppress my preconceptions about what the design should be, instead looking for opportunities to be surprised by new ideas. Joe calls this the "wandering through a garden" approach, and I think that's fair. I call it "reflective design" ( http://www.jamesshore.com/Blog/Reflective-Design.html ) and experiment with taking it as far as I can.

You can see me using this approach in my "Let's Play TDD" series. ( http://www.jamesshore.com/Blog/Lets-Play ). Some people find the first several episodes of that series frustrating because I'm making so many "obviously bad" design decisions. But in fact, I'm wandering through the garden. (And making some dumb mistakes. Live-coding steals about half my brain cells.)

I don't claim to understand the "London School" all that well. I originally ran across mock objects in 2000 via Steve et al's Endo-Testing paper ( stalatest.googlecode.com/svn/trunk/Literatur/mockobjects.pdf‎ ). I loved the idea and immediately applied it (by hand!) very heavily on a not-small Java XP project ( http://cf.agilealliance.org/articles/system/article/file/1028/file.pdf -- that paper doesn't specifically talk about mocks, though). It didn't work out well for us, though, and we eventually evolved to a better design that didn't use mocks.

That said, I don't think we were using mocks in the "London School" style. I have a lot of respect for Steve and Nat, and I believe that they have a style that does work well. One of these days I hope to sit down with one of them and have a face-to-face conversation where we dig into how our contexts are different--not so much about our design approaches, but about the forces that impact our design approaches. Perhaps they work on larger codebases than I do. Or smaller. Or more predictable (as Joe suggested). Or with more up-front design. Or less. Or with more diverse team members. Or more skilled. Or...

As for me, I'm happy to continue wandering through the garden. Every so often, I wonder up to test doubles. ( https://github.com/jamesshore/lets_play_tdd/blob/7285139f76fce50c65311bec6502ad5fa103a867/src/com/jamesshore/finances/ui/_ConfigurationPanelTest.java -- look at bottom three tests. ) And then I wander away again. ( https://github.com/jamesshore/lets_play_tdd/blob/9be53bad9d30d19592c33184dff1577b2027d3e0/src/com/jamesshore/finances/ui/_ConfigurationPanelTest.java -- look at the same three tests. ) Sometimes I do proper interaction testing. (https://github.com/jamesshore/lets_play_tdd/blob/9be53bad9d30d19592c33184dff1577b2027d3e0/src/com/jamesshore/finances/values/_ValidDollarsTest.java -- search for "RenderTargetStub" ). Sometimes I do something that seems totally gawdawful, except that it cuts the gordian knot of complexity elsewhere. ( https://github.com/jamesshore/lets_play_tdd/blob/9be53bad9d30d19592c33184dff1577b2027d3e0/src/com/jamesshore/finances/persistence/UserConfiguration.java -- look at the writeFile() method, or setObserver().) And sometimes I just throw up my hands and say, "I'm going to let this sit for a while until I come up with a better idea." ( https://github.com/jamesshore/lets_play_tdd/blob/9be53bad9d30d19592c33184dff1577b2027d3e0/src/com/jamesshore/finances/ui/_ApplicationFrameTest.java -- see the tearDown() method.) It all depends on the needs of the design in the moment.

Meanderingly yours,
Jim

PS: You can see a good example of me wandering through the garden starting with episode 188 of http://www.jamesshore.com/Blog/Lets-Play/ and continuing through episode 201. That's a case where I decided to move away from test doubles.

Philip

[Philip Schwarz]

One of the people who uses mocks sparingly is Roy Osherove, author of The Art of Unit Testing.

Here are his thoughts on the Classicist/Mockist divide from Refactoring and Design Skills for Test Driven Development SA2013.mp4

"Now there is this thing called mockist versus stateist. If you ever heard this discussion before, about people who use mock objects versus people who dislike mock objects: it is absolute rubbish..that thing doesn't really state what is the actual problem here. A lot of people who do state-based testing, where you test the state of your code,  don't want to check interactions at all of something,  they don't use interactions for designing, they would treat something as a black box. And people who use mock objects, apparently use mock objects for everything, just assert that, assert this. That is also not true. So whenever you hear this, yuo should understand that really, what we are talking about, is that mock objects are important, but not most of the time. Maybe 5% of your tests should have mocks, but nothing else. When we talk about a unit of work,  it could be one method, it could be  a class, it could be mulitple classes,  working together, but the end result of invoking  one public method somewhere should end up either with a return value, or a noticeable state change, in that the system's behavour is different, by invoking something at the same level, or calling a thrid party out of our control, or out of the unit's control. And people usually say that mockists  will actually call anything  a third party, so instead of calling a third party, you will call any party and you will prove that, and that is not true. So in that regard, mock objects are actually not a bad thing, but they have to be used in moderation."

At about 27 minutes in he talks about the book Growing Object-Oriented Software Guided by Tests, and briefly covers a few subject that developers using mock objects will find interesting. Here is an excerpt with a couple of his statements:

"So in a way, mock objects can be used to design communication protocols, interactions between objects, but this can also be very tricky, because if you use mock objects to define communications between objects that are really part of of the same coherent unit/context, that are just internally talking with each other, that are not  really doing anything specific 'outside' then you might actually create very brittle tests. … When I first read the book I had problems with that, because they used mocks to define communication protocols, I believed that the more you use mock objects, the more brittle your tests are….So I actually spoke with Steve Freeman (coauthor of the book), and we both actually agree that not everything should have a mock object. Not every communication should be actually checked, but only communications from outside the unit of work. 

…

Mock objects are important, but not most of the time. Maybe 5% of your tests should have mocks, but nothing else. …Mock objects are not a bad thing, but they have to be used in moderation."

Recently Osherove tweeted: 

what changed from 1st to second edition of AOUT? http://www.techdays.ru/videos/6832.html

Here are a couple of interesting excerpts from the presentation: 

"Today we are going to discuss everything that I got wrong in the first edition of my book the Art Of Unit Testing, which is quite a lot, so I actually went to goodreads.com and I marked my book as just 3 stars, cause it is not that good now. The second edition fixes a lot of the problems that I had in the first book. So this talk is going to be about everything that I learned in the more than 6 years since I started writing the first edition of the book.

I learned a lot of things. What does unit testing mean? We'll talk about the unit of work. We'll talk about...

...so one realisation that I have had is that usually you don't need mock objects in your tests, usually you try to have the first two types of tests:

return values, and system state change, and if you can get those two and not have mock objects at all in your tests, you are in a good position,

because mock objects by definition test internal interactions, and so by definition those tests are less maintainable, and they are harder to read, and 

they are longer, so maybe 5% of your tests should have mock objects in your tests, but not all of them, so if all your tests have mock objects

and you assert against things just because you can, you are doing too much, you should stop, please, for your own sake, because in five years you 

are just going to rewrite the system anyway...it is just going to cause a lot of pain."

...

[here he goes over the idea of the three types of unit test described in http://osherove.com/blog/2012/5/15/what-does-the-unit-in-unit-test-mean.html]

...

Because there are three types of unit tests, I updated the naming convention, because the convention I used was just the name of the method that's being

tested, but that is no longer true: it is the beginning, the entry point to the unit of work that is being tested. So any unit of work starts with a

public method somehow...so that is the beginning of the name of our test, and the middle part is the logical action or the input, or the actionOrInput

depending on the 3rd party, and the end is based on the type of end result...

1) UnitOfWork_Input_ExpectedOutput e.g. Addition_PositiveNumbers_ReturnSum

2) UnitOfWork_LogicalAction_ExpectedChangeInBehaviour e.g. Addition_WhenCalled_ResetsTheNextSum

3) UnitOfWork_ActionOrInput_ExpectedCallToThirdParty e.g. Addition_NegativeNumbers_CallsLogger

Philip

[Grzesiek Gałęzowski]

Philip,

Thank you for putting so many great references in one place!

I actually agree with what Nat wrote, less with what Roy Osherove did, although I would need to read the 2nd edition of Art of Unit Testing to get a feel of what he means by "unit of work". I read somewhere Steve Freeman making exceptions from mocking only for value objects (not mentioning strictly functional behaviors of y=f(x) type, where mocking just doesn't fit at all). I, for one, use mocks a lot, but at one time, I was deveolping an authorization tree where there were groups, devices and users inside and it had to answer questions like "is user X allowed to use device Z?" etc. I knew that there would be some node objects, some caches for quick node lookup depending on criteria etc., but I had not knowledge enough to design the interactions between those, so I started using non-mock approach ("if I put inside a group, add a user and add a device to the same group, when I ask whether the user is allowed, is should answer yes...") and it came out quite fine. Later, as the responsibilities between the components became more clear, I used more mocks on some parts of the tree ("if I create a tree and register with it three mocked custom caches, when I add an item to the tree, it should be registered with all the caches"), not to jump into combinatorial explosion, but both "styles" proved useful.

I remember from GOOS that Steve and Nat were also talking about combinatorial explosion. Allow me to cite the book: 

If we test at too large a grain, the combinatorial explosion of trying all

the possible paths through the code will bring development to a halt. Worse,

some of those paths, such as throwing obscure exceptions, will be impractical to

test from that level. On the other hand, if we test at too fine a grain—just at the

class level, for example—the testing will be easier but we’ll miss problems that

arise from objects not working together.

How much unit testing should we do, using mock objects to break external

dependencies, and how much integration testing? We don’t think there’s a single

answer to this question. It depends too much on the context of the team and its

environment.

So I think it's not wise to completely reject mocking from test maintainability point of view. BUT, what about the encapsulation? Maybe mocks are a fine way of reducing combinatorial explosion, but at the cost of coupling to internal implementation details of a class?

In my opinion, In places where we would use dependency injection anyway, this isn't true at all. Let's take the following test as an example (C#, NUnit, NSubstitute):

[Test] public void
ShouldOpenConnectionThenSendAndThenCloseItWhenAskedToSendPacket()
{
  //GIVEN
  var connection = Substitute.For<Connection>(); //Connection is an interface
  var sending = new Sending(connection);
  var packet = Any.Instance<Packet>();

  //WHEN
  sending.PerformFor(packet);

  //THEN
  Received.InOrder(() =>
  {
    connection.Open();
    connection.Send(packet);
    connection.Close();
  });
}

Is this test coupled to the private implementation of the Sending class? Of course not. Why is that so? Because the Sending class uses dependency injection, so it basically says it's up to its client to supply a fitting dependency. Meaning that the client of Sending class must be able to make a choice of a Connection interface implementation that will work with Sending class. And the client cannot make this choice without knowing both the interface AND the protocol (remember Liskov Substitution Principle!) of the class. The example I'm now going to give will be idiotic (forgive me), but I think it makes the point: what if someone opened the connection in Close() method and closed in Open() ? The system would not work.

We have more examples of this. E.g. when implementing a JEE servlet, we have to know what methods and in which order are called on the servlet by the JSP container. When implementing ASP.NET web page, we need to know the same and documentations for both of these frameworks tell us what methods are called and in what order. So this is not a "private implementation detail". Even if it's not part of the public API, it's still part of the public contract of classes that use our servlets or ASP.NET pages. Actually, even unit testing tools such as jUnit or NUnit tell us that "setup" methods are called before each test method and "teardown" methods are called after each test method. I didn't see anybody calling this "breaking encapsulation". So when we use dependency injection on a class, we actually make the way it uses its dependencies part of the public contract. As a prt of this public contract, the interactions should be tested for. 

Of course, there is also a grey area where dependency injection is only used to support injecting mocks, so this is more of a supporting rule than a main one.

grzesiek

[Philip Schwarz]

Another person identified by Rainsberger as exemplifying the “Detroit school” approach is Arlo Belshee.

About a year ago he wrote The No Mocks Book, to which Steve Freeman (coauthor of GOOS) replied with Some mocks, which includes the following:

One of the things we realised when writing Growing Object-Oriented Software, is that the arguments about mocks in principle are often meaningless. Some of us think about objects in terms of Alan Kay’s emphasis on message passing, others don’t. In my world, I’m interested in the protocols of how objects communicate, not what’s inside them, so testing based on interactions is a natural fit. If that’s not the kind of structure you’re working with right now, then testing with mocks is probably not the right technique.

This post was triggered by Arlo Belshee’s on The No Mocks Book. I think he has a really good point, buried in some weaker arguments (the developers I work with don’t use mocks just to minimise design churn, they’re as ruthless as anyone when it comes to fixing structure). His valuable point is that it’s a really good idea to try setting yourself style constraints to re-examine old habits, as in this object-oriented calisthenics exercise. As I once wrote, Test-Driven Development itself can have this property of forcing a developer to stop and think about what’s really needed rather than pushing ahead with an implementation.

Here are a couple of entries from the comments section of Some mocks:

Arlo Belshee
I agree that the optimal testing design will have a number if test doubles involved. It may even, depending on the code, have some full behavioral mocking. However, most usages of mocking that I see in typical code end up being duplications of the implementation expectations of the depended on code. Thus, a change in the dependency requires remembering to update all the tests for dependents.
This is time consuming at best, and error prone at worst (if I miss a dependency).
I also find that any general purpose tool, applied liberally, reduces thinking. This decreases thinking about alternate designs. I’ve had to learn dozens of design techniques in order to change from code easily tested with stubs to code easily tested with just inputs and outputs.
I really like the tell, don’t ask design style. I use it often. And when I use it, I test it with mocks. I just also use a bunch of other styles in other places. Sometimes I need to encapsulate activity and expose data (compiler passes, for example). Sometimes I need to encapsulate both data and communications, and expose behavior. Sometimes I just want to expose who is listening to what when, and don’t want to execute behavior or examine state.
Overall, I use test doubles now and then and full mocks rarely. They are critically useful in the right places. I just always make sure that I’m not using them where a better design would do.

 

steve.freeman @arlo Thanks for responding. I too have a line about going too far and then pulling back, “if you haven’t overdone it you don’t know where the boundaries are.”Yes, there’s a lot of dreadful use of mocks out there–and of procedures, data structures, objects, functions, variables, etc. So it’s important to distinguish between disagreement with a technique and an instance of its use (after all, the title is “No Mocks”).I’d really like to see you follow through with more refactoring of the example to see where it takes you. I understand your intent, but I wonder how it would pan out.

 

steve.freeman For me, the way of thinking. Done properly, I don’t believe that mocks are any more (or less) constraining than any other unit test approach–sometimes the right thing to do is to delete tests. I think I’d argue that mocks that got in the way are still a symptom of design problems and that just deleting them is not really addressing the issue.

 

Philip

On Thursday, 2 January 2014 07:36:05 UTC+1, Grzesiek Gałęzowski wrote:

[Philip Schwarz]

In Jan 2011 Jason Gorman (one of the organisers of the UK Software Craftmanship conference) blogged: Classic TDD or "London School"?. 

One question I get asked by teams adopting Test-driven Development is "should we use the 'Classic' or the 'London school' of TDD?"

Before I set out my answer, I should probably explain what I think is meant by "Classic school of TDD" and "London school of TDD".

If you've read books like Kent Beck's excellent Test-driven Development By Example, you will be familiar with the "Classic school". What distinguishes this school is the emphasis on a technique called triangulation.

...

The London school's definitive text is Growing Object Oriented Software Guided By Tests by Steve Freeman and Nat Pryce. 

...

Our goal here is reliable code and good internal design, which means that both schools of TDD are at times necessary - the Classic school when we're focused on algorithms and the London school when we're focused on interactions. Logic dictates that software of any appreciable complexity cannot be either all algorithms or all interactions (although, to be fair, I have actually seen applications like these - and wish to never see such horrors ever again).

This is why my TDD training courses have one day of "Classic TDD" and another day of "London school" TDD. It's not an either/or decision.

There is a GOOS thread discussing the post. Here are a couple of comments from Freeman:

Looks like it's time for Nat to repeat his rant :)

The two "schools" of TDD are really addressing different aspects of a codebase, neither is sufficient on its own.

...

Actually, there are some differences between us and Kent. From what I remember of implementation patterns, in his tradition, the emphasis is on the classes. Interfaces are secondary to make things a little more flexible (hence the 'I' prefix). In our world, the interface is what matters and classes are somewhere to put the implementation.

Like any technique, there is a huge "misunderstood" school. The "London" tag is an accident of history, but we did have a strong local group at the time that led to a number of innovations.

 

Philip

On Thursday, 2 January 2014 07:36:05 UTC+1, Grzesiek Gałęzowski wrote:

[Hemal Varambhia]

Dear Philip,
Thank you for a very comprehensive response. I think I am more statist than mockist (saves me having to think too much!) but as Uncle Bob said engineering is about making compromises. I am theoretical physicist by trade who constantly sought the right answer. Things are changing though!

Many thanks for your research,
Hemal

[Hemal Varambhia]

Dear Ron,

I didn't notice your reply! Terribly sorry, and thank you for your kind comment! :)

I don't use mocks or spies either for the same reason. I doggedly keep my tests decoupled from implementation details of the production code (it's all very black box). I do, however, use stubs when developing web service clients :P Reflecting on your post a bit made me realise just now that stubs do expose implementation details (the fact that we're even using their interface in developing the algorithm), however its nowhere near as invasive as mocks.

Yours sincerely,

Hemal

[Grzesiek Gałęzowski]

Hemal,

This may be true as long as you make all (or most) objects in your system manage their dependencies - then it's really a black box and interactions are implementation detail. As soon as you want a composable systems (where you can change the behavior of the system by plugging different objects or the same objects in a different manner without changing the consumers of those objects), interactions stop being implementation detail (because with dependency injection it's up to the client to supply an implementation of a dependency and this dependency must adhere to Liskov Substitution Principle) and start being requirements for objects to be plug-compatible with each other. To ensure this plug-compatibility, both interface and protocol that a class requires must be known publicly.

It's not mocks or stubs that expose those interactions. Dependency injection does. And if you want to have a composable system, dependency injection is a must. So these interactions become part of the public contract of a class. 

Of course, there is a possibility one will expose real implementation details *through* interactions, but one may do so as well through the public API. Thus, using mocks per se is no more invasive than another style - it just depends on what you design your system around.

Best regards,

grzesiek

[Hemal Varambhia]

Dear Grzesiek

In my line of work, in additional to developing RESTful web services, I also build software that talks to web services so in designing the clients for testability, I tend to build wrappers around the web service and pass the wrapper to the client I am developing through TDD as a stubbed wrapper. So it looks like I am using dependency injections but being, as you may have been able to tell very naive, I actually arrived at this design just by thinking of the simplest thing possible that could work and enable me to write tests.

Yours sincerely,
Hemal

[Hemal Varambhia]

Phillip,
If you do proceed with this book or pamphlet, it would be a great. I would be genuinely interested in obtaining a copy.

Hemal

[S. Trupp]

The question i have, if we as Stateists abandon encapsulation don’t use much inheritance and Polymorphism is not OO specific. 

What about Object Oriented Design? Are we Functional or Procedural Programmer and haven’t realised that? Or need OOP new definition today?