[GOOS] TDD: Why We Don't Party Like It's 1999

[philip schwarz]

http://www.infoq.com/presentations/tdd-variants

[philip schwarz]

Some excerpts:

"A lot of things happened since TDD was introduced, since the 1st edition of XP explained, and a lot of them were adaptations for environments that we were in, some of them were adaptations for people's different working styles, and all that lot of stuff, but we came up with this body of what we call the knowledge of TDD, or like TDD works, etc, but if you look at it closely, there are many opposites, there are many contradictions, like exactly opposite things" 

"Wet Tests, DRY Code" versus "Maintaining Your Test Rig"

"The 4 Rules" versus "Listening to the Tests" 

"Design by Wishful Thinking" vs "Make it Work, Make it Beautiful,Make it Fast" 

"Confidence vs Executable Documentation vs Design Feedback"

"Mock Only What You Own" versus "Mock Across Architecturally Significant Boundaries" 

"Tests Organised Into Suites Named After Classes" vs "Tests Organised by Feature Or Category" 

"1. I don't care about design feedback from _writing_ my tests!" 

"2. Design that can be changed is better than design that is good" 

"3. Listen to that voice that says, Should it really be this hard?" 

Philip

On Sunday, 28 September 2014 19:15:30 UTC+1, philip schwarz wrote:

http://www.infoq.com/presentations/tdd-variants

[philip schwarz]

A few excerpts about listening to the tests, design feedback, and GOOS:

"The 4 Rules versus Listening to the Tests. This is how s/w gets designed. How software in the TDD process gets designed. Listening to the tests is a different practice.

And listening to the tests is, when you are writing a test, if writing the test is hard, it is telling you something, it is telling you something about the class that you are testing. If it is hard to write that test, it could be either, it could be one of, the API of for your class is bad, or that you have another design problem in your system

that is inner related to that, it could be that the number of dependencies that you have in your class is too high, or they are too complicated, and it could be one more thing, [forgot what]"

"There are traditionally two schools, Detroit and London. Detroit came from Kent Beck and London came from GOOS and BDD and some other things. They, I don't want to say that 

they are coherent, I mean these contradictions, some of them are not, they are not all GOOS, they are not all London versus Detroit, so I didn't want to say, I didn't want to, it is an important distinction, but I didn't want to say that these are all one side versus the other."

"Confidence vs Executable Documentation vs Design Feedback. This set of conflicts, I have seen claims that each one of these is the most important thing about TDD. Kent Beck usually says, not confidence but he says that TDD is about removing the fear of changing your code...In GOOS they tell us that design feedback is the most important part of writing a unit test, and that is the design feedback I was talking about, listening to the tests."

[Nat Pryce]

I don't think we said anything was *the most important part* of writing any specific kind of test.  Perhaps the speaker  misunderstood what we were trying to get across in the internal vs external quality section.

--Nat

--

---
You received this message because you are subscribed to the Google Groups "Growing Object-Oriented Software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to growing-object-oriente...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
http://www.natpryce.com

[philip schwarz]

Excerpt on "Mock Only What You Own versus Mock Across Architecturally Significant Boundaries":

"This is probably the most direct contradiction in that what's recommended is exactly the opposite. Mock Only What You Own, this is the London school. Mock Across Architecturally Significant Boundaries, which London school would say: don't mock those, we don't want that." 

[Steve Freeeman]

I think, perhaps, he should have talked to us before explaining what we say...

S

[Nat Pryce]

Not sure those are necessarily at odds.  In the ports-and-adaptors architecture, I'm happy to mock port interfaces.  And those are implemented by adaptors, which connect application domain(s) to technical domain(s).  To me it sounds as if mocking adaptors *is* mocking across architecturally significant boundaries.  But, it's also mocking types that I own, so also I reduce my risk of unexpected change being undetected by my tests.

--

---
You received this message because you are subscribed to the Google Groups "Growing Object-Oriented Software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to growing-object-oriente...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
http://www.natpryce.com

[philip schwarz]

An excerpt on Jason trying the GOOS approach:

"So, GOOS, our team was committed to using this as the method of TDD, and it had ideas in it that were new to me. Not exactly new to me, they were starting to get really popular in the circles that I was in, that's part of the reason that I went to this job, we learned about, and we did this. The problem was we had trouble. We never achieved the same kind of flow. We had trouble changing our software. We had trouble figuring out what to do with our tests when we would refactor & it just didn't flow the same way"

[philip schwarz]

Excerpts on the GOOS graph for the degree of feedback provided by unit, integration, and end-to-end tests on the internal/external quality of our code (I found a copy of the graph here http://meekrosoft.files.wordpress.com/2010/10/gooagbt-chart.png) :  

"So I want to show you a graph. This is from GOOS. It is really interesting, this tells us that...and what I think about this, it is really interesting, this is definitely not how I am used to thinking, this is definitely different, and what is interesting about it is that the claim is that the claim is that unit tests don't tell us how well our system works, it tells us more about how well our code is designed, so one of the interesting points that I realise I have glossed over is that the London school is often called mockists, whereas the Detroit school is often called classicists or state-ists sometimes, and that is about how we assert things. In the London school, we make assertions about how objects communicate with each other, and not that they have necessarily the correct result, that they have the correct communication patterns that we can change, that we can plug things into, and so, since we use mocks, in the London school, what happens is that each test that we write encodes how in it, some way, how the collaborators work, or how they must work or respond, when they are asked things."

"What I have discovered is that when there is a lot of this there are a number of refactors that I just cannot make. I can't inline a class. If I have 3 classes, and I decide that they need to be broken up into 3 different classes, because the separation of responsibilities wasn't the way that things change (in the real world?), I can inline one class, and then inline another class, and then extract a class, and then extract another class, if I have tests on each class, all my tests are broken. Some people say OK, cool, you can write new tests to drive those out and if you find that is valuable then do that, but my caveat is I would like you to think whether you want to, whether you really want to or whether sticking to a kind of form for a, remember when I was talking about, we like to have things named Foo and FooTest, right? is that our overly sensitive geek pattern matching going on right there, we need to do that, right? I would like to propose a different graph here..."

[Matteo Vaccari]

It seems to me that he's missing the idea of using mocks to design protocols/interfaces. Expecially reused protocols and interfaces.
--
Mandata dal telefonino.

--

[Uberto Barbini]

uh? I also think that TDD is mostly a design technique and feedback on
design is the main advantage in writing unit test.
I thought GOOS was telling the same. :)


Uberto

[Grzegorz Gałęzowski]

Hi, Philip

Personally, I think there's not so much difference and condtradictions.

"Wet Tests, DRY Code" versus "Maintaining Your Test Rig" - as I see it, GOOS advices maintaining test rig at least on acceptance level.

"The 4 Rules" versus "Listening to the Tests" - I think that listening to tests enforces the 2nd rule which is "Expresses every idea that we need to express". This is also understood as "maximizing cohesion", which is exactly what happens when listening to tests. e.g. if the tests starts becoming too large, then it seems that a class covers too many ideas implicitly and it means we need to explicitly express them.

"Design by Wishful Thinking" vs "Make it Work, Make it Beautiful,Make it Fast" - when reading GOOS worked example, I had the feeling that with the first one or two acceptance tests that were used to drive the walking skeleton, Steve and Nat were doing exactly that - "make it work" first. Only after some kind of structure emerged, did they move to refactoring and introduced mock-based unit tests. Also, I was thinking about the "design by wishful thinking" and came to a conclusion that by defining an end to end test first, the thinking is not so wishful - we already know when the input arrives and where the output should come out. Thus, when we start test driving collaborators, we already know where we are heading.

"Confidence vs Executable Documentation vs Design Feedback" - I checked recently and GOOS mentions all three. Confidence is mentioned in chapter 1 at least few times. Only that each level of testing gives us a different kind of confidence. Furthermore, mock-based unit tests give us the kind of confidence we won't get from something else, i.e. that an object will behave in a certain way when composed with different collaborators, as long as these collaborators follow the protocol the object expects.

"Mock Only What You Own" versus "Mock Across Architecturally Significant Boundaries" - as Nat said, I don't think these are contradictory. Even if we take the whole system and mock only ports (i.e. we test a path through whole "inner hexagon" in a test) it still does not contradict GOOS in my mind - this is just another level of testing. As GOOS says: 

"There’s a balance between exhaustively testing execution paths and testing integration. If we test at too large a grain, the combinatorial explosion of trying all the possible paths through the code will bring development to a halt. Worse, some of those paths, such as throwing obscure exceptions, will be impractical to test from that level. On the other hand, if we test at too fine a grain—just at the class level, for example—the testing will be easier but we’ll miss problems that arise from objects not working together."

"Tests Organised Into Suites Named After Classes" vs "Tests Organised by Feature Or Category" - again, as I see it, in GOOS, acceptance tests are organized by feature or category.

Not sure if I got everything right. Anyway, that's how I feel about it.

grzesiek

[philip schwarz]

Yes.

See below for a few more pointers to Jason's thoughts on 'listening to the tests', TDD and GOOS.

On April 23rd, 2014, Jason tweeted:

@eraserhd Here @dhh talks about TDD failing us.  He references TDD's "original" goals and intent - but names GOOS's! http://david.heinemeierhansson.com/2014/tdd-is-dead-long-live-testing.html 

@eraserhd An excellent example of why I'm talking about this at self.conference.  We can't actually remember what TDD was. // @dhh

Here is what (I believe) he is referring to in DHH's post:

"The current fanatical TDD experience leads to a primary focus on the unit tests, because those are the tests capable of driving the code design (the original justification for test-first)."

So it looks like Jason thinks that "a primary focus on the unit tests, because those are the tests capable of driving the code design" is GOOS's goal and intent for TDD.

On May 14th Jason tweeted: 

@eraserhd Thinking about writing a blog post about mocks and how the _benefits they claim_ are simply a restatement of the problems they bring.

On May 15th, when Gary Bernhardt wrote his great post on test isolation:

@garybernhardt Blogged: "Test Isolation Is About Avoiding Mocks" https://www.destroyallsoftware.com/blog/2014/test-isolation-is-about-avoiding-mocks 

@garybernhardt That post is the first time that I've ever managed to directly explain the design pressure of isolation in detail. It only took seven years.

Jason tweeted as follows:

@eraserhd There's at least two kinds of TDD.  If you don't like TDD because it makes your code hard to change, *please* try the other kind.

@eraserhd Curious about this idea that mocking, when done well, does not make refactoring more difficult. Can't imagine how. // @garybernhardt

@eraserhd .@garybernhardt I've encountered codebases with mocks-not deeply nested- where I couldn't refactor safely. This is why I ask.

@eraserhd .@garybernhardt Both safety and ease troubles, but I definitely see that the safety frustrations come equally from my own expectations.

@eraserhd .@garybernhardt I mean, I *expect* safety from units, which isn't fair if it's not intended.

Unfortunately Gary's replies can no longer be found on twitter:

@garybernhardt May 16 Twitter dropped about thirty of my replies on the floor overnight.

Philip

To unsubscribe from this group and stop receiving emails from it, send an email to growing-object-oriented-software+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
http://www.natpryce.com

[philip schwarz]

Thanks for that Grzegorz.

About Jason's claimed conflict between "The 4 Rules" and "Listening to the Tests"...Corey Haines has written a small book on the 4 rules (free sample available at Leanpub) and he has no problem listening to the tests while applying the rules:

"Over the years, I’ve come to see the 4 rules of simple design as the most useful concrete, coding-time principles to keep in mind."

"Automated unit test suites can have a tendency towards fragility, breaking for reasons not related to what the test is testing. This can be a source of pain when maintaining or making changes to a system. Some people have even gone to the extreme of moving away from unit- or micro-tests and only writing full-stack integration tests. Of course, this is the wrong reaction. Instead, we should investigate the source of the fragility and react with changes to our design."

"When we talk about how tests can influence design, a fundamental guiding principle is “difficult to test means a design issue.” That is, if you find it hard to write a test for something, change your design. I consider this a natural step in the evolution towards an effective test-driven development style. As one gets more experience and begins the shift from test-first to test-driven development, the reaction to pain changes:

• Hard to test -> change tests

This is a form of using tests as verification.

• Hard to test -> change design

This is a form of using tests as design pressure."

Philip

[Steve Freeeman]

Having read this post, it looks to me like he still hasn't got it. He's stubbing value objects (or at least I hope they're value objects), which makes the tests brittle and opaque. He's also trying to unit test in the presence of ActiveRecord, which is often entertaining.

Increasingly, I find myself thinking we should use our TDD skills as a competitive advantage, and give up trying to fix the world.

S

[philip schwarz]

I sent Corey Haines a tweet about Jason's view, and sent Jason one about Corey's book: 

@coreyhaines FYI as author of https://leanpub.com/4rulesofsimpledesign - in http://www.infoq.com/presentations/tdd-variants @eraserhd talks of 4 Rules versus Listening to the Tests 

@eraserhd https://leanpub.com/4rulesofsimpledesign doesn't speak of a conflict between 'The 4 Rules' and 'Listening to the Tests' https://leanpub.com/4rulesofsimpledesign

Here was the ensuing exchange:

@coreyhaines @philip_schwarz @eraserhd I'll check it out, although I don't see why there is a versus with those. Hmm…

@angelaharms @coreyhaines iirc, he was saying he doesn't need painful tests to guide architecture @philip_schwarz @eraserhd

‏@coreyhaines @angelaharms still doesn't feel like a versus to me. More like an also. @philip_schwarz @eraserhd

@jbrains @coreyhaines @angelaharms @eraserhd @philip_schwarz Same for me. We don’t claim that one ought to need tests to drive good design.

‏@eraserhd @jbrains @coreyhaines @angelaharms @philip_schwarz Not arguing against listening to tests, but against having tests = design is good

@jbrains @eraserhd @angelaharms @coreyhaines @philip_schwarz Jesus. Who’s claiming that craziness?!

@jbrains @eraserhd @angelaharms @coreyhaines @philip_schwarz I do find that when I make code more testable, it tends to be cheaper to change.

@philip_schwarz @jbrains @eraserhd @angelaharms @coreyhaines "the qualities that make an object easy to test also make our code responsive to change" -#GOOS

@philip_schwarz 1/ @eraserhd (@jbrains @coreyhaines @angelaharms) but in the following you are saying (right?) that listening to the tests is not a TDD

2/ design activity: "The 4 Rules versus Listening to the Tests. This is how s/w gets designed. How software in the TDD process gets 

3/ designed. Listening to the tests is a different practice...I don't care about design feedback from writing my tests. If I see it, like 

4/ if I notice that something is bad or broken when I am writing my tests, cool, I know that, I am not going to ignore it, but I don't

5/ want to write tests so that I figure that out, and that's because, bad code, I know what it looks like, right? If something is too

6/ complicated, it is pretty obvious, and if I can make it better I want to do that." 

P.S. Here is a more extended quote from (GOOS's 'Listening to the Tests') than the one I tweeted above:

"We’ve found that the qualities that make an object easy to test also make our code responsive to change.
The trick is to let our tests drive our design (that’s why it’s called test-driven development). TDD is about testing code, verifying its externally visible qualities such as functionality and performance. TDD is also about feedback on the code’s internal qualities: the coupling and cohesion of its classes, dependencies that are explicit or hidden, and effective information hiding—the qualities that keep the code maintainable.
With practice, we’ve become more sensitive to the rough edges in our tests, so we can use them for rapid feedback about the design. Now when we find a feature that’s difficult to test, we don’t just ask ourselves how to test it, but also why is it difficult to test."

Philip

[philip schwarz]

As an aside on the four rules, when Jason explained the rules, he forgot about DRY:

 "I can only remember three of the rules at any one time...the 3rd one is the one that I am going to forget this time, oh, yes, 'no duplication', the 3rd one is 'no duplication', in the logic of the code"

I wonder if that was a rhetorical device to convey a message similar to Corey Haines' book, i.e. "this rule isn’t about code duplication; it is about knowledge duplication", or similar to Sandi Metz's message in 'All the Little Things' http://www.confreaks.com/videos/3358-railsconf-all-the-little-things: "duplication is far cheaper than the wrong abstraction...the first rule we teach novices is DRY...and I am not saying it is a bad rule...we teach them that rule...because they can't understand anything else, ...[when you have enough experience] you can tolerate a little duplication and wait on a better abstraction...prefer duplication over the wrong abstraction "

Philip

On Thursday, 2 October 2014 19:28:31 UTC+1, Grzegorz Gałęzowski wrote:

[philip schwarz]

I thought that Jason's answer that he is not arguing against listening to the tests seems to contradict his point that "I don't care about design feedback from writing my tests", so I dug up an old GOOS post on listening to the tests and asked him directly:

@philip_schwarz @eraserhd is listening to the tests http://www.mockobjects.com/2007/03/synaesthesia-listening-to-test-smells.html  part of the design process of TDD? /cc @coreyhaines @jbrains @angelaharms

If I get an answer, I'll let you know.

Philip

[philip schwarz]

Jason speaks of mockists versus classicists (stateists)...this is going to bore a few people but just for the benefit of anyone new to the group, I asked Jason the following (on twitter):

When you say "the London school is often called mockists, whereas the Detroit school is often called classicists, or state-ists sometimes...what I have discovered is that when there is a lot of this [using mocks a la London school] there are a number of refactors that I just cannot make" are you saying there are two contrasting approaches? 

What do you make of the following?: 

"I benefit the most from practising both, feeling comfortable with both, and even feeling comfortable refactoring from one style of design to the other" @jbrains - https://groups.google.com/d/msg/software_craftsmanship/04LHBxg9hv8/FP_OpLf36GAJ

"Neither a Stateist nor a Mockist be" - @unclebobmartin https://groups.google.com/d/msg/software_craftsmanship/04LHBxg9hv8/TpAIKfS3oBQJ 

"I think the whole division of TDD practice into 'mockist' vs 'state based' is pointless, distracting and does a disservice to people trying to learn and use TDD" - @natpryce - https://groups.google.com/d/msg/software_craftsmanship/04LHBxg9hv8/l_9hHLm1boAJ

"Talking about Mockist TDD vs Statist TDD leads people to think in terms of adopting a 'TDD style' and rigidly following a process, rather than understanding how to select from a wide array of tools those that are appropriate for the job at hand." - @natpryce - https://groups.google.com/d/msg/software_craftsmanship/04LHBxg9hv8/l_9hHLm1boAJ

If I get an answer, I'll let you know.

Philip

[Mike Stockdale]

"Isolated testing has an easily identified villain: the deeply-nested mock object."  Starts with a bogus strawman - nobody who uses mocks advocates deeply nested mocks.

--
Cheers,
Mike Stockdale

fitSharp
Syterra Software Inc.

[J. B. Rainsberger]

On Sun, Oct 5, 2014 at 2:49 PM, Mike Stockdale <jedi...@syterra.com> wrote:

 

"Isolated testing has an easily identified villain: the deeply-nested mock object."  Starts with a bogus strawman - nobody who uses mocks advocates deeply nested mocks.

On the contrary, nested mocks => indirection without abstraction. I guess we need to make this clearer.

-- 

J. B. (Joe) Rainsberger :: http://www.agiletutor.com :: http://www.jbrains.ca ::
http://blog.thecodewhisperer.com
Free Your Mind to Do Great Work :: http://www.freeyourmind-dogreatwork.com

[Seb Rose]

 

 

 

On Sun, 5 Oct 2014, at 08:19 PM, J. B. Rainsberger wrote:

On Sun, Oct 5, 2014 at 2:49 PM, Mike Stockdale <jedi...@syterra.com> wrote:

 

"Isolated testing has an easily identified villain: the deeply-nested mock object."  Starts with a bogus strawman - nobody who uses mocks advocates deeply nested mocks.

 

On the contrary, nested mocks => indirection without abstraction. I guess we need to make this clearer.

 

"Stop it!"

 

-- 

J. B. (Joe) Rainsberger :: http://www.agiletutor.com :: http://www.jbrains.ca ::

http://blog.thecodewhisperer.com

Free Your Mind to Do Great Work :: http://www.freeyourmind-dogreatwork.com

--

 

---

You received this message because you are subscribed to the Google Groups "Growing Object-Oriented Software" group.

To unsubscribe from this group and stop receiving emails from it, send an email to growing-object-oriente...@googlegroups.com.

[philip schwarz]

>Isolated testing has an easily identified villain: the deeply-nested mock object."  Starts with a bogus strawman

for those who have not yet read the post, it is not Bernhardt that commits the straw man fallacy. In the first part of the "Is TDD Dead?" Google Hangout, Kent Beck said the following:

 

"I just don't go very far down the mock path. I look at code where you have mock returning mocks returning mock and my experience is if I have, if I use TDD I can refactor stuff, and then I heard these stories, people say "well I use TDD and now I can't refactor anything" and like I could not understand that, and then I started looking at their tests and well, if you have mocks returning mocks returning mocks, your test is completely coupled to the implementation, not the interface, but the exact implementation of some object, you know, three streets away, of course you can't change anything without breaking the test. So that for me is too high a price to pay.That is not a trade off I am willing to make just to get piecemeal development"

As Bernhardt explains at the end of his post, he doubts that Beck committed the fallacy, but he says that many others have committed it:

"This post was triggered by Kent's comment about triply-nested mocks. I doubt that he intended to claim that mocking three levels deep is inherent to, or even common in, isolated testing. However, many others have proposed exactly that straw man argument. That argument misrepresents isolated testing to discredit it; it presents deep mocks, which are to be avoided in isolated testing, as being fundamental to it; it's fallacious. It's at the root of the claim that mocking inherently makes tests fragile and refactoring difficult. That's very true of deep mocks, but not very true of mock-based isolation done well, and certainly isn't true of isolation done without mocks."

 

Philip

[philip schwarz]

>we should use our TDD skills as a competitive advantage, and give up trying to fix the world. 

Nice. Reminds me of the following:

THE 7 HABITS OF HIGHLY EFFECTIVE PEOPLE 

HABIT 5: SEEK FIRST TO UNDERSTAND, THEN TO BE UNDERSTOOD

Philip

[philip schwarz]

>There are traditionally two schools, Detroit and London

The following tweet from a few months before Jason gave his talk throws more light on the divides he speaks of:

28 Feb 2014 @eraserhd .@angelaharms @jbrains I see emergent design in Detroit coming from refactoring. In GOOS, from isolation tests being hard.

Philip 

[Steve Freeeman]

This whole discussion is fundamentally broken. Pointless strawman divisions.

There is more than one tool in the box.

S

[Malte Finsterwalder]

On 7 October 2014 09:53, Steve Freeeman <st...@m3p.co.uk> wrote:

This whole discussion is fundamentally broken. Pointless strawman divisions.

There is more than one tool in the box.

+1 !!!

 

We should rather discuss what the tradeoffs of the different styles are and when they best apply. That would yield real benefit and help people to recognize and appreciate both styles.

And yes, there is also personal preference involved. There are several situations that I can equally test with both styles where there is no clear advantage for either style.

Greetings,

   Malte

[philip schwarz]

Steve,

I forgot to ask you at the time: when you said "Having read this post, it looks to me like he still hasn't got it", what did you mean? 

At first I thought that maybe as you read "Test Isolation Is About Avoiding Mocks" (https://www.destroyallsoftware.com/blog/2014/test-isolation-is-about-avoiding-mocks) you temporarily thought that it was written by Jason, (the author of "Why We Don't Party Like It's 1999", the talk being discussed in this thread) rather than by Gary Bernhardt, and so in saying "it looks to me like he still hasn't got it" you were you were reinforcing your previous statement aimed at Jason: "I think, perhaps, he should have talked to us before explaining what we say."

But then I thought that very unlikely, and so I thought by "still hasn't got it" you meant that it wasn't the first time you witnessed Gary Bernhardt stubbing value objects and/or unit testing in the presence of ActiveRecord?

What did you mean?

Thanks,

Philip

On Saturday, 4 October 2014 22:17:58 UTC+1, Steve Freeman wrote: