TDD, UTDD, ATDD, AD&D redux...

[Andrew C. Greenberg]

This post is something of a follow-up to https://groups.google.com/forum/#!topic/clean-code-discussion/to5UHcVEQgo (A Friendly Amendment), in which I proposed an amendment to the UB Cannon:

Over the years I have come to describe Test Driven Development in terms of three simple rules. They are:

1. You are not allowed to write any production code unless it is to make a failing unit test pass.
2. You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.
3. You are not allowed to write any more production code than is sufficient to pass the one failing unit test.

(emphasis supplied). This was reiterated in UB’s recent video, where he characterizes these rules as an essential “discipline” of software development.

My proposal was to replace the word “unit” with “automated”, deferring to Beck’s famous tome:

For the vast majority of us … following these two simple rules can lead us to work to our potential:

1. Write a failing automated test before you write any code
2. Remove duplication

(emphasis supplied). And it is significant to note that KB’s definition of a unit test is fairly limited, essentially to tests that can be run independently from one another.

The motivation for my amendment was that several “whole application” development strategies use — indeed by design — automated integration and acceptance tests to begin building a testable “walking skeleton” of the application, dropping down to unit tests as specific functionality is identified. I used a number of examples in support of this practice, pointing to the Corey Haines videos and other “whole application” development screencasts. Response was mixed, but largely along the lines of:

1. This is not TDD, it might be BDD or ATDD or something else, but it ain’t the real deal.
2. This ignores the insights of the clean architecture (DDD-style development), which “always” starts with unit tests.

and the like. This is not a fair summary, but I have to say that after watching UB’s recent video on “Advanced TDD” (where he restated the cannon rules as an essential discipline), and the current clean coders “Java Case Study,” (using Fitnesse acceptance tests to develop outline production code), I have been suffering some cognitive dissonance.

I would be curious if people might reconsider the suggestion that using acceptance/integration testing to drive out initial program structure is inconsistent (by amending “unit” test to read “automated” test in the cannon) with the principles of DDD and TDD? Forgive me if I seem to be beating a dead horse, but isn’t the most recent case study exactly what I proposed? Can we agree that doing so (to build a walking skeleton for example) is NOT a violation of fundamental TDD discipline?

​

[Frederik Krautwald]

I could go with automated if tests were changed to be called checks. Testing is a completely other ballgame than what we as developers (oops... software craftsmen) do. Testing cannot be automated. Testing is about doing things to the test object or system that hasn’t been accounted for; doing the unexpected.

[Dave Schinkel]

"I have to say that after watching UB’s recent video on “Advanced TDD” (where he restated the cannon rules as an essential discipline), and the current clean coders “Java Case Study,” (using Fitnesse acceptance tests to develop outline production code), I have been suffering some cognitive dissonance."

totally, I see so many twists to how people are doing BDD and when they drop down and how to the lower-level tests FROM the highter level BDD.  It's still grey to me how you can have test suites around different Units when your BDD is so general.  Seems like you'd be doing a lot more SUT code before you get those BDD tests to pass, so less lower level unit tests I infer if you have very general BDD use case tests.  It's still very grey to me how granular you get FROM the BDD tests and the dance in how you get to those lower level tests and how you segregate or if you aren not to segregate them out to cover different layers of your sub app code.

[Frederik Krautwald]

The problem comes down to when acceptance tests are written by programmers, and not "the business", then they aren’t acceptance tests, but, in fact, unit tests, and should be written in code. Acceptance tests are tests written by "the business" for "the business". The programmers, for that matter, could, and, possibly, should be totally oblivious to the acceptance tests, as they are, in essence, none of their concern. Use cases, however, are a concern of the programmer, because use cases dictate what the program should do. Programmers can and should write unit tests to ensure that their code does what it is supposed to do, i.e., unit tests are tests written by programmers for the programmers. A unit is whatever the programmer decides it to be. This means that a unit test is there to satisfy programmers’ confidence in the code they write to fulfill use cases. The only reason to write tests in a BDD language is if "the business" is somehow involved.

So, of course, it becomes unclear when acceptance tests, written in a BDD language, try to substitute use cases in an inadequate manner and leaving it for the programmers to interpret the meaning of those tests, when they aren’t even written for the programmers.

[Dave Schinkel]

Let me be clear. When I say "acceptance tests" I am not referring to "Acceptence criteria". The business should write the acceptance criteria not the tests. The tests are what devastating start with doind BDD and debs should be writing the actual code that implements the gherkin.

[Frederik Krautwald]

Acceptance test were proposed by Kent Beck in a poll on the XP Mailing List, and was accepted as a standard term in XP some 15 years ago. Originally, they were called functional tests because they try to test the functionality of a user story. "The business" specifies the scenarios to test when a user story has been correctly implemented. It is the business’ responsibility to verify the correctness of the acceptance tests. Acceptance tests should be automated, but this is not a requirement.

XP also came up with Developer Bill of Rights which declares a right to detailed requirements and specifications. It is that description that is the acceptance test. It is, however, not the business’ responsibility to translate the tests into convenient code. This can be done by the programmers, a QA department, an automated tool, or whoever wants to do it. In fact, acceptance testing can be the business sitting in front of the computer and testing the software functionality, providing thumbs up / thumbs down.

Unit tests and acceptance tests have only one thing in common and that is the word test. Acceptance tests doesn’t "drop down" to unit tests, as you implied above. As I already mentioned, unit tests are for the programmers to have confidence in the code they write and is quite, if not completely, unrelated to whatever acceptance tests specified by the business (product owner). It is, therefore, no wonder why this "grey area" emerges for programmers, like yourself, who try to let their unit tests dictate by attempting to trickle down from a BDD language such as Gherkin.

If this were, in fact, possible to have unit tests expanding from a few Gherkin sentences, without adding more information, it would require that the Gherkin sentences somehow had information hidden within that could be expanded by some means. I am pretty sure this violates a couple of laws and that information theorists would be very excited to know about this "magic". For sake of simplicity, The letter A cannot expand into the text A Grand Scheme System without some other rule set being applied by some mechanism.

[Dave Schinkel]

yea totally not on the same page.

" unit tests are for the programmers to have confidence in the code they write and is quite, if not completely, unrelated to whatever acceptance tests specified by the business (product owner)"

We write BDD tests against the Gherkin specified in stories.  I literally take the Gherkin and throw it into my Mocha Describes().  How is this not unit tests driven by requirements.  BDD tests are TDD unit tests.

[Sebastian Gozin]

BDD tests are not TDD unit tests.
Implementing BDD tests in Mocha doesn't change this. Though it makes it harder for the people writing the requirements to read them and thus remember what the system does.

When a developer makes changes to a system he will also change the tests because he believes he needs to.
He can thus inadvertently change behavior the business does not wish to see changed.

BDD tests prevent this because the developer knows these are tests for which he does not have the authority to change them.

[Frederik Krautwald]

Fist, let’s clear up some definitions to avoid further misunderstandings of concepts and objects in question.

BDD is Behavior-Driven Development; a software development process. TDD is Test-Driven Development; also a software development process.

BDD originated in 2003 with the agiledox tool to generate technical documentation automatically from JUnit tests. A year later, Dan North, in order to test his hypotheses about de-emphasizing the test terminology in favor of behave, released JBehave. In 2006, Dan North in collaboration with Chris Matts proposed the given-when-then template to expand the scope of BDD to business analysis and documented this approach in "Introducing BDD".

BDD practitioners should provide functional documentation in form of user stories augmented with executable scenarios or examples. Instead of referring to tests, the practitioner prefers the terms scenario and specification. Instead of referring to the unit tests of a class, BDD practitioners speak of the specifications of the behavior of the class. Instead of referring to functional tests, BDD practitioners speak of the specifications of the product’s behavior.

BDD requires no particular tools or programming languages, and is primarily a conceptual approach; to make it purely a technical practice is missing the point altogether.

A user story’s behavior is its acceptance criteria. This acceptance criteria can, in BDD, be described using the given-when-then canvas, i.e, context, event, outcome. As acceptance criteria, in BDD, should be executable, it is translated in to code. This code is not the same as a unit test written following TDD practice.

TDD is an extension to unit testing that follows the cycle: 1) write a test, 2) write functional code until the test passes, 3) refactor; while BDD is a refinement of practices from TDD and Acceptance Test-Driven Development (ATTD). Unit testing doesn’t have a clear definition, but there are some common elements. First, the notion that unit tests are low-level and focus on the small parts of a system. Second, unit tests are most often written by programmers using tools. Third, unit tests are expected to be faster than any other tests.

The now classic ATM example illustrates the difference between behavior-driven acceptance criteria translated into code and test-driven unit tests. The acceptance criteria code deals explicitly with behavior and not the intricate functional units of the software code, and is written to satisfy the business. On the  other hand, unit tests deals explicitly with the intricate functional units of the software code.

I hope this explanation makes it clear to see the difference between the two development processes and clears up the terminologies.

[Gmail]

I get BDD it's a style of TDD.  What I am saying is BDD is still unit tests that test drive your code.

[Sebastian Gozin]

Sure.
It's just 2 different actors who write and maintain them.

[Gmail]

I KNOW THIS.  There are "BDD Unit Tests"...driven by user story scenarios.

[Sebastian Gozin]

I'm sorry I misunderstood. It was not intentional.
Odd, I received an e-mail with a question but it is not here...

[Frederik Krautwald]

Executable acceptance criteria written in code are not unit tests, and executable acceptance criteria are the only code BDD is suggesting.

[Dave Schinkel]

yes...I get it.

In mocha, executable acceptance criteria are defined via describe() and context().  Inside those are your BDD Unit tests using it().

[Frederik Krautwald]

I have a hard time understanding why you insist calling these tests unit tests, when they clearly are not. This have been pointed out, to a large degree, in the previous comments.

The code snippets you are referring to using Mocha, or whatever library of your liking, are executable acceptance criteria. If, indeed, you have chosen to encapsulate your actual unit tests within those executable acceptance criteria, I venture to say that you are violating both SRP and SoC. How will you run your unit tests without also executing the acceptance criteria, and how will you execute acceptance criteria without also running the unit tests? The acceptance criteria deal explicitly with the business / product owner’s acceptance of the software. Unit tests deal explicitly with developers’ confidence in the code they write. A unit test can, for example, test a return value of a function which is intermediate and only serves as an input to another function. The acceptance criteria have no interest in these intermediate steps. There exists a clear separation of concerns.

BDD doesn’t describe any "tests", but I already covered that. So calling something "BDD tests" or even "BDD unit tests" is not within the scope of BDD and seriously adds confusion about these self-made terms.

[Dave Schinkel]

How can you say BDD Doesn't include unit tests?    When I say unit tests those are your lower-level tests that which you have used your higher level BDD Acceptance executable tests to "guide" you

BDD includes lower-level unit test which you are saying should be separated out (able to run independently of the BDD executable Acceptance tests that you wrote prior that ultimately drove you to create the lower-level unit tests).  I get your point about making sure they run seperate.  That's where the "glue" comes in however the test framework your'e using allows you to create that glue (e.g. Fitnesse has Fitness wire-able acceptance tests that wire up through an intermediate Java or C# class, or Cucumber for example's glue are the spec definitions that ultimately call the SUT code).

If I'm reading what you said right, your'e saying there are BDD Acceptance executable tests that should be independent of any lower-level code for the SUT (the implementation thereof that make those BDD executable Acceptance tests pass (green).  Right, that makes sense.  

but lets make sure I understand your last reply and that my reply above clears it up. 

[dschinkel]

and when I was saying  "BDD tests"  I was referring to those lower level unit tests.  Not the BDD executable Acceptance specs..just to clarify your first sentence...going back a little before my last reply

[Dave Schinkel]

maybe we should all have a Google Hangout.

[Frederik Krautwald]

But your terminology is confusing. If you mean unit tests, then call them unit tests. BDD, as proposed by Dan North, doesn’t mention anything about unit tests in the workflow. In fact, Dan North replaced JUnit with JBehave to test behavior and wanted to avoid the word test altogether. If you haven’t already, then have a look at Introducing BDD. In fact, BDD is a conceptual approach. It is, as we both agree, an outside-in approach, but it doesn’t dictate unit tests at all. As long as you follow the behavioral scenario and specification approach and the software satisfies the acceptance criteria, it is BDD, whether you have unit tests or not. Thus, there is no such thing as a BDD unit test, at least not in the common terminology of BDD.

That all said, I do understand that since the BDD approach leads from outside-in, when you need to write your classes (if you use classes), it makes sense to also write unit tests for them. However, the unit tests are written to satisfy the developer; not the business or product owner.

I’d like to return to when you wrote initially.

It's still very grey to me how granular you get FROM the BDD tests and the dance in how you get to those lower level tests and how you segregate or if you aren not to segregate them out to cover different layers of your sub app code.

But this is where the dog lies buried. It is grey, because BDD doesn’t test implementation. Implementation is hardly part of the acceptance criteria. Take the typical BDD style user story:

As a customer,

I want to withdraw cash from an ATM,

so that I don’t have to wait in line at the bank.

and a belonging scenario:

Given the account is in credit

And the card is valid

And the dispenser contains cash

When the customer requests cash

Then ensure the account is debited

And ensure cash is dispensed

And ensure the card is returned

This is the detailed acceptance criteria. (A less detailed could simply state: the customer can use a card to withdraw cash.) And it can and should be made executable. However, it doesn’t care at all how the software for the account is programmed, how the card is validated, how the cash enters the dispenser and is kept track of, how the account is debited, how the cash is dispensed, or how the card is returned. All this is left to the developer, i.e., the implementation is completely up to the developer; thus, the unit testing is completely up to the developer. In fact, the developer could leave out unit testing altogether. And, in fact, making the acceptance criteria executable is not a requirement of BDD, i.e., not a must, but a should.

How the developer wants to drive his code is up to the developer. If the developer wants to write unit tests, he is free to do so. He can write them before or after production code. However, following TDD practice, tests come first. In TDD, it is the use case that drives the tests. The above scenario is part of the use case. It is not a full use case, because it lacks alternative routes or outcomes. The developer is trying to satisfy a use case. To be confident that his code satisfies the use case, the developer writes tests, specifically unit tests. These are detached from the acceptance criteria. How the developer figures out which unit tests he needs, is a matter of the developer’s creativity, knowledge and experience. The acceptance criteria surely cannot drive them automatically (at least not yet). If they could, then why have developers? Then the business could just write an executable acceptance criteria and, wooptie, the software is written. No. That’s not how it works, and so, as a consequence, how to become granular from the acceptance criteria or specification is a grey area and is only solved by creativity, knowledge and experience.

Regarding the Hangout, what do you suggest?

[Dave Schinkel]

let me rephrase my last comment

I have to be honest.  I thought I was over complicating things, but hell you bring it to a new level :).   You create your acceptance tests.  Then you have super glue in the middle that decouples you from the lower-level implementation (lower-level implementation driven by TDD Tests - I know you'll cringe, now, I hope you do).  Whether I call the lower-level unit tests BDD Unit Tests, TDD Unit Tests, etc. the key term is "lower level implementation tests" that live on the right side of the super glue, left side having the BDD Acceptance tests that originally drove you to your lower-level unit tests.

AHHH.  I no longer will engage in this silly debate.

On Thursday, September 3, 2015 at 8:49:03 AM UTC-5, Dave Schinkel wrote:

Frederik,

I have to be honest.  I thought I was over complicating things, but hell you bring it to a new level :).  Whether I call the BDD acceptance Tests Unit Tests or not THEY ARE TESTS.  You create your acceptance tests.  Then you have super glue in the middle that decouples you from the lower-level implementation (driven by TDD Tests - I know you'll cringe, now, I hope you do). 

And Like you said:

"when you need to write your classes (if you use classes), it makes sense to also write unit tests for them"

You just said it.  I don't care whether I call the lower-level use case tests TDD tests, unit tests, or BDD Unit tests.  I really don't fing care.

[Frederik Krautwald]

You are, of course, free to call it whatever you like -- keyboard item tests, cake topping, or whatever -- but it is your invented terminology. Here’s what Uncle Bob had to say on the subject:

My definition is: A Unit Test is a test written by a programmer, for that programmer.  
In other words, as a programmer you write a test that satisfies some need that you have.  It doesn't matter what "level" the tests is written at, or whether you are testing a features, or a connection, or a protocol.  What matters is that you, the programmer, needed to write the test for your own reasons.
I define "Acceptance Tests" as tests written by "the business" for "the business".  If the business never writes or reads acceptance tests, and if programmers write them instead, then they are unit tests, and should be written in code, not in some BDD language.  The only reason to write tests in a BDD language is if "the business" is somehow involved.

[Dave Schinkel]

yea I disagree.  Devs can write acceptance tests but based off use case scenarios that the business wrote.  Because I know a friend who says this works quite well.  Their PMs write the Gherkin in the stories.  The developers then take the Gherkin right out of the stories, literally copy and paste into cucumber tests.  And that drives their lower level unit tests.

Take for example Uncle Bob and Micah.  In the Java Series on Clean Coders that's exactly what they did.  Someone wrote some scenarios and they took them and ran with it.

Take for example Corey Haines's video on clean coders.  Same deal.  But since he's writing his own app, for himself, he didn't have the business.  But what I'm saying is he did write the cucumber tests and I'm sure that if he was at a company where the PMs and/or QA with PMs wrote the Gherkin, I bet Corey would take that and run with it and start writing acceptance tests.  I could be wrong, I should ask Corey but that's my assumptions.

Business are never gonna write acceptance tests.  Your PMs are not gonna go into Fitnesse or Cucumber and start writing that trust me.  Their role is defining the scenarios on which us developers take and start from IMO...in code.