Project start: what to do after first accept. test? (example)

[Ole Rasmussen]

I really want to use TDD (GOOS style) on some of my projects, but I struggle so much getting the first code written using TDD, with almost no design in place, that I always give up after some time. I’m trying to follow the GOOS flow: acceptance test à (unit test à implement, unit test à implement…). Let me explain by a contrived but somewhat representative example.

Task: create a class (no gui) that can fetch a list of grades from a website. The grades are fetched by logging in to a HTML form (HTTP post to a specified URL) with a provided username and password and parsing the output of the HTML page listing the grades. A grade consists of number (the grade itself) and the date it was given (a string).

So here we are with a simple requirement and NO design whatsoever. I must start by writing an acceptance test, the question is a test of what? Well the first test must establish the “walking skeleton” and must verify that my app can fetch grades from the "real" website. Also the only feature right now is to fetch the grades, so I can’t think of anything else to test:

AcceptanceTests.java:
public void canLoginAndFetchGrades() {
  GradeFetcher g = new GradeFetcher(“http://someUrlWithLoginForm.com”)
  List<Grade> grades = g.FetchGrades(“someUsername”, “somePassword”)
 
  // Check that all the grades are correct
 assertThat(grades.get(0).getNumber(), is(equalTo(10)));
 assertThat(grades.get(0).getDate(), is(equalTo(“11-02-2012”)))
 // .. and so on for the rest

I create all types so the test compiles and see it fail because it just returns an empty list of grades. Now what? This is usually where it breaks down for me: I have written the first acceptance test but don’t know what to do next. This seems like a good acceptance test: when this test passed the software probably works as intended.
I think I need to write a unit test before implementing any behavior, but the question is what should I test? There is obviously a lot of functionality needed for this acceptance test to pass: sending the correct HTTP request with the correct headers etc, parsing the response correctly, inserting correctly into arraylist and so on. I have a feeling that I want to test those in separation so I can grow the application bit by bit, I just don’t know where to start.

In the end my confusion may be a result of a misunderstanding of what an acceptance test is, when I should write one but most important what to do afterwards. It may also be only that I just don't know what to do afterwards:)
I have always thought of an acceptance test as a “feature” that is visible to and required by the customer (me, as a user of the API), and I believe that is also the view in GOOS. In my example I would classify all these as things that should be tested by acceptance tests:
- Can login and fetch grades
- Handle invalid credentials
- Handle invalid url
- Handle timeout (of HTTP request)

These “features” are what is visible to the customer. I imagine that there are other things that are NOT relevant to the customer like:
- How exactly the login sequence is handled (what http request is sent, what headers is in it etc)
- How the output on the HTML page is parsed

These things are obviously NOT relevant to a user of the GradeFetcher class so they are not acceptance criteria. They must however be tested by unit tests somewhere inside the system.

[Matteo Vaccari]

Hello Ole.

I tell you what I would do. I would start by commenting out or ignoring the AT; then I would write a first test for the GradeFetcher.

I'm a GradeFetcher, and I receive a request to fetch grades.  What do I do?  I must talk to an HTTP server and parse the HTML it returns. Since I don't want my unit test to depend on what happens to be on the real server, I could just decide that I have a collaborator that does just that and returns my html.  Since the format of the html might be complicated and I don't think I want to deal with this right now, I might assume another collaborator that does just that.  So my first unit test for the GradeFetcher would probably be something that defines the communication protocol with these two collaborators. I would use JMock to prototype their interfaces and get the first test to pass.

And so on.  I am a GradesHtmlParser, they tell me to parse this HTML, what do I do? :-)

Matteo

[Donaldson, John]

Yes, I agree with Matteo. Also, I would want to separate the login from the fetching. So two acceptance tests:

-          Can login

-          Can fetch grades

 

John D.

[Ole Rasmussen]

Those are really nice suggestions. But why would you ignore the AT? According to GOOS we must always write an AT before the feature we are about to implement, right?. I also like the fact that I can see when I'm finished; when the AT passes.

[Matteo Vaccari]

I would ignore the AT it until it passes :)  It usually takes many unit tests before the AT passes, and I don't want to work with the red bar.

This is an example of the Child Test pattern (see TDD By Example). If a test is too big, comment it out and write a smaller one.

Matteo

[Ole Rasmussen]

Oh that was what you meant. I completely agree then. I thought you meant ignore it forever aka. delete it :)

[Josue Barbosa dos Santos]

I like to see if I finished the feature, so I don´t comment the acceptance test. Also it guides me to what is the next thing that I should do to complete the feature.

What I do is to create two sets of tests. Acceptance tests and unit tests. Unit tests must be fasts and and red only by minutes (seconds is preferable). Acceptance tests don´t need do be so fast and can be in red for a longer time than unit tests.

-- 
Abraços,
Josué
http://twitter.com/josuesantos

[Nikolay Sturm]

* Ole Rasmussen [2013-01-22]:

> I really want to use TDD (GOOS style) on some of my projects, but I
> struggle so much getting the first code written using TDD, with almost
> no design in place, that I always give up after some time.

I have exactly the same problem. What I had some success with recently,
was starting with hardcoded data. In your case, I would just implement
enough of GradeFetcher to return canned data that satisfies the test.
With a passing test in place, I would slowly implement real behaviour,
replacing my canned data.

HTH,

Nikolay

[Rick Pingry]

I have been playing with exactly what Nikolay is talking about...  

1. Creating a canned version of a GradeFetcher class that just returns what I want and makes the Acceptance Tests pass
2. Then use Unit Tests that start with the interface defined in the acceptance tests, completely separately, not connected with the actual code
3. When complete with the actual GradeFetcher NEEDED implementation (GradeFetcherHtml ?), the refactoring involves replacing the canned GradeFetcher class with the actual one, and if done correctly, the acceptance tests should still pass.

This methodology seems consistent at the Unit Testing and TDD as well.  Nikolay et.al., how has that been working for you?  I am just getting started but I like it so far.  I am concerned that there may be places where I stub out too much in the beginning, and I end up shipping with canned stuff accidentally because the Acceptance Tests still pass.  Can we come up with a rigorous way to make sure we are replacing all of the canned implementations with the actual ones?

The first thing that comes to mind is that you only have one Acceptance Test at a time, so you only work on that one.  This bothers me a little for two reasons.  One is that this canning what we want (purposefully using the term "Canned" rather than a fake or stub because of their overloaded meanings.  Fakes, Stubs, and Mocks are used only during isolationist using testing, whereas this canned stuff is used in actual production code, but only temporarily)  based development process is recursive as I work from the Inputs to the Business Logic to the Inputs.  As I build this, I may have various canned classes in place until they are all removed.

In addition to making these depth plunges with canned implementation classes, I am making a greater effort lately to focus on User Stories.  I mean to really focus on the User Interface and be able to use Canned implementations very early on through the entire User Story.  Rather than having functional tests at every little step (or perhaps in addition to), we start with creating an Acceptance Test that walks all the way through a User Story and get that done as early as possible.  My clients are enamored with watching these functional acceptance tests run and see them as great documentation along with the users guide.  This running acceptance test serves as a tutorial.  

Doing this gives me feedback very early on from the stake-holders (yeah!).  As far as they are concerned, it looks "Finished" but its results are canned for a particular scenario.  I imagine there will be lots of "Canned" classes that we will then start implementing, and this is where the real work on the business logic happens.  Can anyone think of a rigorous method for making sure I replace all my canned stuff?

I can think of maybe 2.

1. Write a test that is slightly different from the acceptance test that would cause the canned code to fail...
2. Use a naming convention for the classes (maybe include in the name "Canned") and systematically delete the canned implementations and cause the existing tests to fail until the real implementations are properly replaced.

I kind of like 2 better.  It seems like with #1 I can still forget, or that to write the variant I would need to already know I was trying to discover :O

Thanks once again for being a group I can think through my ideas on :D

Ideas?

[Ole Rasmussen]

JohnD:

> Yes, I agree with Matteo. Also, I would want to separate the login from the fetching. So two acceptance tests:

> - Can login

> - Can fetch grades

Makes sense. The problem is there is no way to tell (from the outside) whether the GradeFetcher has logged in or not. It's a part of a larger operation (fetching the grades), not an operation in itself. And we cannot (or want not) change the interface.

Do you see any way we could test this (with an AT)? Maybe verifying some interactions, but I'm not so sure it's possible here.

Nikolay Sturm:

>What I had some success with recently, 

>was starting with hardcoded data. In your case, I would just implement 

>enough of GradeFetcher to return canned data that satisfies the test. 

>With a passing test in place, I would slowly implement real behaviour, 

>replacing my canned data.

It's an interesting idea but doesn't it mute the usefulness of ATs? One of the properties of an AT is that when it passes we are -done-. Done as in 'tell the customer this is done'. With your solution ATs will pass when we are not even close to being done. Even worse they will also pass when we ARE done (replace the canned impl. with a real impl) and thus "passed" now means two different things. When an AT is passing you won't be able to tell why, which in my opinion is exactly what we SHOULD be able to do with an AT.

I -think- this concern is actually what Rick is describing in his post. ATs passing for more than one reason create all sorts of problems like he describes. For example the possibility of accidentally shipping with canned implementations.

[Nikolay Sturm]

* Ole Rasmussen [2013-01-24]:

> >What I had some success with recently, was starting with hardcoded
> >data. In your case, I would just implement enough of GradeFetcher to
> >return canned data that satisfies the test. With a passing test in
> >place, I would slowly implement real behaviour, replacing my canned
> >data.

> I -think- this concern is actually what Rick is describing in his
> post. ATs passing for more than one reason create all sorts of
> problems like he describes. For example the possibility of
> accidentally shipping with canned implementations.

Let me first remark, that using canned data serves a special purpose,
overcomming my initial paralysis and kickstarting my TDD process. It's a
training wheel I hope to get rid of eventually. As every technique, it
has its drawbacks. The goal should certainly be to write code without
canned data. But for now, I find this technique helpful.

To answer your concerns, when I use canned data, I push it from class to
class until it reaches its final destination. There, unit tests ensure
proper behaviour and removal of canned values. If you want to be extra
sure, you can also vary the inputs and outputs in your acceptance tests.

HTH,

Niklay

--
"It's all part of my Can't-Do approach to life." Wally

[Donaldson, John]

Ole,

 

I may have assumed too much, but I expected you would have, not only “fetching grades” but also other operations. All of which needed to be done while logged in. And that it would be possible to log in, and log out without doing anything.

 

So this would seem to need three stories:

-          A user can log in with known account and password (and fail otherwise)

-          A logged-in user can do an operation (if not, then they can’t)

-          A logged-in user can fetch grades (I’m going to fake a “logged in” state, credentials or whatever to make this pass)

 

John D.

 

From: growing-object-o...@googlegroups.com [mailto:growing-object-o...@googlegroups.com] On Behalf Of Ole Rasmussen
Sent: 24 January 2013 20:17
To: growing-object-o...@googlegroups.com
Subject: Re: [GOOS] Project start: what to do after first accept. test? (example)

 

JohnD:

--
 
 
 

[Steve Freeman]

One common technique is not to start with logging in. If you think in terms of user workflow it's an obvious first step, if you think in terms of functionality it's less obvious. So an option is to start with either a single user system or to force the connection to be logged in.

S

[Colin Vipurs]

We take this approach, but one thing we do is make it obvious in the
AT that you're returned canned data and when you move on to returning
real data modify the test to state that's what you're doing.
Depending on what you're doing will affect how you do this, but having
your expected return values identify them as canned has worked well
for us.

As an example, in one of our front end systems, data travels through 2
internal services, so the first pass at the acceptance test would have
a Then phase of:

Then I received 'hardcoded data from the front end service'

Data here is hardcoded in the front end service (as the test states
explicitly). When this passes, we commit and move on and modify the
test to:

Then I received 'hardcoded data from the first internal service'

And then on to

Then I received 'hardcoded data from the final internal service'

And finally

Then I received 'real data from the final service'

Obviously the strings above are actually domain specific pieces of
information that identify themselves as where they came from.

[Steve Freeman]

However you get there, at the end the test should show what is going in and what is coming out of the system.

I'm not sure I'd do that sort of 'then' clause, it make more sense to me to make a 'when' clause--which pumps content in--and assert what should happen at the outside of the system.

One approach is to mark the test as unfinished and fail progressively later in the test as you complete features until it's all done.

S

[Alexandre Aquiles]

Maybe a @WithCannedData (or @WorkInProgress) annotation would be useful.

There's a a project called Pending that provides annotations and JUnit rules/categories to mark tests as "pending". It is slightly different from "canned" because it mutes failure reports for tests that are expected to fail.

Alexandre Aquiles

about.me/alexandreaquiles

PS: Cucumber doesn't run features and scenarios tagged with @wip by default. That is also what Thucydides' @Pending does.

--

[Steve Freeman]

even better, another trick is an annotation/markup that fails if the test passes. No unseen surprises...

S

On 25 Jan 2013, at 15:25, Alexandre Aquiles wrote:
> Maybe a @WithCannedData (or @WorkInProgress) annotation would be useful.
>

> There's a a project called Pending <https://github.com/ttsui/pending> that

[Alexandre Aquiles]

Actually, that's exactly what Pending does: 

"A pending test is expected to fail. Should the test pass the unexpected success will be reported."

The nice thing is that, when you're done, Pending will make the now passing Acceptance Test to fail, because it's not "pending" anymore!

It seems like the "GOOS approach".

I think you wouldn't want a @PendingImplementation Acceptance Test for too long. It would be work half done.

___

The "canned data" approach (correct me if I'm wrong) seems to work as a prototype: the expected outputs are canned. You would do this to avoid being stuck, clear your thoughts and learn what's expected.

So, the AT will pass, but you should tag it as a different kind of successful AT.

Alexandre Aquiles

about.me/alexandreaquiles

--

[J. B. Rainsberger]

On Fri, Jan 25, 2013 at 1:38 AM, Nikolay Sturm <goo...@erisiandiscord.de> wrote:

 

Let me first remark, that using canned data serves a special purpose,
overcomming my initial paralysis and kickstarting my TDD process. It's a
training wheel I hope to get rid of eventually. As every technique, it
has its drawbacks. The goal should certainly be to write code without
canned data. But for now, I find this technique helpful.

I do it all the time, and I love it. I get to focus on one thing at a time. I hardcode data so that I can do a little exploratory testing while I build some little part of the feature. Replacing hardcoded data with a lookup in a table somewhere (wherever) is so easy, I don't need to prove up front that I can do it. It can wait.

Not a training wheel for me at all.

-- 

J. B. (Joe) Rainsberger :: http://www.myagiletutor.com :: http://www.jbrains.ca ::
http://blog.thecodewhisperer.com
Free Your Mind to Do Great Work :: http://www.freeyourmind-dogreatwork.com

[J. B. Rainsberger]

On Fri, Jan 25, 2013 at 11:25 AM, Alexandre Aquiles <alexandr...@gmail.com> wrote:

 

Maybe a @WithCannedData (or @WorkInProgress) annotation would be useful.

There's a a project called Pending that provides annotations and JUnit rules/categories to mark tests as "pending". It is slightly different from "canned" because it mutes failure reports for tests that are expected to fail.

I hate to be "that guy", but I find writing this down on an index card to remind me to generalise the implementation works great. What am I missing?

-- 

[Alexandre Aquiles]

Yeah, you're right. The simpler, the better!

--
 
 
 

--
Alexandre Aquiles

about.me/alexandreaquiles

[Nikolay Sturm]

* J. B. Rainsberger [2013-01-26]:

> I do it all the time, and I love it. I get to focus on one thing at a time.
> I hardcode data so that I can do a little exploratory testing while I build
> some little part of the feature. Replacing hardcoded data with a lookup in
> a table somewhere (wherever) is so easy, I don't need to prove up front
> that I can do it. It can wait.
>
> Not a training wheel for me at all.

Good to know I am not on the wrong track with this, thanks.

cheers,

Nikolay