Acceptance Tests: a very newbie question

[NS]

Hello,

I'm hearing a lot of good things about the GOOS book.

But...

Does this book discuss or show examples(s) on how to write end-to-end acceptance tests for one or more of these flavors of applications:

   1. Java EE based web applications

   2. Swing based desktop applications

   3. general, MVC-based applications with GUI (ie, not console-based client/server type of applications)

I'm new to TDD (and even automated testing), so I'm not sure what tools typically get used for writing end-to-end / Acceptance Tests just as...

   4. JUnit gets used for Unit Tests

   5. jMock gets used for Mock Tests

Would the forum members kindly share the tools they are using for Acceptance Tests?

In essence, I don't get ALL of the following all together: 

  6. In TDD, you are supposed to write tests first, before ANY code gets written! 

  7. The GOOS book is (apparently) recommending to write Acceptance Tests (over excessive class-level Unit Tests).

  8. But the GUI is not ready YET... not before the first line of code has been written. A chicken-and-egg problem?

  9. For a GUI, MVC-based app, is it enough to have the Acceptance Tests driven by the Controller? Or, must they be driven by the View?

Would greatly appreciate your kind responses and insights. I'm in a great hurry to adopt TDD but am feeling truly overwhelmed by the literature around it. Some books on TDD appear to be almost a decade old, so I'm not sure if the techniques covered in them would still be the state-of-art in 2015. In other words, I won't be able to tell from these books themselves (even if I could purchase all of them AND read all of them cover to cover) if their content is still valid and has not been superseded by something better.

Many thanks in advance.

Regards,

/NS

[Daniel Wellman]

Hi NS!

I can relate; when I started I felt overwhelmed and wasn't sure where to start. It took me a while to learn how to apply the techniques in different situations, and after many years I am still learning. I have found that ultimately practicing this helps me make my code more flexible and reliable, and I quite enjoy both practicing TDD and the relief I get from having confidence in my code. 

It may not be easy, but I have found it to be worth it!

My answers below:

On Aug 29, 2015, at 11:19 AM, NS <pingn...@gmail.com> wrote:

Hello,

I'm hearing a lot of good things about the GOOS book.

But...

Does this book discuss or show examples(s) on how to write end-to-end acceptance tests for one or more of these flavors of applications:

   1. Java EE based web applications

   2. Swing based desktop applications

   3. general, MVC-based applications with GUI (ie, not console-based client/server type of applications)

There is a detailed example of building an application with a Swing GUI that talks to a remote service over XMPP. In addition to covering some of the fundamental techniques, there are sections on dealing with databases and multithreaded applications. There's not specifically a chapter on HTTP (if I recall correctly) but there are several references and blog posts online - and discussions in this forum about that. 

I'm new to TDD (and even automated testing), so I'm not sure what tools typically get used for writing end-to-end / Acceptance Tests just as...

   4. JUnit gets used for Unit Tests

   5. jMock gets used for Mock Tests

Would the forum members kindly share the tools they are using for Acceptance Tests?

For Java, I prefer JUnit and JMock with Hamcrest matchers. Sometimes I use Cucumber for acceptance tests. 

In essence, I don't get ALL of the following all together: 

  6. In TDD, you are supposed to write tests first, before ANY code gets written! 

Yes.

  7. The GOOS book is (apparently) recommending to write Acceptance Tests (over excessive class-level Unit Tests).

No, it shows how to start with an acceptance test and use unit tests to drive out the design of your system. 

  8. But the GUI is not ready YET... not before the first line of code has been written. A chicken-and-egg problem?

I didn't understand this. Did you mean that you are working on an application and the GUI is being designed separately?

  9. For a GUI, MVC-based app, is it enough to have the Acceptance Tests driven by the Controller? Or, must they be driven by the View?

It depends. If you can design your application on a way that avoids a lot of logic in the view, you can get a lot of confidence in your acceptance tests by avoiding the view. These tests tend to be faster and more reliable than tests that use the view. 

Typically I find a need a balance of a few view-driven tests to make sure everything is hooked together correctly. Then I prefer more fast unit tests, with a few slower tests that check the parts that integrate with the external inputs and outputs - eg the database, talking to web services, etc.

Would greatly appreciate your kind responses and insights. I'm in a great hurry to adopt TDD but am feeling truly overwhelmed by the literature around it. Some books on TDD appear to be almost a decade old, so I'm not sure if the techniques covered in them would still be the state-of-art in 2015. In other words, I won't be able to tell from these books themselves (even if I could purchase all of them AND read all of them cover to cover) if their content is still valid and has not been superseded by something better.

If you've never done TDD before you may want to try Kent Beck's "TDD by Example" first. It's a shorter book and doesn't get into detailed like HTTP or databases, but I think it explains the fundamentals quite well. Given the example architectures you were looking at, I'd suggest following with GOOS after that one. 

Dan

Many thanks in advance.

Regards,

/NS

--

---
You received this message because you are subscribed to the Google Groups "Growing Object-Oriented Software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to growing-object-oriente...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Matteo Vaccari]

Hi NS,

Some of the answers to your questions are covered in the GOOS book.  If you are looking for an alternative  (or a prequel) to GOOS, I suggest TDD By Example by Kent Beck. It's still very relevant.  Also the videos by Kent Beck published with Pragmatic Press.

Anyway, the way to learn from a TDD book is to follow along the examples with your IDE.

My suggestion to get started is to concentrate first on doing TDD well and pospone learning about end to end AT.

Matteo

[NS]

Hi Daniel,

On Saturday, August 29, 2015 at 10:12:34 PM UTC+5:30, Daniel Wellman wrote:

On Aug 29, 2015, at 11:19 AM, NS <pingn...@gmail.com> wrote:

  8. But the GUI is not ready YET... not before the first line of code has been written. A chicken-and-egg problem?

I didn't understand this. Did you mean that you are working on an application and the GUI is being designed separately?

 

I've read your response for point #9 but let me elaborate anyway what I meant to say above.

My understanding of an end-to-end acceptance test is, a test that:

• emulates an enduser (who will 'accept' the delivered system, finally)...
  
• exercising all the major layers of the system...
  
• deployed in (almost) a realworld- / Production-like environment. 

Which means, you must use tools like Selenium, LoadRunner, or Grinder (although the latter two are used primarily for performance testing) that allow you to record all the GUI actions of yours in order to (easily) build your acceptance test. 

In absence of such tools, you'd have to manually (and very tediously) build the equivalent GUI driving 'script' yourself somehow.

So, what I was trying to say in my original post was, 

    IF, 

        a) given that an end-to-end acceptance test is not *supposed* to be driven

           by the non-visual, non-iteractive Controller code behind a GUI View but

           driven rather by the GUI View itself;

           

                *AND*

                

        b) given that TDD requires the (acceptance) tests to be written first

        

    THEN,

        doesn't it become a chicken-and-egg problem (unless of course you 

        choose to tediously build the View driving script manually)? 

 

  9. For a GUI, MVC-based app, is it enough to have the Acceptance Tests driven by the Controller? Or, must they be driven by the View?

It depends. If you can design your application on a way that avoids a lot of logic in the view, you can get a lot of confidence in your acceptance tests by avoiding the view. These tests tend to be faster and more reliable than tests that use the view. 

 

Well, what if the View is highly rich / interactive / GUI-event-driven, containing not business logic but rather enough View-state and View-logic to warrant its (automated regression) testing, *AND* such a View happens to be part of the acceptance test demanded by the end-user? For example, 

• The enduser may demand that "if I click widget 1, then widgets 2, 3, and 4 must be disabled, etc. 
  
• Some dropdowns may list different choices based on enduser's interaction with other widgets in the same View.
  
• Etc.
  

 

Typically I find a need a balance of a few view-driven tests to make sure everything is hooked together correctly. Then I prefer more fast unit tests, with a few slower tests that check the parts that integrate with the external inputs and outputs - eg the database, talking to web services, etc.

While speed matters and is obviously welcome, I'm not so much worried about speed, especially in the context of end-to-end acceptance testing when one of its big (potential) gifts (as I understand so far) could be an automated regression testing of the whole system -- from the GUI down to the database. Thus, I would very much want to include my highly interactive GUI in my acceptance test coverage, if possible.

Maybe, I'm missing the definition of an acceptance test in context of TDD. 

Regards,

/NS 

[NS]

Thanks Matteo, will check the book out. I did run into this book, btw, in my googling earlier but didn't know existence of the videos. Will check'em out too.

Regards,

/NS

[Daniel Wellman]

Hi NS,

On Sat, Aug 29, 2015 at 10:36 PM, NS <pingn...@gmail.com> wrote:

My understanding of an end-to-end acceptance test is, a test that:

• emulates an enduser (who will 'accept' the delivered system, finally)...
  
• exercising all the major layers of the system...
  
• deployed in (almost) a realworld- / Production-like environment. 

Which means, you must use tools like Selenium, LoadRunner, or Grinder (although the latter two are used primarily for performance testing) that allow you to record all the GUI actions of yours in order to (easily) build your acceptance test. 

If it's truly end-to-end, then yes, it would exercise all the major layers of the system through however a user uses the system.  

That said, you may not need an acceptance test for *every* feature - sometimes you might decide unit tests are enough.  And you may not want *every* acceptance test to be end-to-end; if you think about what code an acceptance test exercises, it exercises your code as well as all the underlying third-party code like your application's core libraries, GUI libraries, database driver code, database, operating system, etc.  As some point I get enough confidence that all those components are working together correctly and I don't need to prove that the next feature exercises all those components correctly in addition to the relatively small amount of new code I'll write or refactor.

In absence of such tools, you'd have to manually (and very tediously) build the equivalent GUI driving 'script' yourself somehow.

Yes, exactly.  But I don't prefer to use a tool to record all the GUI actions - I find the scripts that GUI recording tools tend to be hard to understand and frequently brittle.  Those tools can be fast to generate the script the first time, but as the GUI changes I find they tend to break or be hard to understand (or worse don't work after the first recording!)  Instead, I prefer to write the test first and think about *how* to make it easy (or as easy a possible) to automate my GUI code.  For web UIs, that means thinking about my CSS or DOM structure and adding little hooks to make it easy to find the elements I want -- such as DOM IDs for unique buttons, CSS selectors for describing the relation between components, etc.

 

So, what I was trying to say in my original post was, 

    IF, 

        a) given that an end-to-end acceptance test is not *supposed* to be driven

           by the non-visual, non-iteractive Controller code behind a GUI View but

           driven rather by the GUI View itself;

           

                *AND*

                

        b) given that TDD requires the (acceptance) tests to be written first

        

    THEN,

        doesn't it become a chicken-and-egg problem (unless of course you 

        choose to tediously build the View driving script manually)? 

I choose to build the view driving script manually and use the feedback from creating those tests to help me design my UI so it's easy to automate.  It is sometimes tedious to build those GUI automation steps and I don't always enjoy it, but I almost always enjoy the end result much more than a record-and-playback tool.  I also don't always make my acceptance tests end-to-end.

 

 

Well, what if the View is highly rich / interactive / GUI-event-driven, containing not business logic but rather enough View-state and View-logic to warrant its (automated regression) testing, *AND* such a View happens to be part of the acceptance test demanded by the end-user? For example, 

• The enduser may demand that "if I click widget 1, then widgets 2, 3, and 4 must be disabled, etc. 
  
• Some dropdowns may list different choices based on enduser's interaction with other widgets in the same View.
  
• Etc.
  

You can write GUI code in a way that is more easily unit testable, too.  Sometimes you can do it without using the actual GUI components and instead fake them out -- see some of the articles like "The Humble Dialog Box" by Michael Feathers.  It's an older article, but I think the philosophy and principles can still be helpful.

  

Typically I find a need a balance of a few view-driven tests to make sure everything is hooked together correctly. Then I prefer more fast unit tests, with a few slower tests that check the parts that integrate with the external inputs and outputs - eg the database, talking to web services, etc.

While speed matters and is obviously welcome, I'm not so much worried about speed, especially in the context of end-to-end acceptance testing when one of its big (potential) gifts (as I understand so far) could be an automated regression testing of the whole system -- from the GUI down to the database. Thus, I would very much want to include my highly interactive GUI in my acceptance test coverage, if possible.

The problem is that automating those external components you can't easily control (like databases, third-party web services, etc.) and those that are very sensitive to timing issues (like the GUI and multi-threaded components) is that the tests require a lot of work to make repeatable and stable.  The more them you have, the more susceptible your tests are to being flaky, and when a test is flaky I find I start to lose confidence in my build, and this impedes my progress on delivering new, useful features.  If you run 100 of these automated tests in a build and they are even 99% reliable, that means you're probably going to have one test fail on every run.  Some teams run their end-to-end automated acceptance tests in a separate build a few times a day and treat those failures differently than the unit tests failing.

That said, I don't think there's an exact formula for how to do this process that every team will follow -- you may need to try something, see how it works for you and your team, and adjust appropriately.

 

Maybe, I'm missing the definition of an acceptance test in context of TDD. 

From what you describe, my interpretation is that your primary point of confusion was thinking that the GUI tests must be written using a record-and-playback tool.  Did I get that right?  Otherwise, it sounds like you are on the right path!

And since it sounds like you're new to TDD, and possibly your team is too, remember that when you try something new, you'll probably go through a period where you are far less productive than you were before doing TDD.  That part is completely normal -- in one model of how change works, that's called the "Chaos Stage" (http://stevenmsmith.com/ar-satir-change-model/).  It can be very tempting in that stage to say "This isn't working, let's go back to what we used to do!"  You might want to find others who can help you and support you through this change, plan extra slack (possibly quite a lot, depending on the risk inherent in the project) in your delivery schedule, and leave time to discuss your progress and learnings as a team.

You also might find that if you and your team are new to TDD, you may not want to start by practicing end-to-end acceptance test-driven development.  I find the end-to-end parts the *most difficult* to get right, largely due to difficulties with building reliable test infrastructure across multiple processes.  It might be easier to start by practicing TDD on a smaller scale where you can learn to listen to the tests and see how the feedback from TDD guides you to build your production code. 

Cheers,

Dan

[NS]

On Sunday, August 30, 2015 at 11:15:39 PM UTC+5:30, Daniel Wellman wrote:

On Sat, Aug 29, 2015 at 10:36 PM, NS <pingn...@gmail.com> wrote:

Maybe, I'm missing the definition of an acceptance test in context of TDD. 

From what you describe, my interpretation is that your primary point of confusion was thinking that the GUI tests must be written using a record-and-playback tool.  Did I get that right?  Otherwise, it sounds like you are on the right path!

Yes, that's correct.  

Regards,

/NS

[Nat Pryce]

On 30 August 2015 at 03:36, NS <pingn...@gmail.com> wrote: 

    IF, 

        a) given that an end-to-end acceptance test is not *supposed* to be driven

           by the non-visual, non-iteractive Controller code behind a GUI View but

           driven rather by the GUI View itself;

           

                *AND*

                

        b) given that TDD requires the (acceptance) tests to be written first

        

    THEN,

        doesn't it become a chicken-and-egg problem (unless of course you 

        choose to tediously build the View driving script manually)? 

If you test through the GUI and write tests first AND REFACTOR RELENTLESSLY then refactoring away duplication will drive abstractions out of your test code that model the user's interactions with the system. Your next tests written against those abstractions will not be so tedious to write. And as you continue to write tests and refactor, those abstractions will improve, becoming more expressive, and more closely modelling the tasks of the users and the business domain the users are working in.  

Pretty soon it should  be MUCH faster to write tests first than to use a record/replay GUI testing tool. In projects where I've run a lot of acceptance tests through the GUI, writing new tests eventually involved a few minutes sitting at the IDE with a domain expert (business analyst, client, or whoever).

--Nat

 

http://www.natpryce.com

[NS]

That sounds like *arriving* at Model-View-Presenter (MVP), but *via* refactoring. If yes, then owing to its TDD-friendliness out-of-the-box among other benefits (like Presenter reuse, loose View-Model coupling), I might as well *start out with* MVP, especially its "dumb-View" flavor. I could, then, call my entire Presenter layer as pretty much the GUI, and in turn, call my Presenter-driven end-to-end test an acceptance test.

[Yoann R.]

Some say that one of the major thing achieved through testing first is that from the tests should emerge design.

If you can, try to avoid any hypothesis about pattern you could use. Let them in some dark place in your brain, and use them when your process will lead you to them.

Actually, it is like waiting for the question to provide the answer.

I hope I do not speak rubbish like  some green Jedi I know of... Whenever I have to teach TDD to colleagues, it always cumbersome to tell them how to do it... It often ends with trust, that I should lead them to some nice place at the end... :)

Have a nice day...

[Nat Pryce]

On 1 September 2015 at 05:26, NS <pingn...@gmail.com> wrote:

If you test through the GUI and write tests first AND REFACTOR RELENTLESSLY then refactoring away duplication will drive abstractions out of your test code that model the user's interactions with the system. Your next tests written against those abstractions will not be so tedious to write. And as you continue to write tests and refactor, those abstractions will improve, becoming more expressive, and more closely modelling the tasks of the users and the business domain the users are working in.  

Pretty soon it should  be MUCH faster to write tests first than to use a record/replay GUI testing tool. In projects where I've run a lot of acceptance tests through the GUI, writing new tests eventually involved a few minutes sitting at the IDE with a domain expert (business analyst, client, or whoever).

That sounds like *arriving* at Model-View-Presenter (MVP), but *via* refactoring. If yes, then owing to its TDD-friendliness out-of-the-box among other benefits (like Presenter reuse, loose View-Model coupling), I might as well *start out with* MVP, especially its "dumb-View" flavor. I could, then, call my entire Presenter layer as pretty much the GUI, and in turn, call my Presenter-driven end-to-end test an acceptance test.

I was talking about the structure of the *test code*, not the application itself.  To test through the GUI the tests need to: 

 * query what's showing in the UI

 * send events to the UI

 * synchronise with asynchronous behaviour of the UI (and rest of the system)

 * model the structure of the UI screens/dialogs/pages so that each is defined only in one place in your code

 * model the navigation routes around the UI

All that would be, as you say, quite tedious to write if you had to write it anew for every test.  But if you continually refactor your test code, then abstractions will emerge from the refactoring that make it easier and easier to write tests.

At the same time, continually refactoring the system as it grows will (in my experience and as we described in GOOS) drive the design of the system towards a Ports-and-Adapters* architecture, so that you can test more and more significant behaviour of the system without testing through the UI at all.

--Nat

* aka Hexagonal or Ring architecture. 

 

--

---
You received this message because you are subscribed to the Google Groups "Growing Object-Oriented Software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to growing-object-oriente...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

http://www.natpryce.com

[Matteo Vaccari]

On Tue, Sep 1, 2015 at 1:22 PM, Nat Pryce <nat....@gmail.com> wrote:

On 1 September 2015 at 05:26, NS <pingn...@gmail.com> wrote:

If you test through the GUI and write tests first AND REFACTOR RELENTLESSLY then refactoring away duplication will drive abstractions out of your test code that model the user's interactions with the system. Your next tests written against those abstractions will not be so tedious to write. And as you continue to write tests and refactor, those abstractions will improve, becoming more expressive, and more closely modelling the tasks of the users and the business domain the users are working in.  

Pretty soon it should  be MUCH faster to write tests first than to use a record/replay GUI testing tool. In projects where I've run a lot of acceptance tests through the GUI, writing new tests eventually involved a few minutes sitting at the IDE with a domain expert (business analyst, client, or whoever).

That sounds like *arriving* at Model-View-Presenter (MVP), but *via* refactoring. If yes, then owing to its TDD-friendliness out-of-the-box among other benefits (like Presenter reuse, loose View-Model coupling), I might as well *start out with* MVP, especially its "dumb-View" flavor. I could, then, call my entire Presenter layer as pretty much the GUI, and in turn, call my Presenter-driven end-to-end test an acceptance test.

I was talking about the structure of the *test code*, not the application itself.  To test through the GUI the tests need to: 

 * query what's showing in the UI

 * send events to the UI

 * synchronise with asynchronous behaviour of the UI (and rest of the system)

 * model the structure of the UI screens/dialogs/pages so that each is defined only in one place in your code

 * model the navigation routes around the UI

All that would be, as you say, quite tedious to write if you had to write it anew for every test.  But if you continually refactor your test code, then abstractions will emerge from the refactoring that make it easier and easier to write tests.

At the same time, continually refactoring the system as it grows will (in my experience and as we described in GOOS) drive the design of the system towards a Ports-and-Adapters* architecture, so that you can test more and more significant behaviour of the system without testing through the UI at all.

I agree with all of the above.  Yet it is a bit daunting if you're just starting.  A faster way to get started is to start building a page of the UI that shows canned data returned from a function.  You can easily check manually that it works in the browser (or by restarting it if it's a Swing app).  Then you start test-driving the real behaviour you need in the function that returns the canned data (rewriting it gradually to something that returns the real data.)  This process is not as good as GOOS, but it makes it easier to get started, expecially in environments where the machinery to drive the UI is cumbersome (e.g., Android!)

Matteo

[Steve Freeman]

I would suggest not getting too hung up about completeness if you’re completely new to the technique.

Start with the bits you can understand and work from there - but /keep pushing/. Unless you’ve gone too far, you can’t know where the boundaries really are.

For videos, I quite like JB Rainsberger’s new series.

S

[Nat Pryce]

Also JB's talk, Integrated Tests are a Scam, is v good overview on how to avoid end-to-end tests altogether.

--

---
You received this message because you are subscribed to the Google Groups "Growing Object-Oriented Software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to growing-object-oriente...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
http://www.natpryce.com

[Matt Wynne]

On 1 Sep 2015, at 11:55, Steve Freeman <st...@m3p.co.uk> wrote:

For videos, I quite like JB Rainsberger’s new series.

May I also humbly recommend https://cucumber.io/school ?

:D

cheers,

Matt

---

ma...@mattwynne.net

[Steve Freeman]

OK, as long as it's clear that Cucumber is not required :)

S

[Matt Wynne]

You’re rocking my world Steve.

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "Growing Object-Oriented Software" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to growing-object-oriente...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

cheers,
Matt

---
ma...@mattwynne.net

[Suhas Walanjoo]

Nat,

Could you please provide links to the talks you recommended?

[Nat Pryce]

On 8 September 2015 at 14:26, Suhas Walanjoo <suhas.w...@gmail.com> wrote:

Nat,

Could you please provide links to the talks you recommended?

There are a bunch of videos online.  Not sure which is best.

https://www.google.co.uk/search?q=integrated%20tests%20are%20a%20scam

--

http://www.natpryce.com

[Suhas Walanjoo]

Thanks a lot, Nat!

[Daniel Wellman]

And J. B.'s online TDD intro video series is here:

http://online-training.jbrains.ca

I highly recommend it.

Dan

[Serkan Camurcuoglu]

If integrated tests are a scam, why do we start outside in TDD with an end-to-end test?

On Tuesday, September 8, 2015 at 6:41:55 PM UTC+3, Nat wrote:

On 8 September 2015 at 14:26, Suhas Walanjoo <suhas.w...@gmail.com> wrote:

Nat,

Could you please provide links to the talks you recommended?

There are a bunch of videos online.  Not sure which is best.

https://www.google.co.uk/search?q=integrated%20tests%20are%20a%20scam

 

On Sep 1, 2015 5:46 PM, "Nat Pryce" <nat....@gmail.com> wrote:

Also JB's talk, Integrated Tests are a Scam, is v good overview on how to avoid end-to-end tests altogether.

On Tuesday, 1 September 2015, Steve Freeman <st...@m3p.co.uk> wrote:

I would suggest not getting too hung up about completeness if you’re completely new to the technique.

Start with the bits you can understand and work from there - but /keep pushing/. Unless you’ve gone too far, you can’t know where the boundaries really are.

For videos, I quite like JB Rainsberger’s new series.

S

--

---
You received this message because you are subscribed to the Google Groups "Growing Object-Oriented Software" group.

To unsubscribe from this group and stop receiving emails from it, send an email to growing-object-oriented-software+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
http://www.natpryce.com

--

---
You received this message because you are subscribed to the Google Groups "Growing Object-Oriented Software" group.

To unsubscribe from this group and stop receiving emails from it, send an email to growing-object-oriented-software+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "Growing Object-Oriented Software" group.

To unsubscribe from this group and stop receiving emails from it, send an email to growing-object-oriented-software+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

http://www.natpryce.com

[Steve Freeman]

That's JB taking a provocative position :)

S

> On 9 Sep 2015, at 15:35, Serkan Camurcuoglu <scamur...@gmail.com> wrote:
> If integrated tests are a scam, why do we start outside in TDD with an end-to-end test?
>
>
>
> On Tuesday, September 8, 2015 at 6:41:55 PM UTC+3, Nat wrote:
> On 8 September 2015 at 14:26, Suhas Walanjoo <suhas.w...@gmail.com> wrote:
> Nat,
>
> Could you please provide links to the talks you recommended?
>
> There are a bunch of videos online. Not sure which is best.
>
> https://www.google.co.uk/search?q=integrated%20tests%20are%20a%20scam
>
>
> On Sep 1, 2015 5:46 PM, "Nat Pryce" <nat....@gmail.com> wrote:
> Also JB's talk, Integrated Tests are a Scam, is v good overview on how to avoid end-to-end tests altogether.
>
> On Tuesday, 1 September 2015, Steve Freeman <st...@m3p.co.uk> wrote:
> I would suggest not getting too hung up about completeness if you’re completely new to the technique.
>
> Start with the bits you can understand and work from there - but /keep pushing/. Unless you’ve gone too far, you can’t know where the boundaries really are.
>
> For videos, I quite like JB Rainsberger’s new series.
>
> S
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups "Growing Object-Oriented Software" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to growing-object-oriente...@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> http://www.natpryce.com
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups "Growing Object-Oriented Software" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to growing-object-oriente...@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups "Growing Object-Oriented Software" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to growing-object-oriente...@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> http://www.natpryce.com
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups "Growing Object-Oriented Software" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to growing-object-oriente...@googlegroups.com.

[Raoul Duke]

integrated != integration, and coupling != cohesion.

[Nat Pryce]

On 9 September 2015 at 15:35, Serkan Camurcuoglu <scamur...@gmail.com> wrote:

If integrated tests are a scam, why do we start outside in TDD with an end-to-end test?

Good question. This is something we did not call out strongly enough in GOOS.

I start with an e2e test early in a project. Early on...

(a) the system is small enough for the test(s) to run fast

(b) I want to get my walking skeleton up and running, so care about integration with the system's external dependencies, runtime environment, deployment process etc.

(c) the system is so small it doesn't need to be divided into different modules

(d) [most importantly for me] I don't know enough yet to decide how design the internal structure of the system 

As the system grows, an internal structure emerges through refactoring. In my experience (and as we described in the worked example in GOOS), TDD guides the structure that emerges so that the application logic is decoupled from the infrastructure it runs on.

It is then possible to write acceptance tests against the domain model in memory rather than end-to-end. From then on, starting with an acceptance test does not mean starting with an end-to-end test.

--Nat

--

http://www.natpryce.com

--
http://www.natpryce.com

[Josue Barbosa dos Santos]

>>It is then possible to write acceptance tests against the domain model 

>>in memory rather than end-to-end. From then on, starting with an 

>>acceptance test does not mean starting with an end-to-end test.

You do this very much or are you just saying that it is possible?

Personally, I always start with an acceptance end-to-end test. I have also acceptance tests against the domain model in memory. But they are created after for fast feedback. 

[Nat Pryce]

On 18 September 2015 at 12:27, Josue Barbosa dos Santos <josue...@gmail.com> wrote:

>>It is then possible to write acceptance tests against the domain model 

>>in memory rather than end-to-end. From then on, starting with an 

>>acceptance test does not mean starting with an end-to-end test.

You do this very much or are you just saying that it is possible?

I do this.

I've occasionally worked on systems where it's not practical.

On one system, functionality was spread between a large amount (10+MLOC) of C code provided by a vendor and a smaller amount (800KLOC) of Java code. Commercial contracts and poor code quality meant that we were stuck with the architecture, so we covered the code with system tests before refactoring what we could.  But new code was written to be tested with in-memory fakes.

On another system, we replaced 30+KLOCs of DDD-style domain model, services, ORM, repositories etc. with 10s of lines of JMS message listeners and some routing rules configured in message brokers.  In that case, being able to run acceptance tests against the entire system gave us the option to radically change the implementation, cutting the amount of code by three orders of magnitude (and coincidentally increasing performance by roughly the same degree).

But these situations are rare.  Usually, being able to run the significant logic purely in memory is a big win.

The ideal situation is to decouple the definition of the acceptance tests from their implementation, so that the same tests can be run against in-memory logic or against a deployed system.  This is a nice aspect of spec-by-example tools like FIT or Cucumber.

--

http://www.natpryce.com

[Josue Barbosa dos Santos]

Thanks Nat.

[Matt Wynne]

> On 19 Sep 2015, at 19:37, Nat Pryce <nat....@gmail.com> wrote:
>
> The ideal situation is to decouple the definition of the acceptance tests from their implementation, so that the same tests can be run against in-memory logic or against a deployed system. This is a nice aspect of spec-by-example tools like FIT or Cucumber.

+1

cheers,
Matt

---
ma...@mattwynne.net

[Colin Vipurs]

+1 on this, but you don't need spec tools for this if you're disciplined.  Writing your acceptance tests as "specs" and making use of a DSL for a layer of abstraction can yield the same results.

--

---
You received this message because you are subscribed to the Google Groups "Growing Object-Oriented Software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to growing-object-oriente...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Maybe she awoke to see the roommate's boyfriend swinging from the chandelier wearing a boar's head.

Something which you, I, and everyone else would call "Tuesday", of course.