Clean architecture / never test the GUI ?

[Michael Azerhad]

Hello Uncle Bob,

In one of your conference, you explained the benefit to DEFER decisions about technical details when building a software (web interfaces, database etc...): 

I don't know if you ever read this book: http://www.amazon.com/Growing-Object-Oriented-Software-Guided-Tests/dp/0321503627  but it seems to promote exactly the contrary. One of its idea is:

_ Start by writing a Walking Skeleton with an end-to-end focus.  "We've seen too much projects failing because integration of components was not define earlier in the project phase".

Thus, in this case, an architect would firstly focus on: What do we think TECHNICALLY we need to achieve our goal.. "hummm  Swing, ok ..  MYSQL humm okkk that seems good". The very opposite way of yours :)

With this Walking Skeleton working, each feature would then be started by writing an acceptance test (using BDD why not), oriented end-to-end at the beginning to. (Selenium for Web, WindowLicker for Swing) etc...

I'm too confused about the way I have to choose.

On one hand, I do agree with you:  testing plain old object is far more fast, than always doing something like: "go to login page, ok, log in...." "go to login page, ok, log in" with Selenium, this one already slow.

In your conference, it takes only 1 minute and 40 seconds to test ALL your amount of use cases.

On other hand, I don't really figure out why you can thus say: "Ok use cases tests are all GREENN, yeahhh, ship the whole !"  .... You don't want to check for the GUI? Running test on the GUI? Check for Database integration?

If there is a huge problem is the GUI part (forget to call a use-case, wrong mapping between controllers and URL or some other failures..), how will you be able to see it?  QA ? But you say yourself:  "QA is dead !"  :)

I've just started a (future) huge project, and I would like to take a good way of testing, that's why I ask this question about testing :)

By the way, thanks a lot for all stuff you're doing, I'm a big fan :)

Michael  

[David Hunt]

+1 I've been wondering about that myself.

Usually the regressions I get are  related to event wire-up, URL routing, app config settings, incorrect SQL etc. None of which are caught by unit tests.

I don't do fully end-to-end automated tests because the effect on productivity is appalling, but I certainly can't rely on the green bar either. I have to do some manual testing before a release.

Having said that, the manual testing is easy. If one of the above areas has a bug, it's usually very easy to spot. Except for SQL bugs.

Dave

--
The only way to go fast is to go well.
---
You received this message because you are subscribed to the Google Groups "Clean Code Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clean-code-discu...@googlegroups.com.
To post to this group, send email to clean-code...@googlegroups.com.
Visit this group at http://groups.google.com/group/clean-code-discussion.
 
 

[Sebastian Gozin]

I don't think Uncle Bob and the GOOSGT book are at odds here. It's not because you picked mysql for your database right now that you already committed to mysql until the ends of time. You can design your code so it does not care about the persistence strategy used and then you quickly slap up a mysql implementation to demo whatever you have right now. Or perhaps mysql is too involved and you just keep things in memory, or in files.

I somewhat dislike how recently people in the BDD world suggest that you shouldn't be spending so much time on automating everything. Which I think is related to them involve the web and the database too much in their Cucumber specs.
From personal experience I've noticed that if your acceptance tests tell you the business logic is correct then fixing databases and gui's does not take a lot of time once the problem has been discovered. The opposite however tends to grind my progress to a halt.

[Michael Azerhad]

Hi Uncle Bob :)

Thanks for the time taken to answer, it's really worthwhile and make things clear.

Looking forward to watch the next episode :)

Michael

On Wednesday, June 19, 2013 3:09:36 PM UTC+2, Uncle Bob wrote:

The difference between my approach and the GOOS book is a common issue.  First, there _is_ a pretty big difference.  They start at the periphery of the system and work in.  I start in the center of the system and work out.  Oddly, that's about where the difference ends.  The "hexagonal architecture" that they promote is remarkably similar to the "Clean Architecture" that I promote.  

Why do these two techniques result in the same architecture?  Because our goals and emphases are the same, and we're both firing tracer bullets.  Both techniques are agile, both techniques recommend developing in tiny iterations, and both techniques push you end to end within the confines of an iteration.  Both wind up with acceptance tests running through the GUI at the end of the iteration.  And both want those acceptance tests written at the start of the iteration.  

Somehow people have gotten the idea that I don't think you should have tests at the GUI.  This is probably because I often stridently assert that you should not test business rules through the GUI, and people infer that to mean _no_ tests of the GUI.  OF COURSE you will test the GUI.  I just don't want you testing business rules through the GUI.  When write tests at the GUI, I want them to be GUI tests.  And yes, this means that the technique of writing BDD style cucumber tests through the GUI bothers me deeply.  I think the Rails community has gotten way off course with this approach.  

Whether you follow my approach, or the GOOS approach, you will still isolate the GUI and the Database from the business rules.  You will still create an architecture where the GUI and Database are plugins.  The only difference is whether, in the course of an iteration, you are moving from the outside in, or the inside out.  

While that difference in direction creates almost no difference in the structure of the application, it creates a _profound_ difference in the structure of the tests.  Inside out tests have a _lot_ fewer mocks.  Outside in tests are _dominated_ by mocks.  When you code from the inside out your tests depend on return values and state.  When you code from the outside in, your tests depend more on mocks that spy on the way your system work.  Inside out tests depend on results.  Outside in tests depend on mechanism and algorithm.  

This last issue will be the topic of one of the upcoming episodes in the Advanced TDD thread.

[Philip Schwarz]

>This last issue will be the topic of one of the upcoming episodes in the Advanced TDD thread.

look forward to that

Philip

[Seb Rose]

1) Walking skeleton predates GOOS. Cockburn wrote about it last century http://alistair.cockburn.us/Walking+skeleton.

2) When people speak about deferring decisions they're usually concerned with avoiding premature commitment. Cockburn specifically says that the Walking Skeleton "need not use the final architecture, but it should link together the main architectural components. The architecture and the functionality can then evolve in parallel."

3) In an old article (http://martinfowler.com/ieeeSoftware/whoNeedsArchitect.pdf), Martin Fowler says "one of an architect’s most important tasks is to remove architecture by finding ways to eliminate irreversibility in software designs." He's talking about making design decisions that protect us from making hard-to-change architectural commitments. Too much flexibility and the system becomes over-complex, too little and you're at the mercy of every change (whether due to your product owner or a changing 3rd party component)

4) BDD has never recommended driving development through the UI. BDD encourages us to focus on the behaviour of the system. Exercise the system at the layer appropriate for the behaviour that you are developing. It's worth keeping the Testing Pyramid in mind, and probably the Testing Iceberg too (http://claysnow.co.uk/?p=175315341). And always remember that the GUI itself is just another component.

5) I don't agree with Uncle Bob that outside-in and inside-out approaches result in profoundly different test structures. Specifically I have seen no evidence that outside-in tests necessarily lead to more mocks, or to mocks/doubles that rely on mechanism rather than results.

6) More contentiously, I think that we're all working outside-in anyway. TDD by it's very definition works from the outside of a component, driving its design and implementation through an emerging public API. Further, if there's no clear link from your user's requirements through to the code you're implementing something is going very wrong, so we're all working from the outside after all.

[Uncle Bob]

On Sunday, June 23, 2013 3:17:52 PM UTC-5, Seb Rose wrote:

1) Walking skeleton predates GOOS. Cockburn wrote about it last century http://alistair.cockburn.us/Walking+skeleton.

Yes, it's been called many names.  Tracer bullet.  Spike.  etc.  The idea is core to the agile way of working.  Get something simple working and then evolve it.  We can go all the way back to John Gall's "Systemantics" book for the justification:

 Gall's Law:

• A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system.

2) When people speak about deferring decisions they're usually concerned with avoiding premature commitment. Cockburn specifically says that the Walking Skeleton "need not use the final architecture, but it should link together the main architectural components. The architecture and the functionality can then evolve in parallel."

Correct.  You can still build a valid walking skeleton from the outside in so long as you are not committing to particular GUI and DB frameworks (among others).   This requires a very strict discipline of isolation and decoupling.  That decoupling often drives a mockist approach.

3) In an old article (http://martinfowler.com/ieeeSoftware/whoNeedsArchitect.pdf), Martin Fowler says "one of an architect’s most important tasks is to remove architecture by finding ways to eliminate irreversibility in software designs." He's talking about making design decisions that protect us from making hard-to-change architectural commitments. Too much flexibility and the system becomes over-complex, too little and you're at the mercy of every change (whether due to your product owner or a changing 3rd party component)

Roger that.  The role of an architect is to keep options open, not to close them.
 

4) BDD has never recommended driving development through the UI. BDD encourages us to focus on the behaviour of the system. Exercise the system at the layer appropriate for the behaviour that you are developing. It's worth keeping the Testing Pyramid in mind, and probably the Testing Iceberg too (http://claysnow.co.uk/?p=175315341). And always remember that the GUI itself is just another component.

While this is true, it is all too often ignored -- especially in the rails community.  The standard mechanism in that community is to write cucumber/capybara tests that bind the described behavior to the web system in a tightly coupled manner.  This is unfortunate.

5) I don't agree with Uncle Bob that outside-in and inside-out approaches result in profoundly different test structures. Specifically I have seen no evidence that outside-in tests necessarily lead to more mocks, or to mocks/doubles that rely on mechanism rather than results.

That's odd, since that is more or less the basis of the GOOS book.  Start at the edges and mock your way into the middle.  I prefer to start in the middle, with the business rules, and step my way to the edges.  I use mocks, but usually only at the boundaries between decoupled layers.  I find, when I work that way, that I don't need a mocking framework, because my mocks tend to be very simple stubs or spies.  I don't need elaborate expectation mechanisms.

I'm not trying to say which course is better.  Indeed, I'm relatively certain that Steve and Nat have put together more systems than I in recent years.  I'm just stating my observations and preferences.

6) More contentiously, I think that we're all working outside-in anyway. TDD by it's very definition works from the outside of a component, driving its design and implementation through an emerging public API. Further, if there's no clear link from your user's requirements through to the code you're implementing something is going very wrong, so we're all working from the outside after all.

Yes and no.  It's true that we have to start with requirements somehow; and that those requirements are "outside" the deepest innermost functions of the system.  However, it is entirely possible to state requirements in such a way that they are UI and DB agnostic; allowing use to start at the outside of the inner circle.  Of course if the truth is told, I don't start there either.  I start with the use cases themselves and work inwards towards the business objects, and outwards towards the UI and DB.   

[Philip Schwarz]

FYI - readers of this thread may also be interested in the following thread I started in the GOOS group: Robert Martin compares GOOS outside in approach with his inside out approach 

Philip

On Wednesday, 19 June 2013 14:09:36 UTC+1, Uncle Bob wrote:

The difference between my approach and the GOOS book is a common issue.  First, there _is_ a pretty big difference.  They start at the periphery of the system and work in.  I start in the center of the system and work out.  Oddly, that's about where the difference ends.  The "hexagonal architecture" that they promote is remarkably similar to the "Clean Architecture" that I promote.  

Why do these two techniques result in the same architecture?  Because our goals and emphases are the same, and we're both firing tracer bullets.  Both techniques are agile, both techniques recommend developing in tiny iterations, and both techniques push you end to end within the confines of an iteration.  Both wind up with acceptance tests running through the GUI at the end of the iteration.  And both want those acceptance tests written at the start of the iteration.  

Somehow people have gotten the idea that I don't think you should have tests at the GUI.  This is probably because I often stridently assert that you should not test business rules through the GUI, and people infer that to mean _no_ tests of the GUI.  OF COURSE you will test the GUI.  I just don't want you testing business rules through the GUI.  When write tests at the GUI, I want them to be GUI tests.  And yes, this means that the technique of writing BDD style cucumber tests through the GUI bothers me deeply.  I think the Rails community has gotten way off course with this approach.  

Whether you follow my approach, or the GOOS approach, you will still isolate the GUI and the Database from the business rules.  You will still create an architecture where the GUI and Database are plugins.  The only difference is whether, in the course of an iteration, you are moving from the outside in, or the inside out.  

While that difference in direction creates almost no difference in the structure of the application, it creates a _profound_ difference in the structure of the tests.  Inside out tests have a _lot_ fewer mocks.  Outside in tests are _dominated_ by mocks.  When you code from the inside out your tests depend on return values and state.  When you code from the outside in, your tests depend more on mocks that spy on the way your system work.  Inside out tests depend on results.  Outside in tests depend on mechanism and algorithm.  

[Uncle Bob]