Cucumber and testing through the UI

[Rafael George]

Hi guys, 

I was wondering something regarding hexagonal rails, I'm trying to write my feature files to be an acceptance test without hitting the UI just because I think I can see the UI with my eyes and I know what is happening in that moment. I'm trying to hit the service layer from cucumber steps and from there I'll try to unit test that particular service. 

How are you doing this process? I'm starting to think that tools like capybara are a mistake.

[Dave Newton]

I can see the UI with my eyes too.

As long as I have an infinite amount of time to review the entire app's UI every time I make a change.

Which I don't.

Tests are persistent; people aren't.

--
You received this message because you are subscribed to the Google Groups "Objects on Rails" group.
To unsubscribe from this group and stop receiving emails from it, send an email to objects-on-rai...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

e: davel...@gmail.com

m: 908-380-8699

s: davelnewton_skype

t: @dave_newton

b: Bucky Bits

g: davelnewton

so: Dave Newton

[Rob Park]

On Tue, Oct 8, 2013 at 9:46 AM, Rafael George <george...@gmail.com> wrote:

If you're familiar with the "test pyramid" (http://martinfowler.com/bliki/TestPyramid.html), you're only testing 2 of the levels. It's great you're testing at those levels, but you should consider having some tests that actually hit the UI IMO.

What would be a big mistake (that is all too common) would be to invert the pyramid and create an "ice cream cone" (http://watirmelon.com/2012/01/31/introducing-the-software-testing-ice-cream-cone/).

@robpark

 

[Rafael George]

But in that case isn't Fowler arguing on having a tests suite using Selenium? I think that doing Cucumber features from within a Rails application should be hitting the service layer directly and maybe creating another smaller test suite with selenium for UI stuff as Fowler suggest. Why do we have to do that from our Rails application? Which are the advantages of that?

[Matt Wynne]

Here are my thoughts on this, and believe me I've thought about this a lot.

The idea of a hexagonal architecture is to decouple your domain logic from the details of how it manifests in the real world. The reason this idea came about is that Alistair Cockburn wanted to be able to test domain logic without having to work out how to drive the awkward win32 GUIs he was working with back in the day. But it turned out to also help you separate your code really nicely. Another example of the deep synergy between testability and good design[1].

In another universe, we have people writing Cucumber tests against Rails apps. Many of those people were sent down a blind alley[2] by the original Cucumber-Rails web_steps.rb and despite us removing that code and shouting from the rooftops[3] about the problems with writing scenarios in that style, continue to fuck up their feature suites[4].

Getting the level of abstraction right in your cukes is a really interesting challenge, I find. The more you manage to make them describe the *what*, rather than the *how* of the behaviour you want, the more robust they are to the commonplace tweaks and changes that happen to every UI over time. Also, the more readable they are when you come back to them weeks or months after you wrote them.

There's another benefit of writing more declarative, abstract scenarios. The same scenario can be run against the core domain of your app, or against the full stack. Because the language isn't coupled to the UI, it could be applicable to either level. For Relish's billing system, for example, we run the same set of features[5] against the billing sub-system in isolation, and again against the full Rails UI. This allows me to quickly test the scenarios against the code domain, but also have confidence the thing wires up together.

I covered this idea in part 2 of The Cucumber Book, and also in this live coding session[6].

I'm not saying this is applicable in every case, but the challenge of thinking about "If I ran this scenario against my domain model, would it still make sense?" is a good one to help you get the level of abstraction right in your cukes. Tools like Capybara are not a mistake, but using them too liberally can be, in my experience. End-to-end tests are slow, and as Rob already pointed out in this thread, having too many end-to-end tests and not enough unit tests (the testing "ice cream cone") is definitely something you want to avoid.

[1] http://www.youtube.com/watch?v=4cVZvoFGJTU

[2] http://watirmelon.com/2010/12/10/cucumber-imperative-or-declarative-that-is-the-question/

[3] http://aslakhellesoy.com/post/11055981222/the-training-wheels-came-off

[4] http://vimeo.com/39660655

[5] https://www.relishapp.com/relish/relish/docs/billing

[6] http://skillsmatter.com/podcast/agile-scrum/bdd-as-its-meant-to-be-done

--
You received this message because you are subscribed to the Google Groups "Objects on Rails" group.
To unsubscribe from this group and stop receiving emails from it, send an email to objects-on-rai...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

cheers,

Matt

--

tel. +44 (0)20 8720 6842

http://mattwynne.net || https://twitter.com/mattwynne || http://the-cucumber-book.com || http://bddkickstart.com || http://www.relishapp.com

[Rafael George]

Thanks Matt and the others for all this info. I think I said that about capybara because sometimes is kinda frustrated for me to do things the Rails way. I will check all this info and try to think about them and reflect; also I'm thinking in just building a gem without any Rails connection and then put that gem which will have all the domain logic into Rails. 

Maybe that's the way to go for an entire hexagonal Rails app. 

[Matt Wynne]

On 9 Oct 2013, at 14:13, Rafael George <george...@gmail.com> wrote:

Thanks Matt and the others for all this info. I think I said that about capybara because sometimes is kinda frustrated for me to do things the Rails way. I will check all this info and try to think about them and reflect; also I'm thinking in just building a gem without any Rails connection and then put that gem which will have all the domain logic into Rails. 

Maybe that's the way to go for an entire hexagonal Rails app. 

I think so.

Tooky and I have had a couple of aborted attempts at doing some screencasts about this. I haven't had the opportunity to build a new Rails app for a while, but if I did that's the way I'd approach it - write some cukes to describe the behaviour, run them against a separate gem to build a domain model that satisfies them, then plug the gem into a Rails app.

On Tuesday, October 8, 2013 9:46:13 AM UTC-4, Rafael George wrote:

Hi guys, 

I was wondering something regarding hexagonal rails, I'm trying to write my feature files to be an acceptance test without hitting the UI just because I think I can see the UI with my eyes and I know what is happening in that moment. I'm trying to hit the service layer from cucumber steps and from there I'll try to unit test that particular service. 

How are you doing this process? I'm starting to think that tools like capybara are a mistake.

--
You received this message because you are subscribed to the Google Groups "Objects on Rails" group.
To unsubscribe from this group and stop receiving emails from it, send an email to objects-on-rai...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Charles Hoffman]

I am so confused on this feature testing stuff. Here's some things that have been bothering me for a while.

Are we testing [through] the UI or not? It feels like I hear features described as testing from the user point of view through the UI in one breath, and in the next breath the same person will say things like:

> The more you manage to make them describe the *what*, rather than the *how* of the behaviour you want, the more robust they are to the commonplace tweaks and changes that happen to every UI over time.

...

> Because the language isn't coupled to the UI, it could be applicable to either level.

It almost seems like we're saying, we want to test through the UI, without ever saying anything about that UI. I can't be the only one out here thinking this doesn't make any sense at all, right?

The concern, driving the impulse against coupling to the UI, seems to be that the UI gets fiddled with by designers and front-end devs and this breaks our tests needlessly. I think this is a valid concern but are there better ways of addressing it?

I've never understood why it is so common (speaking of web_steps and things like it) to reference form fields by their label, which seems to me to be the least stable thing about it. Using its id might be more stable, but is probably less readable, and it still might get changed in the process of CSS or javascript work. The most stable thing would be its name, assuming that front-end devs are smart enough to leave it alone because it has to match up with attributes on the model and probably table and column names in the database which they aren't going to go to the trouble to mess with. Unfortunately, those are the least readable: "user[login]" and so on.

What about some strategies like:
*Just as you "don't mock what you don't own", don't test what you don't own. Or, conversely, you don't get to fiddle with things without having to keep up or at least think about any relevant tests: maybe leave UI testing to the UI devs.
* educate front-end developers on what parts of the html code other parts of the system rely on. Analogy: if I signed on to your team in a DBA or db-dev role, and I started changing column and table names to names I thought were better, and a bunch of your model code broke, and when confronted on this I just reply with "well that's not my job dude, I'm the database guy" what would you tell me?

Another thing that gets to me is, I see a lot of this: there's a feature for logging in, and then, every other feature where the user operation requires them to be logged in to do it, accomplishes this by first running through the steps of that logging-in feature. This seems like it could be one of the big things causing all this consternation over slow feature suites: it's logging in dozens of freaking times. If you have one feature for the login, can't other features be set up to just start from an already logged-in state? That state could be established at the start of the feature in a simpler, quicker way like how it's done in controller specs or something. "Given I Am Logged In" should not have to piss around with "I Fill in 'b...@example.com' In 'Login' / "I Fill in 'password' in Password"/ "I Click The Damn Button" every single time. This could also hold for things like "I Am On The Such-And-Such Page": instead of visiting or navigating to that page, just already be there. You probably already have feature that make sure the right paths/links _get_ to that page, why do it a dozen more times just so you can test what a user can do _on_ that page?

All of this might actually just be compatible with exactly the principles you're setting out here for good feature writing, so maybe I'm just drilling down to practical specifics, techniques that would make for better features. Probably the same things I'm bothered by above, bother you too.

Also, I'd love to see some examples of these principles for good features done in the form of rspec-rails features, as an alternative to Cucumber. Or really, with as many different testing tools as possible.

--
[chuck hoffman]
[sounds, words, and code]
[what else is there?]
[http://hoff2.com]