Dependent Scenarios

[Steve Klabnik]

Hello fellow Cukers!

I have a bit of a quandary on my hands, hoping you call can help. I'm in the process of advocating that we switch to Cucumber internally to test our API[1]. I think this makes a lot of sense, as our product is an API, so the language-neutral aspect of Cucumber is really appealing. And I'm really happy with the few scenarios I've worked out so far. But I have a problem....

Our API relies on hypermedia. This means that we need to often make scenarios that are dependent on previous ones. Basically, GivenScenario from long, long ago is exactly what I want.

For example, I may have a scenario around creating a new credit card in our system. I then want to make a scenario that creates a card, and then uses the URL in the response to create a second request to charge that card. I'd want a third scenario that creates a card, then uses the URL in the response to create a second request to change the address on the card. I'd want a third scenario which creates a card...

Now: I _could_ make the dependencies in Ruby, and call them from previous ones. But then the Ruby version may get out of synch with the scenario version. I _could_ make it so that the common steps are in Background, but then I'd be organizing according to Backgrounds and not use cases, which is what I'd prefer. I could just duplicate everything in every case, but that seems bad too, if something changes, it'd have to change everywhere...

What should I do here? I think API testing is a really interesting use case for Cucumber, and I'm specifically trying to build better tooling for the hypermedia case. I totally get why 'normally' this would be bad.

Thoughts? Thank you for your time.

- Steve

1: https://github.com/balanced/balanced-api/pull/431

[Tim Walker]

[tim] Inline below

On Tue, Dec 10, 2013 at 11:08 AM, Steve Klabnik <st...@steveklabnik.com> wrote:

Hello fellow Cukers!

I have a bit of a quandary on my hands, hoping you call can help. I'm in the process of advocating that we switch to Cucumber internally to test our API[1]. I think this makes a lot of sense, as our product is an API, so the language-neutral aspect of Cucumber is really appealing. And I'm really happy with the few scenarios I've worked out so far. But I have a problem....

Our API relies on hypermedia. This means that we need to often make scenarios that are dependent on previous ones. Basically, GivenScenario from long, long ago is exactly what I want.

For example, I may have a scenario around creating a new credit card in our system. I then want to make a scenario that creates a card, and then uses the URL in the response to create a second request to charge that card. I'd want a third scenario that creates a card, then uses the URL in the response to create a second request to change the address on the card. I'd want a third scenario which creates a card...

[tim] Seems like you could either make these steps in the scenario you're testing, or use a factory to put the data in an expected state for each additional scenario. "Given a credit card", "Then I make a charge" and "Given a credit card", "Then I change the address", etc. 

 

Now: I _could_ make the dependencies in Ruby, and call them from previous ones. But then the Ruby version may get out of synch with the scenario version. I _could_ make it so that the common steps are in Background, but then I'd be organizing according to Backgrounds and not use cases, which is what I'd prefer. I could just duplicate everything in every case, but that seems bad too, if something changes, it'd have to change everywhere...

What should I do here? I think API testing is a really interesting use case for Cucumber, and I'm specifically trying to build better tooling for the hypermedia case. I totally get why 'normally' this would be bad.

Thoughts? Thank you for your time.

- Steve

1: https://github.com/balanced/balanced-api/pull/431

--
-- Rules --
 
1) Please prefix the subject with [Ruby], [JVM] or [JS].
2) Please use interleaved answers http://en.wikipedia.org/wiki/Posting_style#Interleaved_style
3) If you have a question, don't reply to an existing message. Start a new topic instead.
 
You received this message because you are subscribed to the Google Groups Cukes group. To post to this group, send email to cu...@googlegroups.com. To unsubscribe from this group, send email to cukes+un...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/cukes?hl=en
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Miguel Almeida]

Hi Steve,

On Tuesday, December 10, 2013 6:08:48 PM UTC, Steve Klabnik wrote:

Hello fellow Cukers!

I have a bit of a quandary on my hands, hoping you call can help. I'm in the process of advocating that we switch to Cucumber internally to test our API[1]. I think this makes a lot of sense, as our product is an API, so the language-neutral aspect of Cucumber is really appealing. And I'm really happy with the few scenarios I've worked out so far. But I have a problem....

Our API relies on hypermedia. This means that we need to often make scenarios that are dependent on previous ones. Basically, GivenScenario from long, long ago is exactly what I want.

For example, I may have a scenario around creating a new credit card in our system. I then want to make a scenario that creates a card, and then uses the URL in the response to create a second request to charge that card. I'd want a third scenario that creates a card, then uses the URL in the response to create a second request to change the address on the card. I'd want a third scenario which creates a card...

I think create dependencies to other scenarios might lead to maintenance problems in the future. In your example " I then want to make a scenario that creates a card", why don't you:
Given an existing card

Internally, you can create that card and use the URL for your next stepdefs. You can either simplify creation (e.g., creating it in the db directly) or call the other step defs.


Miguel

 

[Steve Klabnik]

Hey Halfordian, Miguel,

Yes! You have exposed one bit of detail that I forgot to mention, which is important: This is 100% acceptance testing. We will actually be running these scenarios regularly against production with test credentials, as well as against every commit (locally) in CI. That means "create this card in the db" is straight out, as not only is our application written in Python, not Ruby, but the tests won't even be running on the same host! Not to mention that "create a card" as a step would end up being more unit style tests than an actual acceptance test. We have great internal unit coverage of scenarios like that.

- Steve

[Seb Rose]

I generally think of my Features/Scenarios from 2 distinct viewpoints:

1) System documentation - each scenario drives out and validates a single behaviour

2) Smoke tests -  'workflow' scenarios that exercise key paths through the application to ensure (for example) that it has deployed successfully and that performance is holding up

From a system documentation point of view it wouldn't make sense to have one section of system documentation depend on a previous section, and I don't think it makes sense in Cucumber/Gherkin either. So, phrase the Given step in such a way that it conveys the salient points of the desired system context in domain language (even though it may leave some details implicit). Now you can keep each scenario concise (not boring ;). For example:

Given I have an active credit card associated with my account

would create an account, create a card and associate the two. Push the details of how the account and card are created and associated down into the support code.

The support code may well create the domain entities through the API. In some environments, however, it may be possible to take shortcuts and create the entities directly in the database. Or in some other context-specific, 'clever' way. The details of *how* the system gets to the state you need are abstracted out. That leaves you (and other readers of your scenario) free to focus on the behaviour that is being validated.

From a smoke test point of view, we always want to run against a fully deployed system, through the public API. Since we have already validated component behaviour and interactions, this can be a small set of horizontally end-2-end workflows (the top of the Testing Pyramid).

The real issue is when you say "we will actually be running these scenarios regularly against production with test credentials, as well as against every commit (locally) in CI." This is trying to satisfy two very different requirements with a single solution.

With my clients, I use Cucumber tags to control which scenarios run as smoke tests and which as integration tests (i.e. which run fully end-2-end and which may take short cuts setting up the system state). As the runtime of the smoke test (and confidence in the application) increases I change the tags of some scenarios so that they only run in the integration test suite. I blogged about this a while ago: http://claysnow.co.uk/living-documentation-can-be-readable-and-fast/

Hope this makes sense

Seb

 

- Steve

1: https://github.com/balanced/balanced-api/pull/431

--

-- Rules --
 
1) Please prefix the subject with [Ruby], [JVM] or [JS].
2) Please use interleaved answers http://en.wikipedia.org/wiki/Posting_style#Interleaved_style
3) If you have a question, don't reply to an existing message. Start a new topic instead.
 
You received this message because you are subscribed to the Google Groups Cukes group. To post to this group, send email to cu...@googlegroups.com. To unsubscribe from this group, send email to cukes+un...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/cukes?hl=en
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
ACCU - Professionalism in Programming - http://accu.org

http://www.claysnow.co.uk
http://twitter.com/#!/sebrose
http://uk.linkedin.com/in/sebrose

[Steve Klabnik]

Hey Seb!

Yes, "workflows" might be a better way of phrasing what I'm trying to do.

Your "given" makes sense, but how do I make sure that it doesn't get out of step with the Scenario that represents the same given? Do you just assume that there's duplication and make sure to change them both?

- Steve

[and...@andrewsardone.com]

On Tuesday, December 10, 2013 2:34:20 PM UTC-5, Seb Rose wrote:

Given I have an active credit card associated with my account

would create an account, create a card and associate the two. Push the details of how the account and card are created and associated down into the support code.

The support code may well create the domain entities through the API. In some environments, however, it may be possible to take shortcuts and create the entities directly in the database.

I like this setup, Seb – it's one we've been using for our API testing via Cucumber. For one API test suite, we simply have a fixtures-like file[1] that's loaded within a Given-step[2] via some support code. For shared setup we've resorted to Backgrounds, but it has served us well. We can parameterize the Given-step to match on certain fixture files that need to be loaded.

[1]: http://cl.ly/StzP

[2]: http://cl.ly/StiQ

[Seb Rose]

Hi Steve,

On 10 December 2013 19:43, Steve Klabnik <st...@steveklabnik.com> wrote:

Yes, "workflows" might be a better way of phrasing what I'm trying to do.

Your "given" makes sense, but how do I make sure that it doesn't get out of step with the Scenario that represents the same given? Do you just assume that there's duplication and make sure to change them both?

I'd push the detailed implementation down to the support code:

Given(/^I have an account$/) do

  AccountHelper.registerForAccount

end

Given(/^I have an active credit card$/) do

  CreditCardHelper.issueCreditCard

end

When(/^I associate an active credit card with my account$/) do

  AccountHelper.associateCreditCard(CreditCardHelper.myCreditCard)

end

Given(/^I have an active credit card associated with my account$/) do

  AccountHelper.registerForAccount

  CreditCardHelper.issueCreditCard

  AccountHelper.associateCreditCard(CreditCardHelper.myCreditCard)

  # ... and I might even push these 3 lines into a builder: ActiveAccountBuilder.buildCreditCardAccount

end

If the process of associating a credit card with an account changes (say by adding an extra step) then if you forget to modify the /^I have an active credit card associated with my account$/ step accordingly I would expect the AccountHelper.associateCreditCard call to fail. (And I'd expect there to be a detailed scenario that ensures that AccountHelper.associateCreditCard does fail if all the necessary steps have not been fulfilled).

Maybe I've misunderstood your concern. What are you worried might happen? Can you give me a concrete example?

Cheers

Seb

[Steve Klabnik]

Yes, so

> Given(/^I have an active credit card associated with my account$/) do

This is already a scenario in its own right. How do you prevent that
scenario from drifting away from this step?

[Seb Rose]

And that scenario would be something like:

  Given I have an account

  And I have an active credit card

  When I associate an active credit card with my account

  Then my account is active_or_something

Say we add an extra validation step:

  Given I have an account

  And I have an active credit card

  And I submit the correct number of cents debited from my card

  When I associate an active credit card with my account

  Then my account is active_or_something

But we'd also write new scenarios:

  Given I have an account

  And I have an active credit card

  And I submit the incorrect number of cents debited from my card

  When I associate an active credit card with my account

  Then my account is in_error

  Given I have an account

  And I have an active credit card

  And I don't submit the number of cents debited from my card

  When I associate an active credit card with my account

  Then my account is in_error

And now I'd expect any scenario with the step "Given I have an active credit card associated with my account" to fail too (because it's not following the new process).

I feel like I'm missing your point. Sorry.

[Steve Klabnik]

It's all good!

So, both of these steps:

Given I have an account
And I have an active credit card

Are also scenarios. "I have an account" requires sending a POST
request, and creating a credit card requires getting a URL from the
response to making that account, and then making a POST request there
as well. Now, implementing that is all good and fine, but what happens
when 'making an account' changes: Now I have to change the _scenario_
"Creating an account" as well as the step "Given I have an account."
They do the exact same things. Duplication leads to updating in one
place and not updating in another.

Does that clear things up?

[Seb Rose]

Yes, I think so.

Q. If "making an account" changes, and you don't change the implementation of "Given I have an account" won't it break/fail?

A. Yes: then things can't get out of step without you noticing

A. No: why not?

Are you concerned about:

1) the effort involved in keeping the scenario that validates the basic behaviours in sync with the support code that bundles up more complex interactions

OR

2) the possibility that errors/discrepancies might creep in.

I'm suggesting that 2) should not be possible when you are drive out behaviours using BDD/TDD et al. 

And since I don't worry about forgetting to keep things in sync (because the scenarios will fail & tell me when I forget), I also don't worry about the effort of making the extra change.

Contrast this with (the hugely beneficial) technique of writing collaboration & contract tests (documented by @jbrains), which relies on discipline alone to keep mocks and actual implementation in sync.

Cheers

Seb

[Steve Klabnik]

I am concerned about 1. My natural programmer instinct is 'duplication
is bad and will lead to bad things.'

Maybe I can live with it in this case.

[Matt Wynne]

I missed this post, and went and put my reply on the Github ticket instead:

https://github.com/balanced/balanced-api/pull/431#issuecomment-30284388

It looks like you've been having some great discussions here!

cheers,

Matt

[Steve Klabnik]

Yes, this list has been exceedingly helpful, as well as on the Issues thread.

I knew I was about to do this, which is one of the reasons that the
"Rails 4 in Action" thread bummed me out; it made me look very
anti-Cukes when I knew I was about to end up advocating directly for
it shortly... heh.

[George Dinwiddie]

Steve,

On 12/10/13 1:08 PM, Steve Klabnik wrote:
[snip

> I _could_ make it so that the common steps are in
> Background, but then I'd be organizing according to Backgrounds and not
> use cases, which is what I'd prefer.

Have you considered Tagged Hooks?
https://github.com/cucumber/cucumber/wiki/Hooks#tagged-hooks

- George

--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------

[Seb Rose]

On 10 December 2013 23:16, Steve Klabnik <st...@steveklabnik.com> wrote:

I am concerned about 1. My natural programmer instinct is 'duplication
is bad and will lead to bad things.'

I understand this ;)

 

Maybe I can live with it in this case.

I find it helpful to think of this as a form of 'triangulation' rather than 'duplication'. 

--
-- Rules --

1) Please prefix the subject with [Ruby], [JVM] or [JS].
2) Please use interleaved answers http://en.wikipedia.org/wiki/Posting_style#Interleaved_style
3) If you have a question, don't reply to an existing message. Start a new topic instead.

You received this message because you are subscribed to the Google Groups Cukes group. To post to this group, send email to cu...@googlegroups.com. To unsubscribe from this group, send email to cukes+un...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/cukes?hl=en
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[aslak hellesoy]

Steve, please reply in-line on this list. It makes conversations easier to follow. See list rules in the email footer.

On Tue, Dec 10, 2013 at 11:16 PM, Steve Klabnik <st...@steveklabnik.com> wrote:

I am concerned about 1. My natural programmer instinct is 'duplication
is bad and will lead to bad things.'

DRY[1] is an honorable principle, but it doesn't apply in all cases. More specifically, it has a lot more value when applied to production code than to tests.

When there is duplication in production code, and the duplicated code needs to change, you have a problem. You might not know about all the other places you need to change. If you're fixing a bug in duplicated code, it's common that you leave bugs lying around because it's cumbersome to find the duplicated code.

This doesn't apply to tests. Let's assume you have duplication in tests. You need to change the production code touched by these tests. You will actually break those tests, so you know what tests you need to change. Sure, there is a little extra work to do, but that's negligible if you have well-factored test code.

What we usually recommend is to make most stepdef bodies one-liners that just invoke a test library method:

Given /a credit card/ do

  create_credit_card!

end

Now, in your scenario that needs a lot of stuff to have happened in the Given, you just create a stepdef that delegates to a bunch of these methods:

Given /a ton of shit happened/ do

  user = create_user!

  create_credit_card!

  link_card_to_user!(user)

end

David Chelimsky once said "Clarity trumps DRYness in specs" [2]

[1] http://en.wikipedia.org/wiki/Don't_repeat_yourself

[2] http://butunclebob.com/ArticleS.DavidChelimsky.SpecOrganization

Aslak

 

Maybe I can live with it in this case.

[Matt Wynne]

Good, glad we could help.

Since you mention it, what bothers me about the Rails 4 in Action thing is not so much how *you* look, but the message that throwaway comment sends to new Ruby / Rails programmers. If a book tells them the community is moving away from Cucumber, they're less likely to try it out.

I'd be very grateful if you could get it pulled out, but I do understand what publishing schedules are like :)

cheers,

Matt

--

tel. +44 (0)20 8720 6842

http://mattwynne.net || https://twitter.com/mattwynne || http://the-cucumber-book.com || http://bddkickstart.com || http://www.relishapp.com

[Aslak Hellesøy]

On Wednesday, 11 December 2013 at 13:08, Matt Wynne wrote:

On 10 Dec 2013, at 18:56, Steve Klabnik <st...@steveklabnik.com> wrote:

Yes, this list has been exceedingly helpful, as well as on the Issues thread.

I knew I was about to do this, which is one of the reasons that the
"Rails 4 in Action" thread bummed me out; it made me look very
anti-Cukes when I knew I was about to end up advocating directly for
it shortly... heh.

Good, glad we could help.

Since you mention it, what bothers me about the Rails 4 in Action thing is not so much how *you* look, but the message that throwaway comment sends to new Ruby / Rails programmers. If a book tells them the community is moving away from Cucumber, they're less likely to try it out.

Btw, what is this statement based on? Cucumber Gem download stats have been steadily increasing.

Aslak

I'd be very grateful if you could get it pulled out, but I do understand what publishing schedules are like :)

cheers,

Matt

--

tel. +44 (0)20 8720 6842

http://mattwynne.net || https://twitter.com/mattwynne || http://the-cucumber-book.com || http://bddkickstart.com || http://www.relishapp.com

[Steve Klabnik]

> I'd be very grateful if you could get it pulled out, but I do understand what publishing schedules are like :)

Nope, it'll change.

[Steve Klabnik]

> Steve, please reply in-line on this list. It makes conversations easier to follow. See list rules in the email footer.

My bad, I should have read them.

> DRY[1] is an honorable principle, but it doesn't apply in all cases. More specifically, it has a lot more value when applied to production code than to tests.

I think this is an important thing to remember, and something that I
forgot. Thank you.

[Steve Klabnik]

> Btw, what is this statement based on? Cucumber Gem download stats have been steadily increasing.

The massive amount of hatred that I receive any time I mention
Cucumber, both in person and on Twitter. :/

A lot of people have had a lot of bad experiences with Cukes, and get
really annoyed. I personally feel this is because their organization
isn't about collaboration, and so from a 'single programmer' or even
'team of Rubyists' perspective, Cukes doesn't provide a whole lot of
value. Or maybe I should phrase that as "Has a chance of a large
downside and only has a little upside."

When presented as a collaboration tool rather than a testing tool, I
think Cukes makes a lot more sense. But there have been a lot of
people advocating for bad cukes over the years, we all remember
web_steps.rb.

[Andrew Premdas]

On 10 December 2013 23:16, Steve Klabnik <st...@steveklabnik.com> wrote:

Duplication is bad, but only in implementation. Say you have two features:

Given I have an account

And I have an active credit card

and

Given I have an account and active credit card

This duplication isn't necessarily bad. Lets follow this down a bit further to the step definitions

Given "I have an account" do

  ...

end

Given "I have an active credit card" do

  ...

end

Given "I have an account and active credit card" do

  ...

end

Now this duplication could be very bad if you put lots of code in the step definitions. You'd be breaking DRY terribly. However if you just call methods then its not bad.

Given "I have an account" do

  create_acccount owner: @i

end

Given "I have an active credit card" do

  create_credit_card owner@i, status: active

end

Given "I have an account and active credit card" do

   create_acccount owner: @i

   create_credit_card owner@i, status: active

end

If you want you could use extract method on the third step definition

Given "I have an account and active credit card" do

   create_account_and card ...

end

def create_account_and card

   create_acccount

   create_credit_card

end

If you think of features and step definitions as only doing one thing which is translating language into method calls. Then duplication in features and step definitions becomes irrelevant. If the business wants to express how they do things in different ways depending on context, thats fine, so long as the implementation remains consistent. So all cucumber is about is translating language into method calls in your test `World`. The quality of this test world is now determined by the expressiveness of its api and other things like the method count, complexity of parameters etc.. Because this is now all implemented in a programming language you can apply all your programming chops to make this as effective as possible.

As for your original problem, it will disappear if you push all the programming down below step definitions, and use abstraction to reduce the size of longer features.

 

--
-- Rules --

1) Please prefix the subject with [Ruby], [JVM] or [JS].

 

2) Please use interleaved answers http://en.wikipedia.org/wiki/Posting_style#Interleaved_style
3) If you have a question, don't reply to an existing message. Start a new topic instead.

You received this message because you are subscribed to the Google Groups Cukes group. To post to this group, send email to cu...@googlegroups.com. To unsubscribe from this group, send email to cukes+un...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/cukes?hl=en
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

------------------------

Andrew Premdas

blog.andrew.premdas.org

[Andrew Premdas]

Cucumber usage is so immature. There are parallels with rails. Most cukers are writing fat controllers and putting their logic in views. Some have moved onto fat models. Only a few are creating their own service level objects. 

--
-- Rules --

1) Please prefix the subject with [Ruby], [JVM] or [JS].
2) Please use interleaved answers http://en.wikipedia.org/wiki/Posting_style#Interleaved_style
3) If you have a question, don't reply to an existing message. Start a new topic instead.

You received this message because you are subscribed to the Google Groups Cukes group. To post to this group, send email to cu...@googlegroups.com. To unsubscribe from this group, send email to cukes+un...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/cukes?hl=en
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Matt Wynne]

Steve,

On 11 Dec 2013, at 11:21, Steve Klabnik <st...@steveklabnik.com> wrote:

Btw, what is this statement based on? Cucumber Gem download stats have been steadily increasing.

When presented as a collaboration tool rather than a testing tool, I
think Cukes makes a lot more sense. But there have been a lot of
people advocating for bad cukes over the years, we all remember
web_steps.rb.

Do you think we need to do more outreach work to spread this message at Ruby conferences?

It's five years now since mabes wrote http://benmabey.com/2008/05/19/imperative-vs-declarative-scenarios-in-user-stories.html and over two years since http://aslakhellesoy.com/post/11055981222/the-training-wheels-came-off. I think I may be guilty of assuming, from inside my bubble, that everyone already knows this stuff.

Maybe I should take the mortgage-driven talk to Ruby / Rails conf? http://skillsmatter.com/podcast/agile-testing/refuctoring-your-cukes

[aslak hellesoy]

On Thu, Dec 12, 2013 at 11:50 AM, Matt Wynne <ma...@mattwynne.net> wrote:

Steve,

On 11 Dec 2013, at 11:21, Steve Klabnik <st...@steveklabnik.com> wrote:

Btw, what is this statement based on? Cucumber Gem download stats have been steadily increasing.

When presented as a collaboration tool rather than a testing tool, I
think Cukes makes a lot more sense. But there have been a lot of
people advocating for bad cukes over the years, we all remember
web_steps.rb.

Do you think we need to do more outreach work to spread this message at Ruby conferences?

It's five years now since mabes wrote http://benmabey.com/2008/05/19/imperative-vs-declarative-scenarios-in-user-stories.html and over two years since http://aslakhellesoy.com/post/11055981222/the-training-wheels-came-off. I think I may be guilty of assuming, from inside my bubble, that everyone already knows this stuff.

Maybe I should take the mortgage-driven talk to Ruby / Rails conf? http://skillsmatter.com/podcast/agile-testing/refuctoring-your-cukes

I think you should.

Even on this list, the majority of people seem to think that Cucumber == Automated Tests == BDD, which is WRONG.

What people need to understand is:

* Cucumber is a tool for BDD

* Cucumber is a tool for Specification By Example

* Specification By Example is just a better name for BDD

* Specification By Example / BDD means examples (Scenarios) are written *before* implementation

* Specification By Example should happen iteratively, in collaboration with non-technical stakeholders

* Automated Tests are a by-product of Specification By Example

* Writing Automated Tests does *not* imply you're doing Specification By Example

* Using Cucumber for Automated Tests without doing Specification By Example is stupid

* Cucumber is not a tool for Automated Testing, it's a tool for Collaborative, Executable Specifications

Aslak

cheers,

Matt

--

tel. +44 (0)20 8720 6842

http://mattwynne.net || https://twitter.com/mattwynne || http://the-cucumber-book.com || http://bddkickstart.com || http://www.relishapp.com

[Rob Park]

+1

[Roberto Lo Giacco]

What can we do to make all this clearer to our audience? Would some sort of introduction to Cucumber, may be a visual one, help the audience understand?

I totally agree with you regarding the general misunderstanding about Cucumber....

[Anthony Green]

On Wednesday, 11 December 2013 18:44:23 UTC, apremdas wrote:

 

Cucumber usage is so immature. There are parallels with rails. Most cukers are writing fat controllers and putting their logic in views. Some have moved onto fat models. Only a few are creating their own service level objects. 

Someone should propose a talk "Cucumber: That's not what we meant" for CukeUp!

[Tim Walker]

[Tim] Comments below. 

[Tim] Cucumber (Like Fit/Fitnesse/Specflow) is not a testing tool. It is a requirements tool. It is a tool to force attention to the domain ubiquitous language. It is a tool for collaboration (as you stated). It is a tool for managing complexity. It is a tool for ensuring the right software is built and that it stays built. It is a tool for baking in quality. Folks that automate after the software has been developed (with a siloed QA team...shudder!) or think they're automating human activity are just seriously missing the point, in my opinion, and might as well be using QTP and waterfall (they are spending a bundle without significant ROI). Haters are going to hate but, not only do I believe Cucumber is going to grow, I will go out on a limb and say it (and tools like it) is the holy grail of agile software development. Further, if a team does not have automated acceptance as an agile practice, I'd suggest that a velocity of zero is the best they will ever be able to achieve (is what was done in sprint 1 still done in sprint 2?). Extreme position? Maybe. But I've been on too many teams with an automation backlog and technical debt that is out of control because they did not make it a part of their *development* practice. Deming knew this when he said: "Cease dependence on inspection to achieve quality". I bought a pair of trousers for $9 that were the worst quality pants I've ever owned. In the pocket was a slip of paper "Inspected by #11". Did that inspection automatically create quality pants? Nope, it is *impossible* to improve the quality of something through inspection. Thanks, Tim 

[Jeff Nyman]

On Tuesday, January 28, 2014 10:24:25 AM UTC-6, Halfordian Golfer wrote:

[Tim] Cucumber (Like Fit/Fitnesse/Specflow) is not a testing tool. It is a requirements tool.

I agree with just about everything you said, except the above quoted part.

With all due respect, in my opinion that kind of either-or thinking is what limits Cucumber, or at least the guiding principles behind it.

Cucumber absolutely is a testing tool AND it is a requirements tool. (The Tyranny of the Or is rarely necessary.) Specifically, Cucumber -- and tools similar to it -- provides a way to encode expressive and intent-revealing requirements in testable form. Or at least that is certainly one viable way to view it.

The problem many evangelists in the Cucumber community have, in my opinion, is that they continue to think of testing as solely an execution activity. In fact, testing is a design activity with an execution component. (Granted, most testers out there do nothing to dispel the limited viewpoint so I get why it is out there.)

When you work with someone to figure out the way to express the intent of a business scenario (or set thereof), you are in fact performing an act of testing if you apply a broad notion to what testing is and what the most responsible time to do testing is. You may be testing assumptions, as just one example. You may be testing if we can express the idea without ambiguity and without contradiction to other ideas.

[Tim Walker]

[Tim] Inline Below. 

[Tim] Fair point. I was intentionally a little dogmatic because, despite our best intentions, we often treat iterative development as miny-*RADCIT workflow where testing is a "test-after" activity. That is not when we write our cucumbers, we write them before the code is built. That is why it's a requirements tool. In my opinion. Thanks! Tim

--
Posting rules: http://cukes.info/posting-rules.html