Cucumber feature syntax convention for a sequential scenario

[mltsy]

I'm pretty new to cucumber, and I'm just learning the conventions.
One feature I am writing lends itself to a kind of "sequential"
scenario. Consider this post both a question "Is there already a
conventional way to express this?" and (if not) a proposal of a
convention to express a sequence of events in cucumber scenarios
without separating the events into separate scenarios. The main
reason is that I feel that, in some cases, splitting it into separate
scenarios is less semantically accurate than keeping it as a single
scenario.

I discovered that using When... Then ... When ... Then ... works
functionally and semantically (in my opinion) but I wonder if there is
another syntax, or if others agree that that is a good convention to
use for sequential scenarios. Here's the actual feature I'm working
on:

My original conventional feature definition: http://pastie.org/3428567
My proposed short-hand for a sequential scenario: http://pastie.org/3429140

But here's a better example of something that would be a good sequence
candidate:

Feature: Subscribe to the email list
In order to easily receive updates via email
As an end-user
I want to subscribe to the e-mail list with a single click from the
homepage

Scenario: Subscribe using a valid e-mail address
Given I am on the homepage
When I enter a valid e-mail address
Then I should see a successful ajax validation message
And the submit button should light up
When I hit the submit button
Then I should see the "successful subscription" page

Notice the interwoven When and Then statements. What do you think of
this convention for this kind of a scenario? A nice syntactic sugar
for this might be "And When..." or is there a better, established way
to express this?

[Chuck van der Linden]

On Feb 21, 12:58 pm, mltsy <j...@ownlocal.com> wrote:
> I'm pretty new to cucumber, and I'm just learning the conventions.
> One feature I am writing lends itself to a kind of "sequential"

> scenario.  ....

> But here's a better example of something that would be a good sequence
> candidate:
>
> Feature: Subscribe to the email list
>   In order to easily receive updates via email
>   As an end-user
>   I want to subscribe to the e-mail list with a single click from the
> homepage
>
>   Scenario: Subscribe using a valid e-mail address
>     Given I am on the homepage
>     When I enter a valid e-mail address
>     Then I should see a successful ajax validation message
>     And the submit button should light up
>     When I hit the submit button
>     Then I should see the "successful subscription" page
>
> Notice the interwoven When and Then statements.  What do you think of
> this convention for this kind of a scenario?  A nice syntactic sugar
> for this might be "And When..." or is there a better, established way
> to express this?

I think by expressing it this way you are both hiding requirements,
and also obscuring business value. in fact the business value is not
that clear in the original scenario.

bear with me a moment and lets pop the why stack a few times

why is there a validation message, why is there validation? why isn't
the submit button enabled to start with and why have it change state?
Why do we care if a user can be added to the list, what is the bottom
line value in that? Why does the business want to make it easy for
the user to subscribe to the list?

There are probably a few other scenarios here that could be describing
things like "in order to <reason> as <actor> we want email addresses
to be validated before a user can submit them to the list" or perhaps
"...we want to prevent submitting the address until after it is
validated."

If those scenarios are created to describe those parts of the behavior
(and also WHY there is value to the business in doing them) then the
original scenario you give could be perhaps brought down to something
more on the order of

Given I am not already on the mailing list
When I enter a valid email address and click the submit button
Then my address is added to the mailing list

Yes this means more scenarios, maybe even more features, but it also
means you are discretely describing the value of each behavior, and if
that behavior should change, you can update just the scenarios
specific to that behavior, and not a bunch of others where the
behavior is incidental. It also makes the features more declaritive
and less imperative which is good as the latter tends to create
features that are brittle and bore the stakeholders with a lot of
incidental details. For example the validation message, and state of
the submit button have no real bearing on this feature, where the
business value is (I would presume) maximum participation in the
mailing list by making it very easy to sign up. Those details ARE
important to other scenarios which deal with why and how you are doing
validation, and preventing invalid values from being submitted, so
they should be present there, but not over there..

btw, one of the best resources for someone new to cucumber (IMHO
anyway) is the newly released 'the Cucumber Book' from Pragmatic
Programmers, I recommend it highly.. if in the long run it saves you
from just one or two wrong turns, it will more than pay for the price
of the book.

[mltsy]

> I think by expressing it this way you are both hiding requirements,
> and also obscuring business value.   in fact the business value is not
> that clear in the original scenario.
>

> why is there a validation message, why is there validation?  why isn't
> the submit button enabled to start with and why have it change state?
> Why do we care if a user can be added to the list, what is the bottom
> line value in that?   Why does the business want to make it easy for
> the user to subscribe to the list?

Thanks for the recommendations. I think I will check out the Cucumber
Book.

The hardest part about accepting this explanation for me is the
implication that cucumber features shouldn't be used for anything but
expressing business values. For instance, a user interface can have
several aspects that are incidental to business values, but should
nevertheless at least be consistent and integration tested so they
don't look broken. But I guess those aspects could all be construed
as business values in a way as well, and written and tested as such.

I will attempt to write features with this business-value focus and
see how it goes. It really took some convincing though!

Back on the original topic, I guess I can't think of a business value
that would need to be expressed as a sequence of events. Maybe all
that detail can be hidden in step definitions. If I do come across a
situation that usurps that assumption, I'll bring it up again! That's
assuming I can bring myself to accept the practice of always writing
cucumber features that express nothing more than business values :)

Thanks!

[Chuck van der Linden]

well I would say that the feature or scenario expresses the value via
why you want something and what it is you want.
The steps in the scenarios express the behavior of what is wanted..

Maybe it's just me but I like to turn As A, I Want, Because around
and do it as Because, As A.., I Want.. start with WHY it helps
keep that always in mind, why are we doing something, keeps the eyes
on the prize as it were. Then go into what we are doing and how it
should work in order to fulfill that initial reason for doing it.

Opinions are going to vary on this, but what I saw was what looked
like a number of different requirements going into that sequence, but
only one of them was being expressed.

Another really good read in this area is Gojko Adzic's "Specification
by Example"

[Andrew Premdas]

On 22 February 2012 17:49, mltsy <j...@ownlocal.com> wrote:

> I think by expressing it this way you are both hiding requirements,
> and also obscuring business value.   in fact the business value is not
> that clear in the original scenario.
>

> why is there a validation message, why is there validation?  why isn't
> the submit button enabled to start with and why have it change state?
> Why do we care if a user can be added to the list, what is the bottom
> line value in that?   Why does the business want to make it easy for
> the user to subscribe to the list?

Thanks for the recommendations.  I think I will check out the Cucumber
Book.

The hardest part about accepting this explanation for me is the
implication that cucumber features shouldn't be used for anything but
expressing business values.  For instance, a user interface can have
several aspects that are incidental to business values, but should
nevertheless at least be consistent and integration tested so they
don't look broken.  But I guess those aspects could all be construed
as business values in a way as well, and written and tested as such.

This is a really important point. There are lots of things Cucumber shouldn't be used for, indeed can't be used for. Its a tool that is part of a toolset, thats part of a process, that tries to ensure you develop what the business wants. When you use Cucumber you have to balance what you specify in the feature, what you trust the developers to do, and what you trust your review/approval process to do. When you write a Cucumber feature that says "The submit button should be highlighted", it does not guarantee the submit button is highlighted!! I could make this feature pass by using any of the following

1) An empty step definition

2) A capybara call that looks for the highlighted class on the subject button

3) Some complex image comparison program that looks at a photoshop image of the UI and compares it pixel by pixel to the screen when the button is highlighted

All of these could give you a green feature and yet still the button might not be highlighted (in 2 the css for highlighting might not work, in 3 the button might not change (i.e. its always highlighted))

Cucumber does not give you certainty, it actually doesn't even tell you that something works. What it does is

1) Say that something is worth reviewing - when the feature first goes green

2) Says that something that was considered good enough when it was reviewed is 'probably' still good enough - if the feature is still green

3) Tells you when something is not working - when a feature goes red.

By the way just doing this is sufficient to transform your development.

You can't replace review, and interaction with Cucumber. When you realise this, then you see that writing features that 'test aspects of the user interface incidental to business value' is pretty much a waste of time. If you want to know if the user interface is good enough someone is going to have to look at it.

HTH

Andrew

 

I will attempt to write features with this business-value focus and
see how it goes.  It really took some convincing though!

Back on the original topic, I guess I can't think of a business value
that would need to be expressed as a sequence of events.  Maybe all
that detail can be hidden in step definitions.  If I do come across a
situation that usurps that assumption, I'll bring it up again!  That's
assuming I can bring myself to accept the practice of always writing
cucumber features that express nothing more than business values :)

Thanks!

--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cu...@googlegroups.com.
To unsubscribe from this group, send email to cukes+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.

--

------------------------

Andrew Premdas

blog.andrew.premdas.org

[Matt Wynne]

4) Forces you to have a *conversation* about what you all think the software should do.

By the way just doing this is sufficient to transform your development.

You can't replace review, and interaction with Cucumber. When you realise this, then you see that writing features that 'test aspects of the user interface incidental to business value' is pretty much a waste of time. If you want to know if the user interface is good enough someone is going to have to look at it.

High five!

cheers,

Matt

--
Freelance programmer & coach
Author, http://pragprog.com/book/hwcuc/the-cucumber-book
Founder, http://www.relishapp.com/
Twitter, https://twitter.com/mattwynne

[jeremia...@gmail.com]

mltsy <j...@ownlocal.com> writes:

>> I think by expressing it this way you are both hiding requirements,
>> and also obscuring business value.   in fact the business value is not
>> that clear in the original scenario.
>

> The hardest part about accepting this explanation for me is the
> implication that cucumber features shouldn't be used for anything but
> expressing business values. For instance, a user interface can have
> several aspects that are incidental to business values, but should
> nevertheless at least be consistent and integration tested so they
> don't look broken. But I guess those aspects could all be construed
> as business values in a way as well, and written and tested as such.
>
> I will attempt to write features with this business-value focus and
> see how it goes. It really took some convincing though!

What I, and I presume many others do is use Cucumber for the
"high-level" business-level stuff, and then use other testing tools to
specify more granular things.

You could, for example, test the behavior of your validation button
using (for example) rspec.

[mltsy]

This all makes sense to me.

Where my sense of the situation differs is in the fact that it is
*nice* to at least have a warning in place if your user incidental
implementation elements break. For instance, maybe the business
requirement is that a user needs to be able to add to a list of
items. But in practice we decide it will be easy (and nice for the
user) to implement it as a jQuery autocomplete input. This is not a
*requirement* of the business, but it's nice to have. So the feature
would simply test "When I submit a new value to the list; Then the
value is stored permanently in the list" which would be easy enough
with a simple fill-in, submit, check for value.

However, if someone gets clever and updates jQuery or adds some other
javascript to the page, and suddenly the incidental autocomplete
interface is broken, our test won't catch that. It would sure be nice
to have a test that would tell us when that is broken. And rspec or
selenium are available, but Cucumber is nicer :) I find that some
people (not just myself) have a tendency to use Cucumber to outline
user stories with implementation detail to ensure that the incidental
*implementation* is functioning. And I feel that this is a perfectly
valid way to use it if that's what your goal is... right?

Is there something wrong with using Cucumber for integration tests of
your implementation? What if those tests were kept separate from your
business value tests, so that you had a folder of
"business_requirements" which would be used to communicate with the
stakeholders, and another folder of "implementation_tests" for testing
incidental implementation details? Or am I missing another important
point :)

-Joe

[Chuck van der Linden]

The emphasis on business value is for me to remember that we should
have a good reason for everything we do. This is essential to avoid
un-necessary work and effort wasted on something that falls into
'YAGNI' land. If something like validating email is important to the
business, then have a scenario just for that requirement which
describes how it will work. Don't include it as an incidental detail
along the way to some other operation. Among other things this makes
the tests more atomic and lets you run just the tests for a particular
function when that is needed. Also when a test fails, it will be far
clearer what is wrong. Consider 'what you know' when you see the
failing test is 'submit button should enable when email is valid'
compared to if the test is 'signup for mailing list with a single
click'

Everyone has a different style, but one of the key ideas behind BDD
and Cucumber is that features are written to be business facing. To
keep them focused and readable by the stakeholders, it's important to
not go into un-necessary sometimes pedantic detail in the feature
files.

If you have a feature (requirement) that is being fulfilled via for
example having the submit button not be active until something else
happens, then by all means express that in the steps for that specific
scenario/feature. Likewise be specific about values and fields for
scenarios that describe data validation, boundary tests etc. But if
the crux of the feature is 'single click signup' then what values get
put into what fields may also all be un-necessary details to the
stakeholder. All the stakeholder may care about for that scenario is
that 'I have filled in the 'join mailing list form' with valid
information' or something similar to that. Leave what values go into
what fields for your step definitions, or a method called by your
step. Yes you will have to do those things, but is describing those
details in feature steps necessary to express the core behavior the
scenario is describing or do they just get in the way?

Also if you have a specific scenario for the behavior of the button,
then it becomes redundant to be checking that a second time during a
scenario for some other behavior. Trust the tests for the other
details to do their thing and don't repeat that validation inside a
different test. Especially since a failure in a detail validation like
the button enabling will halt that scenario, the test becomes
effectively 'blocked' and cannot actually test what it was supposed to
in the first place. That is a typical problem of big 'chain of
events' type of test scenarios that are typical of record and playback
testing tools that aim to test multiple things with one long scenario
(instead of using focused atomic tests). Those tools will generally
report an error and try to keep going, they have to because the tests
are not atomic and if they stop then a huge section of stuff is
untested. (of course very often the ability to proceed at that point
is nill, and it just results in all sorts of chaos when the test tries
to keep going, but I digress.) Tools like rspec and cucumber report
the error, STOP the scenario and move to the next one. Another reason
to not have long sequence chains of multiple tests in a single
scenario.

[mltsy]

On Feb 23, 12:03 pm, Chuck van der Linden <sqa...@gmail.com> wrote:

>
> The emphasis on business value is for me to remember that we should
> have a good reason for everything we do.  This is essential to avoid
> un-necessary work and effort wasted on something that falls into
> 'YAGNI' land.   If something like validating email is important to the
> business, then have a scenario just for that requirement which
> describes how it will work.  Don't include it as an incidental detail
> along the way to some other operation.  Among other things this makes
> the tests more atomic and lets you run just the tests for a particular
> function when that is needed.  Also when a test fails, it will be far
> clearer what is wrong.  Consider 'what you know' when you see the
> failing test is 'submit button should enable when email is valid'
> compared to if the test is 'signup for mailing list with a single
> click'
>
> Everyone has a different style, but one of the key ideas behind BDD
> and Cucumber is that features are written to be business facing.  To
> keep them focused and readable by the stakeholders, it's important to
> not go into un-necessary sometimes pedantic detail in the feature
> files.
>

(I've changed the subject here to reflect my new concern)

Right, I agree with pretty much all of that now. So pedantic detail
should be avoided in the business-facing features, if you are in fact
using cucumber for its intended purpose (BDD), but I'm saying Cucumber
is also quite useful and nice (in my opinion) for integration tests
that may not be included within the business-facing required
features. I do hear your argument that if it's not a business
requirement, then you're wasting your time implementing it. However,
I don't think that's always the case. There's a spectrum between
requirements and wastes of time. Some of them are the "nice to haves"
which may not be included in business requirements because ultimately,
if they are too difficult to achieve, or the interface changes, they
could be left out.

So I can envision a strategy in which you talk to the client about
business requirements, and write those all up before any
implementation detail is discussed. Then you design a user interface,
etc. and the client decides "well, it would be better if this
interface weren't so clunky... I mean it works fine, but can we do X,
Y and Z easily?" These things go beyond the business requirements.
So I would start another section of features (in a separate directory)
for that stuff, and it would be more implementation specific, because
it's just the way the client would like to see the implementation take
shape.

Is there anything wrong with that? Is there another tool that is
better suited to dealing with that kind of detail?

[Chuck van der Linden]

Well speaking from my own point of view, "I mean it works fine, but
can we do X,
Y and Z easily?" sounds a lot like "these changes have value to me,
provided we can do them cheaply enough"

whether and how you codify that in tests depends on just what is being
asked for. Cosmetic stuff and subjective appearance is hard to test
via automation and probably doesn't belong in a cucumber feature. If
they are talking about functionality, then you can add a feature or
scenario to reflect the new behavior that is being asked for. The
value or reason for something like this may be a high level
requirement regarding ease of use, etc

If your steps are declarative rather than imperative, you probably
don't have to re-write ANY of the existing features (or very few of
them). You may have to alter a few step definitions to reflect the
change if for example the identifier or text displayed for a button
changes. otoh if you steps are like 'and I type xxxxx into the field
labeled yyyy' well then any scenario that uses that step will have to
be updated, and perhaps the step file as well.

it's another argument for having most of the implementation details
not appearing up at the feature level

[Jon Kern]

wonderful advice on writing robust cucumber features that best reflect
business requirements, Chuck

jon

blog: http://technicaldebt.com
twitter: http://twitter.com/JonKernPA

Chuck van der Linden said the following on 2/23/12 1:03 PM:

[Matt Wynne]

This is why I don't like the word 'requirement'. It sounds too much like an instruction from a commanding authority, rather than a request from a collaborative peer.

So I can envision a strategy in which you talk to the client about
business requirements, and write those all up before any
implementation detail is discussed.  Then you design a user interface,
etc. and the client decides "well, it would be better if this
interface weren't so clunky... I mean it works fine, but can we do X,
Y and Z easily?"  These things go beyond the business requirements.
So I would start another section of features (in a separate directory)
for that stuff, and it would be more implementation specific,  because
it's just the way the client would like to see the implementation take
shape.

Do you have a concrete example of what you're talking about?

Is there anything wrong with that?  Is there another tool that is
better suited to dealing with that kind of detail?

--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cu...@googlegroups.com.
To unsubscribe from this group, send email to cukes+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.