Using Data Tables in Examples for Scenario Outlines

[Chris Young]

Hi,

A lot of the scenarios I deal with involve two dimensional data sets.
In the example below we are interested in the viewing figures for a number of TV
channels over time owned by the BBC.

Scenario: Get viewing figures owned by the BBC channels over first five days of 2012
Given I am interested in the channels for BBC
And I am interested in how viewing figures for these channels between 2012-01-01 and 2012-01-05
When I go to http://channelstats/BBC
Then I should see:
| | 2012-01-01 | 2012-01-02 | 2012-01-03 | 2012-01-04 | 2012-01-05 |
| BBC 1 | 2000 | 2002 | 2005 | 2004 | 1900 |
| BBC 2 | 1000 | 1012 | 1001 | 1004 | 900 |
| CBBC | 100 | 101 | 101 | 102 | 80 |

My question is, how do I turn it into a Scenario Outline so I can use the same Scenario
for a number of channel owners.

For the atomic elements it's straightforward enough
But how do you paramaterise the Data Table in the 'Then' step?

What I'm looking for is something like this:

Scenario Outline: Get viewing figures for an owners channels over a given period
Given I am interested in the channels for <ChannelOwner>
And I am interested in how viewing figures for these channels between <StartDate> and <EndDate>
When I go to http://channelstats/<ChannelOwner>
Then I should see:
<ViewingFigures>

Examples: UK Channel Groups

| ChannelOwner | StartDate | EndDate | ViewingFigures |
| BBC | 2012-01-01 | 2012-01-05 | <BBCViewingFigures> |
| ITV | 2012-01-01 | 2012-01-05 | <ITVViewingFigures> |

Placeholder: BBCViewingFigures
| | 2012-01-01 | 2012-01-02 | 2012-01-03 | 2012-01-04 | 2012-01-05 |
| BBC 1 | 2000 | 2002 | 2005 | 2004 | 1900 |
| BBC 2 | 1000 | 1012 | 1001 | 1004 | 900 |
| CBBC | 100 | 101 | 101 | 102 | 80 |

Placeholder: ITVViewingFigures
| | 2012-01-01 | 2012-01-02 | 2012-01-03 | 2012-01-04 | 2012-01-05 |
| ITV 1 | 2000 | 2002 | 3005 | 4004 | 5900 |
| ITV 2 | 3000 | 3012 | 3010 | 3014 | 2900 |
| ITV 3 | 1100 | 1101 | 101 | 102 | 80 |
| ITV 4 | 200 | 201 | 201 | 202 | 180 |

Is there a way of expressing this in Gherkin? I can think of ways round it but would they're all pretty ugly :-{

Thanks

Chris

[Andrew Premdas]

On 4 May 2012 08:14, Chris Young <ch...@chrisyoung.org> wrote:

Hi,

A lot of the scenarios I deal with involve two dimensional data sets.
In the example below we are interested in the viewing figures for a number of TV
channels over time owned by the BBC.

Scenario: Get viewing figures owned by the BBC channels over first five days of 2012
 Given I am interested in the channels for BBC
   And I am interested in how viewing figures for these channels between 2012-01-01 and 2012-01-05
  When I go to http://channelstats/BBC
  Then I should see:
       |       | 2012-01-01 | 2012-01-02 | 2012-01-03 | 2012-01-04 | 2012-01-05 |
       | BBC 1 |       2000 |       2002 |       2005 |       2004 |       1900 |
       | BBC 2 |       1000 |       1012 |       1001 |       1004 |        900 |
       | CBBC  |        100 |        101 |        101 |        102 |         80 |

My question is, how do I turn it into a Scenario Outline so I can use the same Scenario
for a number of channel owners.

First of all any use of a table in a feature is a 'smell'. You should immediately ask

1. Why is this table in this feature

2. How do I know its correct e.g. if I put a typo in it, how do I differentiate between a typo in the feature and an error in the code being tested.

The table within table that you are asking for is so obviously messy and error prone, no one could read that and have confidence that its correct.

You could rewrite this feature as

Given there are channel stats for BBC

When I view channel stats for BBC 

Then I should see the channel stats for BBC channels

This deceptively simple feature will give you all the tools you need to write your feature

Scenario Outline

  Given there are channel stats for <channel>

  When I view channel stats for <channel>

  Then I should see channel stats for <channel>

Examples:

  |Channel||

  |BBC|

  |ITV|

  |SKY|

All the detail of dates, viewing figures etc. is irrelevant to this particular scenario, and should be covered in other scenario/features. 

All best

Andrew

-- There are two rules:

1) Please prefix the subject with [Ruby], [JVM] or [JS]. This allows people to filter messages.
2) Please use interleaved answers http://en.wikipedia.org/wiki/Posting_style#Interleaved_style

You received this message because you are subscribed to the Google Groups Cukes group. To post to this group, send email to cu...@googlegroups.com. To unsubscribe from this group, send email to cukes+un...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/cukes?hl=en

--

------------------------

Andrew Premdas

blog.andrew.premdas.org

[chris young]

Thanks for that Andrew

>First of all any use of a table in a feature is a 'smell'.

Why does the language support it then?

>You should immediately ask
>1. Why is this table in this feature

Because we are specifying by example, using representative data to
test whether the system works with concrete examples rather than just
in the abstract.
The product owner will be happy that the feature works when he can see
that when he asks the question

'What do the viewing figures look like for this channel?'

He can see representative data for that channel being reported to him.

And we can test that the system deals with various edge cases in that
test data.

> 2. How do I know its correct e.g. if I put a typo in it, how do I differentiate between a typo in the feature and an error in the code being tested.

Simple. You generate the inputs to the system under test - e.g. the
data to populate the database from which the system pulls its data -
from the feature. Then you run the data through the system and tests
that it comes out the other end in the expected form.

> The table within table that you are asking for is so obviously messy and error prone, no one could read that and have confidence that its correct.

Thats strong language and not particularly helpful.

I am dealing with complex sets of hierarchical data and need to be
able to write tests for them.

>All the detail of dates, viewing figures etc. is irrelevant to this
>particular scenario, and should be covered in other scenario/features.

All that has done is moved the problem somewhere else! Can you
elaborate?

Thanks

Chris

[chris young]

Adding Prefix back in. Apologies for leaving it out, this happens by
default when replying via the Google Groups Web Interface.

[Aslak Hellesøy]

On May 4, 2012, at 3:22, Andrew Premdas <apre...@gmail.com> wrote:

On 4 May 2012 08:14, Chris Young <ch...@chrisyoung.org> wrote:

Hi,

A lot of the scenarios I deal with involve two dimensional data sets.
In the example below we are interested in the viewing figures for a number of TV
channels over time owned by the BBC.

Scenario: Get viewing figures owned by the BBC channels over first five days of 2012
 Given I am interested in the channels for BBC
   And I am interested in how viewing figures for these channels between 2012-01-01 and 2012-01-05
  When I go to http://channelstats/BBC
  Then I should see:
       |       | 2012-01-01 | 2012-01-02 | 2012-01-03 | 2012-01-04 | 2012-01-05 |
       | BBC 1 |       2000 |       2002 |       2005 |       2004 |       1900 |
       | BBC 2 |       1000 |       1012 |       1001 |       1004 |        900 |
       | CBBC  |        100 |        101 |        101 |        102 |         80 |

My question is, how do I turn it into a Scenario Outline so I can use the same Scenario
for a number of channel owners.

First of all any use of a table in a feature is a 'smell'.

That is a very broad statement. At the very least back it up with an argument. What you're saying below doesn't count - it's about a specific case.

Aslak

[Bill Ross]

For what it's worth, we have both scenarios and scenario outlines
that use tables, and my ears aren't burning. My job is in the banking
industry, so you might see something like this:

Scenario Outline: Batch types in batch file
Given a set of transactions with type counts
| OnUsDebit | OnUsCredit | TransitDebit | TransitCredit |
|<onusDebit>|<onusCredit>|<transitDebit>|<transitCredit>|
When the settlement is run
Then the batch file has sections in order: '<description>'

Examples:
|onusDebit|onusCredit|transitDebit|transitCredit|description|
|0|0|0|1|CREDIT T|
|0|0|1|0|...|
...

Offhand I don't have a reasonable 'non-smelly' approach to contrast. E.g.

Scenario: Batch types in batch file
Given all permutations of batch type counts in a set of transactions
When the settlement is run for each
Then the batch files have the correct sections in the correct order

Aside from the fact that this would require massive resources because
it forces parallel execution, and each instance requires its own cluster
of machines, it doesn't specify what is correct. Maybe TestNG with
a DataProvider is more appropriate for a series of permutations, at
least for avoiding BDD criticism :-)

As it is, we have a test that works and is not too opaque for our
readers. Ka-ching.

Bill

[Rob Park]

On Fri, May 4, 2012 at 4:50 PM, Bill Ross <ro...@cgl.ucsf.edu> wrote:

For what it's worth, we have both scenarios and scenario outlines
that use tables, and my ears aren't burning. My job is in the banking
industry, so you might see something like this:

Scenario Outline: Batch types in batch file
   Given a set of transactions with type counts
     | OnUsDebit | OnUsCredit | TransitDebit | TransitCredit |
     |<onusDebit>|<onusCredit>|<transitDebit>|<transitCredit>|
   When the settlement is run
   Then the batch file has sections in order: '<description>'

Examples:
|onusDebit|onusCredit|transitDebit|transitCredit|description|
|0|0|0|1|CREDIT T|
|0|0|1|0|...|
 ...

Offhand I don't have a reasonable 'non-smelly' approach to contrast. E.g.

Scenario: Batch types in batch file
   Given all permutations of batch type counts in a set of transactions
   When the settlement is run for each
   Then the batch files have the correct sections in the correct order

If you are actually just flipping all the bits (0 to 1 to 0), then you could just say something like "try all permutations" and leave the logic of that to the Ruby, because that's hard to read from your feature.

Or... what may be missing is what each row (scenario for the outline) is what each row is about, which would be better described as individual scenarios.

 

Aside from the fact that this would require massive resources because
it forces parallel execution, and each instance requires its own cluster
of machines, it doesn't specify what is correct. Maybe TestNG with
a DataProvider is more appropriate for a series of permutations, at
least for avoiding BDD criticism :-)

As it is, we have a test that works and is not too opaque for our
readers. Ka-ching.

Side note: when there are failures with this test, you won't be able to discern from the error message what the problem is. You'll have to debug it some other way (IME).

//rob

[Bill Ross]

Rob Park <robert...@gmail.com> wrote:

> On Fri, May 4, 2012 at 4:50 PM, Bill Ross <ro...@cgl.ucsf.edu> wrote:
>
> > For what it's worth, we have both scenarios and scenario outlines
> > that use tables, and my ears aren't burning. My job is in the banking
> > industry, so you might see something like this:
> >
> > Scenario Outline: Batch types in batch file
> > Given a set of transactions with type counts
> > | OnUsDebit | OnUsCredit | TransitDebit | TransitCredit |
> > |<onusDebit>|<onusCredit>|<transitDebit>|<transitCredit>|
> > When the settlement is run
> > Then the batch file has sections in order: '<description>'
> >
> > Examples:
> > |onusDebit|onusCredit|transitDebit|transitCredit|description|
> > |0|0|0|1|CREDIT T|
> > |0|0|1|0|...|
> > ...
> >
> > Offhand I don't have a reasonable 'non-smelly' approach to contrast. E.g.
> >
> > Scenario: Batch types in batch file
> > Given all permutations of batch type counts in a set of transactions
> > When the settlement is run for each
> > Then the batch files have the correct sections in the correct order
> >
>
> If you are actually just flipping all the bits (0 to 1 to 0), then you
> could just say something like "try all permutations" and leave the logic of
> that to the Ruby, because that's hard to read from your feature.

Doing that sequentially is a pain, since steps are not pipelines.
Suppose BDD could be upgraded to allow a sequence of tests to be
specified by a 'pipe step' which would feed the other steps, as
an alternative to scenario outlines. E.g.

GivenPipe all permutations
When [for each permutation] X runs
Then the right thing happens [for each permutation]

However this still doesn't map input params to outputs for the reader.

>
> Or... what may be missing is what each row (scenario for the outline) is
> what each row is about, which would be better described as individual
> scenarios.

That would smell more, since who wants to expand a scenario outline.

>
>
> >
> > Aside from the fact that this would require massive resources because
> > it forces parallel execution, and each instance requires its own cluster
> > of machines, it doesn't specify what is correct. Maybe TestNG with
> > a DataProvider is more appropriate for a series of permutations, at
> > least for avoiding BDD criticism :-)
> >
> > As it is, we have a test that works and is not too opaque for our
> > readers. Ka-ching.
> >
>
> Side note: when there are failures with this test, you won't be able to
> discern from the error message what the problem is. You'll have to debug it
> some other way (IME).

These are regression tests to catch errors. We will know that the
failure occurs for a given set of inputs, and are capable of diagnosing
from there. :-)

Bill

[chris young]

On May 4, 11:50 pm, Bill Ross <r...@cgl.ucsf.EDU> wrote:
>Aside from the fact that this would require massive resources because
>it forces parallel execution, and each instance requires its own cluster
>of machines, it doesn't specify what is correct.

Exactly my problem, without representative test data a scenario like
that proposed by Andrew:

On May 4, 2012, at 3:22, Andrew Premdas <aprem...@gmail.com> wrote:
>Scenario Outline
> Given there are channel stats for <channel>
> When I view channel stats for <channel>
> Then I should see channel stats for <channel>

Is so abstract as to tell you nothing about what it means for it to
execute successfully. As I said in my mail above, it just moves the
problem somewhere else.

I have used the Gherkin language to great effect as a tool for
enabling specification workshops on data intensive subjects like
Content Discovery, where a viewer of an On Demand video service
navigates hierarchies of content and then gets given a version
appropriate to their viewing preferences and internet connection, and
Digital Video Recorder (DVR) to model Clash Management Scenarios.

Now I am using Cucumber.js for JavaScript and Lettuce + Sellenium for
Python driven UI testing and am faced with similarly data rich
scenarios to test.

The great thing about Specification By Example is it makes you think
in terms of real world examples, you get specific about the behaviour
you expect to see and this helps you discover all sorts of edge cases
and subtleties which you do not get when you model things in the
abstract.

To solve the problem that I presented to the group that kicked this
thread off I am leaning towards using a pre-processor to combine the
Scenario Outline and the test data together into something which can
then be executed by Cucumber.js or Lettuce. This has the benefit of
keeping the test data separate from the Scenario descriptions making
it easer to use for the creation of other test artefacts such as
scripts to load databases or stimulate the System Under Test in other
ways.

[Matt Wynne]

Chris,

Step back from Gherkin for a minute and think about how you'd describe these examples on a whiteboard, or in a traditional specification document. What would they look like if you weren't constrained by Gherkin's syntax?

For me, if the viewing figures table is different for each channel, I would probably just have one scenario per channel and not worrying about a bit of duplication.

[chris young]

On May 5, 9:58 am, Matt Wynne <m...@mattwynne.net> wrote:
> Step back from Gherkin for a minute and think about how you'd describe
> these examples on a whiteboard, or in a traditional specification document.
> What would they look like if you weren't constrained by Gherkin's syntax?

Thanks for that Matt. Very helpful suggestion.

Writing the requirements as straight prose I came up with:

"Channel owners should be able to see the performance of their
channels in relation to one another over time in the form of a table
of viewing figures. If there are more than ten channels owned then
the results should be paginated. If there is bum data for a channel
then no data should be displayed for that channel and instead an error
message should be displayed. If a single channel is massively
outperforming the others it should be highlighted etc etc."

Doing this led me to the conclusion that I was actually dealing with a
number of distinct Scenarios. Going back to Gherkin this would better
be expressed as such rather then as a Scenario Outline e.g:

Scenario: Channel Owner with only one channel Scenario: Channel Owner
with 100 channels
Scenario: One channel with bum data, all the rest okay
Scenario: Channel group with a clear winner in the viewing stakes
Scenario: Channel in Terminal Decline Etc etc.

In each Scenario we might be interested in more/different outcomes
i.e. When steps.

The real advantage of this approach is that by stating the purpose of
each scenario we can assess its value to the business.

It also gives us a set of small deliverable increments which because
of their size can flow through development quicker.

Okay so there will be a bit of duplication but that's a trade off
against the increased value we get from a number of discreet, self
contained purpose revealing scenarios.

Looking at it now, I think I was fixating on getting the Gherkin DRY
at the expense of the actual purpose of the feature.

BTW I got the Cucumber book a week ago. Absolutely love it. Getting
so much value from it.

Cheers

Chris

[Matt Wynne]

High five!

Looking at it now, I think I was fixating on getting the Gherkin DRY
at the expense of the actual purpose of the feature.

I think Dan North has a little sidebar in our book all about this - how flow trumps dry when you're telling a story in a test. Let the storyteller in you win out over the engineer when you're writing scenarios.

BTW I got the Cucumber book a week ago.  Absolutely love it.  Getting
so much value from it.

Glad to hear it. :)

Cheers

Chris

-- There are two rules:

1) Please prefix the subject with [Ruby], [JVM] or [JS]. This allows people to filter messages.
2) Please use interleaved answers http://en.wikipedia.org/wiki/Posting_style#Interleaved_style

You received this message because you are subscribed to the Google Groups Cukes group. To post to this group, send email to cu...@googlegroups.com. To unsubscribe from this group, send email to cukes+un...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/cukes?hl=en

cheers,

Matt

--
Freelance programmer & coach
Author, http://pragprog.com/book/hwcuc/the-cucumber-book
Founder, http://www.relishapp.com/
Twitter, https://twitter.com/mattwynne

[Andrew Premdas]

On 4 May 2012 13:30, Aslak Hellesøy <aslak.h...@gmail.com> wrote:

On May 4, 2012, at 3:22, Andrew Premdas <apre...@gmail.com> wrote:

On 4 May 2012 08:14, Chris Young <ch...@chrisyoung.org> wrote:

Hi,

A lot of the scenarios I deal with involve two dimensional data sets.
In the example below we are interested in the viewing figures for a number of TV
channels over time owned by the BBC.

Scenario: Get viewing figures owned by the BBC channels over first five days of 2012
 Given I am interested in the channels for BBC
   And I am interested in how viewing figures for these channels between 2012-01-01 and 2012-01-05
  When I go to http://channelstats/BBC
  Then I should see:
       |       | 2012-01-01 | 2012-01-02 | 2012-01-03 | 2012-01-04 | 2012-01-05 |
       | BBC 1 |       2000 |       2002 |       2005 |       2004 |       1900 |
       | BBC 2 |       1000 |       1012 |       1001 |       1004 |        900 |
       | CBBC  |        100 |        101 |        101 |        102 |         80 |

My question is, how do I turn it into a Scenario Outline so I can use the same Scenario
for a number of channel owners.

First of all any use of a table in a feature is a 'smell'.

That is a very broad statement. At the very least back it up with an argument. What you're saying below doesn't count - it's about a specific case.

I did, and I am happy to do it again, in a little more detail

First of all, lets define a smell, because this term might be to pejorative for some. So 

"A smell is something about a piece of code that says, you should stop and have a think about whether something is wrong."

So why is any use of a table in a feature a smell.

All tables in features have the following characteristics

1. They are presenting details of something that can be abstracted to a higher level concept

2. They are error prone in entry, particularly tables that have many columns or rows

3. They are harder to read when the feature outputs

4. They are harder to debug when something goes wrong. You have to Identify why one row fails and another passes, ditto for columns

5. They require more complex step definitions to process, and again this processing is error prone

Finally they almost always are not DRY, in that the data you define in the table will have to be defined somewhere else (in your application, or its data). This means that when the data changes e.g. a new row or column needs to be added, then the feature has to change.

All these things are detrimental to the overall effectiveness of your features. 

So if you use a table, you should

1. Be aware of the detrimental effects of them, and place some value on this (the cost)

2. Have a quick think about whether the benefits you think you are getting from the table outweigh this cost

Thats pretty much all I'm saying when I state 

"Any use of a table in a feature is a 'smell'."

All best

Andrew

[Andrew Premdas]

First of all see my reply to Aslak

On 4 May 2012 12:10, chris young <ch...@chrisyoung.org> wrote:

Thanks for that Andrew

>First of all any use of a table in a feature is a 'smell'.

Why does the language support it then?

Because sometimes the benefits gained from using a table might outweigh the detrimental effects of using the table, and because when this support was added the detrimental effects of using tables were not understood. Personally I used to use tables all the time, and it took me quite a long time for me to question my usage of them, work out alternatives to using them, and get a better understanding of there overall effect on a suite of features. I also used to use web steps, nest my steps, and use complex regular expressions to try and reduce the number of step definitions I wrote. I don't do any of those any more. One of the beauties of Cucumber is that there are always different ways to express yourself

 

>You should immediately ask
>1. Why is this table in this feature

Because we are specifying by example, using representative data to
test whether the system works with concrete examples rather than just
in the abstract. 

The idea that concrete examples in the feature show more conclusivly that the system works is a fallacy. The nature of any Cucumber feature whatever its language is that it can be green and the business value it says its delivering may not be satisfactory or even implemented at all. You cannot review the functionality of an application by reading the language of a passing cucumber feature. The language of the feature has no relationship to how the system is tested or what data is used to test it. I can write a 3 line feature that tests your system far more exhaustively than your feature with a table. Equally I can write a hundred line feature that doesn't test anything at all. 

You can't review the functionality be reading the feature; you can use representative data without putting it in the feature; and the abstract, or non abstract nature of the language in the feature bears no relationship to the quality or quantity of tests, So why is the table in the feature? 

 

The product owner will be happy that the feature works when he can see
that when he asks the question

'What do the viewing figures look like for this channel?'

He can see representative data for that channel being reported to him.

But he need to look at the application to see this, reviewing the feature after the functionality is implemented is not sufficient.

 

And we can test that the system deals with various edge cases in that
test data.

You can test edge cases by using unit tests, or using a scenario for each particular edge case. A table obfuscates what edge cases you are testing by

1. Testing more than one case in one expression

2. Not having a description for each edge case

 

 

> 2. How do I know its correct e.g. if I put a typo in it, how do I differentiate between a typo in the feature and an error in the code being tested.

Simple.  You generate the inputs to the system under test - e.g. the
data to populate the database from which the system pulls its data -
from the feature.  Then you run the data through the system and tests
that it comes out the other end in the expected form.

What are you saying? That you have to parse the feature to get data for the database or that you have generate the feature from data in your database? Either approach is very unusual for using Cucumber features.

> The table within table that you are asking for is so obviously messy and error prone, no one could read that and have confidence that its correct.

Thats strong language and not particularly helpful.

I'm sorry you feel that way, my reply was aimed to be helpful and show alternative ways of expressing your problem.

 

I am dealing with complex sets of hierarchical data and need to be
able to write tests for them.

Nothing that other people haven't done in the past with cucumber. My point is that features are not the place to express these complex sets of data, if you want to be effective and expressive whilst using Cucumber

 

>All the detail of dates, viewing figures etc. is irrelevant to this
>particular scenario, and should be covered in other scenario/features.

All that has done is moved the problem somewhere else!  Can you
elaborate?

There are two major points here

1. You should try and deal with one simple problem in each scenario, and use the cumulative effect of running many scenarios (and unit tests) to test complex behaviour e.g. the details of dates, viewing figures etc.

2. Part of the reason you are struggling is that you are dealing with too many problems at once, moving some of these elsewhere is a good thing.

 

Overall you should

Break your big problem into a number of small problems

Let your application, unit tests and step definitions deal with all the specifics of the data that you are currently putting into tables in your features. This will make using Cucumber much more productive

 

All best

Andrew

Thanks

Chris

-- There are two rules:

1) Please prefix the subject with [Ruby], [JVM] or [JS]. This allows people to filter messages.
2) Please use interleaved answers http://en.wikipedia.org/wiki/Posting_style#Interleaved_style

You received this message because you are subscribed to the Google Groups Cukes group. To post to this group, send email to cu...@googlegroups.com. To unsubscribe from this group, send email to cukes+un...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/cukes?hl=en

[chris young]

On May 6, 10:00 am, Andrew Premdas <aprem...@gmail.com> wrote:

>But he need to look at the application to see this, reviewing the feature
>after the functionality is implemented is not sufficient.

In this case, he is not reviewing the feature, the feature is being
run against the application as a full stack test using Lettuce +
Sellenium Webdriver. Sellenium is doing the 'looking at' the
application. You can watch it visit the stats website for different
channel group owners and see it do some sanity checks on what is
appearing in the browser.

One of the great things about Cucumber (and its ports to other
languages) is that you can use it at different levels of abstraction
and at different tiers of the stack.

> wrote. I don't do any of those any more. One of the beauties of Cucumber is
> that there are always different ways to express yourself

Indeed there are, so making a sweeping statement like 'any use of a
table in a feature is a 'smell' misses this point.

> The idea that concrete examples in the feature show more conclusivly that
> the system works is a fallacy. The nature of any Cucumber feature whatever
> its language is that it can be green and the business value it says its
> delivering may not be satisfactory or even implemented at all.

Again, this feature is testing the full stack from the top down,
testing the application as it appears in the browser. It is being
tested against inputs to the system based on the same set of test
data. We are exercising the full stack. This gives us some
confidence that the feature has been implemented.

> You cannot
> review the functionality of an application by reading the language of a
> passing cucumber feature. The language of the feature has no relationship
> to how the system is tested or what data is used to test it.

Why not? If you're using it to drive Selenium to test the application
in the browser then it has everything to do with how the system is
tested and what data is used to test it.

> you can use
> representative data without putting it in the feature;

So where would you put it if not in the feature?

> But he need to look at the application to see this, reviewing the feature
> after the functionality is implemented is not sufficient.

I'm not suggesting he do that. Yes he has to look at the application
but this is going to be at best ad hoc and being manual will be labour
intensive and error prone. With an automated test, expressed in
Gherkin, implemented by Lettuce driving Selenium we can ensure we are
frequently and automatically testing the functionality before
presenting it to the product owner.

> You can test edge cases by using unit tests, or using a scenario for each
> particular edge case. A table obfuscates what edge cases you are testing by
>
> 1. Testing more than one case in one expression
> 2. Not having a description for each edge case

So if you see my reply to Matt's suggestion above you'll see I've come
to the conclusion that what I do need is indeed separate scenarios,
one for each edge case. However, each scenario will still contain a
table of expected data as it is that data that best describes the
characteristics of the scenario / edge case.

Unit Tests are essential but they are at a smaller level of
granularity as they are testing individual functions and methods. You
need them in addition to the tests at the feature level not instead
of.

> What are you saying? That you have to parse the feature to get data for the
> database or that you have generate the feature from data in your database?
> Either approach is very unusual for using Cucumber features.

The former. To test the full stack, top to bottom, we need a set of
known inputs and a set of expected outputs. The later will be a
function of the former.

> > I'm sorry you feel that way, my reply was aimed to be helpful and show
> alternative ways of expressing your problem.

I'd love to see this alternative way of expressing the problem. In
your original post you said "All the detail of dates, viewing figures

etc. is irrelevant to this particular scenario, and should be covered

in other scenario/features." Where/what are these? Just saying:

> Given there are channel stats for BBC
> When I view channel stats for BBC
> Then I should see the channel stats for BBC channels

Is so abstract as to tell me nothing.
Where are all the specifics of the behaviour we expect to see so we
can test whether it is there or not?

> 1. You should try and deal with one simple problem in each scenario, and
> use the cumulative effect of running many scenarios (and unit tests) to
> test complex behaviour e.g. the details of dates, viewing figures etc.

So again, I'm happy with having multiple scenarios but I still don't
see why their containing tables is a 'smell'

> 2. Part of the reason you are struggling is that you are dealing with too
> many problems at once, moving some of these elsewhere is a good thing.

Thanks but I'm not struggling! I totally get that you solve big
problems by breaking them down into smaller problems and that is being
done and the individual functions and methods that make up the code
base are covered by unit tests.

What those tests don't cover is what the cumulative purpose of those
units of functionality is, what they should net out at, what the
overall behaviour of the system should be.

That is what these tests are intended to cover.

Chris

[Andrew Premdas]

Chris

-- There are two rules:

1) Please prefix the subject with [Ruby], [JVM] or [JS]. This allows people to filter messages.
2) Please use interleaved answers http://en.wikipedia.org/wiki/Posting_style#Interleaved_style

You received this message because you are subscribed to the Google Groups Cukes group. To post to this group, send email to cu...@googlegroups.com. To unsubscribe from this group, send email to cukes+un...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/cukes?hl=en

My apologies Chris, it seems that my attempt to provide help/answers to your original request have been less than constructive. I hope you find solutions more suitable for you.

All best

Andrew

[chris young]

Hey,

Nothing to apologise for, it's great to have the debate and I
appreciate your interest in my original question.

Chris