Reuse of example tables possible?

[Morten Overgaard Hansen]

Say I have a feature like this:

Feature: Foods in containers

Scenario Outline:
 When a chicken is inserted into a <container>
 Then it gets <condition>

Examples:
  | container | condition |
  | oven      | hot       |
  | freezer   | cold      |
 
Scenario Outline:
 When a roast is inserted into a <container>
 Then it gets <condition>

Examples:
  | container | condition |
  | oven      | hot       |
  | freezer   | cold      |

As can be seen the when, then, and examples are the same for both scenario outlines.

Is it possible to reuse the example tables? I would like to do something like this instead

(or using a similar syntax):

Feature: Foods in containers

Examples: MyExamples
  | container | condition |
  | oven      | hot       |
  | freezer   | cold      |

Scenario Outline:
 When a chicken is inserted into a <container>
 Then it gets <condition>

Using MyExamples
 
Scenario Outline:
 When a roast is inserted into a <container>
 Then it gets <condition>

Using MyExamples

Of cause I could just flatten it out like this:

Feature: Foods in containers

Scenario Outline:
 When a <food> is inserted into a <container>
 Then it gets <condition>

Examples:
  | food    | container | condition |
  | chicken | oven      | hot       |
  | chicken | freezer   | cold      |
  | roast   | oven      | hot       |
  | roast   | freezer   | cold      |

But in my real-life case, the number of scenario outlines are about 10 and the example tables have

like 10+ entries. So the cross product would be enormous.

Is what I'm suggesting in any way possible? Or could I attack my problem in another way, so that I

get a higher degree of reuse and maintainability in some other way?

Any help is appreciated };-P

[Dirk Rombauts]

Hi,

This comes under the heading of attacking the problem another way.

It seems to me that the behaviour of the system is the same for both chicken and roast: if they are put in a freezer, they'll grow cold, and if they're put in an oven, they'll get hot. That leads me to the question: why are you specifying this same behaviour twice? Or 10 times with your real-life data? What do you expect to gain from that?

As a human, I would understand the requirement if you write one scenario outline and some comment in the description:

Feature: Foods in containers

Scenario Outline:

 The same should happen when a roast is inserted instead of a chicken. 
 When a chicken is inserted into a <container>

[Morten Overgaard Hansen]

In my real-life case my "containers" is actually different pieces of hardware that can also be configured in different ways (FPGAs inside).

My "food" likewise is different pieces of hardware and the 2 have to function together to get "condition".

Sometimes new hardware is added in both categories and sometimes the "condition"s are changed.

Therefore it would nice and easy maintain, if the example tables could be reused.

I need to test every combination stated, so what you are suggesting is not really an option, but many thanks for the input.

[Dirk Rombauts]

Hi,

SpecFlow (and Behaviour Driven Development in general) is very good at three things:

1. it helps you to discover and understand the requirements through the discussions you have while writing scenarios and coming up with key examples. This is called "Specification by Example".
2. It keeps a written record of the specification that is always up to date and never grows stale. This is called "Living Documentation".
3. By automating the scenarios, you create an automatic verification of the specification. While writing the automation layer, you determine the logic that the business code needs to support. This is called "Outside-In Development"

SpecFlow/BDD are not good at testing all possible permutations - as you discovered. When people try to do this, sooner or later they run into these problems:

1. The gherkin language does readily support permutations. There are "Scenario Outlines" but no "Scenario Outline Outlines".
2. When trying to understand the specification by reading the scenarios, you suffer from information overload. You don't see the forest because of all the trees.
3. Execution times go through the roof, and the verification will be run less and less frequently. Developers will lose the support of the verification that tells them when they are done.

I know that there are situation where you do need to test all possible permutations. My advice is this: don't use SpecFlow for those cases. I know that SpecFlow and BDD are often marketed as "Test Automation" tools, but that is not where their main value lies.

So I urge you to consider writing this particular set of test cases using a "traditional" unit testing tool (like NUnit). Using well-established code practices you can arrive at a structure that is maintainable, readable and suited to testing all possible permutations.

Cheers,

Dirk

[Morten Overgaard Hansen]

Thanks! That is very valuable feedback. I will try to rethink/rewrite my testingstrategies and -needs. };-P