[Ruby][Gherkin] Effective Way To Write Scenario for Field Validation

[Jeff Nyman]

Greetings all.

I currently work in the clinical trial industry. We are writing up scenarios for how treatment arms of a clinical study are handled. One of the key elements that auditors want to see is that we have discussed the notion of value ranges for certain clinical fields. So as an example, we came up with this during a collaboration:

Scenario: Treatment Arm Value Ranges

  When a value outside the minimum or maximum is entered for a field

    | Field                           | Minimum | Maximum |

    | Treatment Duration              | 2       | 999     |

    | Visits per Subject              | 2       | 999     |

    | Number of CRF pages per subject | 2       | 9999    |

  Then a message indicates which field has an invalid value

The business users want to see that specific information regarding the maximum and minimum values. Further, the developers did as well because they needed to know what specifically to implement. The Then will be changed to incorporate the specific message. (Again, the business users want this so they know we got it right. And the developers want it so they don't have to guess what verbiage to use.)

It's the When that bothers me a bit. I don't like the way it is worded but that's just more of a gut feel rather than something I can point to as "wrong." So I was trying to come up with different ways to word tha, but I'm curious what people here think as well.

(Incidentally, these feature files will be using step definitions that call out to Watir, which will be running these tests against a browser.)

Any input or thoughts is appreciated.

- Jeff

[Andrew Premdas]

First of all I don't think doing validation tests in cucumber like this makes any sense at all. Whilst it may have been useful to put these values in a table to coax some idea of what they should be. Why do they have to remain in the feature once it has been implemented. I'm sure if you showed the whole feature it would give no indication why min value for Treatment Duration should be 2. Why not 27 or 'wibble'. Once you implement this feature you end up with something that is:

- a horrendous violation of DRY

- a really poor piece of documentation

- something that will break if anyone changes the rules about what these minimums and maximums are

- something that is really expensive to run for what it tests ( you can do this sort of thing much faster and cheaper in unit tests)

Finally I suspect once its implemented the only people who will read it will be the developers, and it will just make their lives more difficult.

So instead:

1. Put the values in the code as constants in a module

2. Document the module so it explains why the values are as they are. I'm sure your features does nothing to explain why the minimum treatment duration is 2. 2 what's by the way (days, weeks, visits, semesters ....

3. Generate your documentation as part of your automation process so your business users can see the 'specific information'

3. Write unit tests to show that the validation uses the constants

4. Save the expensive time of feature execution for something more useful

Apologies for the polemic, but hopefully you can find something useful in it.

All best

Andrew

-- 

-- Rules --
 
1) Please prefix the subject with [Ruby], [JVM] or [JS].
2) Please use interleaved answers http://en.wikipedia.org/wiki/Posting_style#Interleaved_style
3) If you have a question, don't reply to an existing message. Start a new topic instead.
 
You received this message because you are subscribed to the Google Groups Cukes group. To post to this group, send email to cu...@googlegroups.com. To unsubscribe from this group, send email to cukes+un...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/cukes?hl=en
 
 

--

------------------------

Andrew Premdas

blog.andrew.premdas.org

[Jeff Nyman]

Interesting. So if I approached this in the way you are describing I would hide all data values. So would this be more in line with your thinking:

When the Treatment Duration is outside the allowed industry values

Then a message indicates that the duration is invalid

As to some of your points:

When you say why should the specific values remain in the feature, our thinking was so that later on, when we test, we know what the values were and what to test for. You might say, "But the automation will have those values stored." A key point that I didn't mention is that the example I showed was a manual test that also acted as the requirement. The automation then uses the manual test as input. We have been trying an approach where we collaborate on the requirements and write them as tests; the example I provided being one such. Those manual tests are then automated and thus we have one source of truth.

You say an interesting thing: "Generate your documentation as part of your automation process so your business users can see the 'specific information'"

I agree! This is definitely something I have been trying to do. I've found this a little problematic with Cucumber because I have to save off the actual values that we use and then output them as part of the report. We use HTML reports and and puts statements have to be in the step definitions and not in modules or they won't show up in the reports. I'm still looking for good examples of how people are using the automation to spit out specific values when more generic steps are executed.

As far as why a minimum should be 2 and not 27, that value is an industry standard -- or at least the current industry standard, that says there must be a duration of 2 or more. I do see what you are saying though in that what I just said there is not called out anywhere in that scenario. Good point. (Does my modified example, provided above, make this more effective?)

Oh, and no need to apologize for the "polemic." :) I truly appreciate your response. I'm learning a lot of this as I go and I know I'm making tons of mistakes. I'm very open to being corrected and showing different ways to do things.

- Jeff

[Andrew Premdas]

More inline

On 16 November 2012 11:33, Jeff Nyman <jeff...@gmail.com> wrote:

Interesting. So if I approached this in the way you are describing I would hide all data values. So would this be more in line with your thinking:

When the Treatment Duration is outside the allowed industry values

Then a message indicates that the duration is invalid

Yeh but generally I wouldn't use cucumber to test individual field validations, its not very efficient at that. I'd probably abstract out to the form overall, e.g.

When I  fill in and submit the wibble form

Then I should see a wibble

When I fill in the wibble form

And I provide an invalid foo

And I submit the form

Then I should see a foo invalid message

or more to my liking

When I fill in the wibble form # autosubmits

Then

When I fill in the wibble form badly #

Then I should see an error

If you're going to test validations exhaustivly use unit tests

 

As to some of your points:

When you say why should the specific values remain in the feature, our thinking was so that later on, when we test, we know what the values were and what to test for. You might say, "But the automation will have those values stored." A key point that I didn't mention is that the example I showed was a manual test that also acted as the requirement. The automation then uses the manual test as input. We have been trying an approach where we collaborate on the requirements and write them as tests; the example I provided being one such. Those manual tests are then automated and thus we have one source of truth.

No I'm saying you should convert your manual test into a group of constants in the application code. Then the automation can use the application code to get the values (in its stepdefs) 

You say an interesting thing: "Generate your documentation as part of your automation process so your business users can see the 'specific information'"

I agree! This is definitely something I have been trying to do. I've found this a little problematic with Cucumber because I have to save off the actual values that we use and then output them as part of the report. We use HTML reports and and puts statements have to be in the step definitions and not in modules or they won't show up in the reports. I'm still looking for good examples of how people are using the automation to spit out specific values when more generic steps are executed.

But cucumber is only one source of documentation. The documentation I'm talking about here is RDOC(or an equivalent) which is much more appropriate for documenting this sort of thing. Cucumber docs should not be spitting out specific values, because Cucumber features shouldn't contain specific values. The business need isn't that there should be a validation error if foo < 2, its:

     if the user puts in an invalid value we should tell them, help them provide a valid value and then let them continue

no specific values are required (making my language better is required, but I'm sure you get the drift)

 

As far as why a minimum should be 2 and not 27, that value is an industry standard -- or at least the current industry standard, that says there must be a duration of 2 or more. I do see what you are saying though in that what I just said there is not called out anywhere in that scenario. Good point. (Does my modified example, provided above, make this more effective?)

With any value in a feature you have to think how likely is this to change, and what happens when it does. Having to rewrite features because the minimum has changed from 2 to 3 is really expensive. Basically you have to go through all your features finding 2's and thinking does this 2 need to become a 3. However if the application code defines a constant for this value all you have to do is change the value of the constant in your application code and rerun the features.

 

Oh, and no need to apologize for the "polemic." :) I truly appreciate your response. I'm learning a lot of this as I go and I know I'm making tons of mistakes. I'm very open to being corrected and showing different ways to do things.

Your welcome, I hope its useful

Andrew

 

- Jeff

--
-- Rules --
 
1) Please prefix the subject with [Ruby], [JVM] or [JS].
2) Please use interleaved answers http://en.wikipedia.org/wiki/Posting_style#Interleaved_style
3) If you have a question, don't reply to an existing message. Start a new topic instead.
 
You received this message because you are subscribed to the Google Groups Cukes group. To post to this group, send email to cu...@googlegroups.com. To unsubscribe from this group, send email to cukes+un...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/cukes?hl=en