Paper on Imperative vs Declarative style / Nested steps

[Matt Wynne]

Here's a paper that might be interesting:

http://www.mdpi.com/1999-5903/3/2/159/pdf

I haven't read the whole thing, but it's apparently about raising the level of abstraction in acceptance tests to what we've been calling the more 'declarative' style, and getting success from that.

What I think is interesting is that the more imperative steps still seem to be part of the framework, part of the 'tester's domain', just at a lower level. This is basically what we'd call nested steps. Robot framework also supports this. I wonder whether we should look to supporting nested steps more at the Gherkin level, and move away from allowing people to call them from Ruby code at all. Telling people to use Ruby methods (I'm looking at you Mr Premdas) is fine if they're comfortable writing Ruby code, but there are plenty of testers who would prefer to work in a more contained environment. I think there was a 'macros' ticket kicking around in lighthouse about this idea some time ago.

Thoughts?

cheers,
Matt

ma...@mattwynne.net
07974 430184

[Andrew Premdas]

<nostalgia> 

Ah I remember the days when Cucumber was a tool for doing BDD, being part of that workflow expressed so well in the beggining of the RSpec book. In those distant days people were really interested in driving development with Cucumber, iterating rapidly, and using Cucumber as a tool to help write good code.

It seems those times are just a distant memory now. Now everyone wants to do big specification up front using Cucumber - or even worse bad specification after the fact. Cucumber it seems is now a tool for "testers". I expect it will be soon part of the ISQTB cirricula and we can all get certified!  

Of course being a testing tool means there is a whole new enterprise market to exploit, with all those "testers' who we can't really expect to be able to write some simple Ruby code, (or code in the language their product is being developed). And the very idea of them <heretical> pairing with a developer </heretical> and learning some coding is just to much to bear.

Seems that Cucumber is destined to move into the world of MDD (the new old hotness)!

</nostalgia>

All best

Grumpy old Premdas

 

cheers,
Matt

ma...@mattwynne.net
07974 430184

--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To post to this group, send email to cu...@googlegroups.com.
To unsubscribe from this group, send email to cukes+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.

--

------------------------

Andrew Premdas

blog.andrew.premdas.org

[Mike Sassak]

I understand your frustration with testers who seem mystified at the
suggestion that maybe it would be nice to be able to write even a
little bit of code, but I don't understand the strength of your
reaction to the article. It seems like a good explanation of the
problems at hand (brittle steps), and some possible ways to allow
testers themselves to address them. What's wrong with that?

Matt, how would macros in Gherkin look? I think I remember being not a
fan of the idea when it came up, but I can't remember any details.

Mike

[Andy Tinkham]

My take on how this would be applied actually went down a different path - rather than allowing the automatic creation of nested scenarios (which is a large part of what I took the ZiBreve app described in the paper to do), I envisioned it as a Formatter similar to the one that shows unused step definitions. You could run it and it would identify places where you use the same steps repeatedly. It seems like this would be a useful way to get to more declarative scenarios. For example, if your spec was

When I see the dialog
And the dialog says "Foo"
And I click OK

And you used that repeatedly, the Formatter could identify that and say "These steps are frequently called together - perhaps they should be collapsed into a broader step" (or use language that's more clear to the user, of course). It seems like identifying these groups and having the users manually collapse them would be very likely to push things towards a more declarative format.

I actually really like this idea - once I get a team established and up and running, this may go on my to do list to see what I can build (assuming no one beats me to it :) ) Thanks for sharing the article, Matt!

Andy
http://testerthoughts.com

(And as an aside, inline commenting is really annoying in Outlook - anyone have suggestions on better ways to get previous stuff quoted and make it clear what's new? (Other than the not helpful "Don't use Outlook" or "Use <XYZ>", of course? :) )

[Andrew Premdas]

I haven't actually read the paper my comments were more about a perceived  general pattern of usage of cucumber as a 'testing tool' which ignores its BDD roots.

They were also meant to have some humour - always a dangerous thing to attempt in an email :)

Finally the best way to address the problem is to stop separating testing and development, pair with 'testers' and teach them how to write good features and step definitions. Adding another layer between the feature and what they actually do is just going to complicate things and at the same time increase the overhead to understanding how a feature is implemented. Instead we should move in the other direction and connect people directly to the tools they are going to need e.g. capybara, and ruby methods as directly and frequently as possible. Then they will quickly learn how things work, how expressive ruby is and how important it is to KISS

All best

Andrew

[Robert]

A great read Matt. Thanks for sharing.

Now, on to a few comments from a tester. :-)  Disclaimer ... disclaimer ... I have had no formal education in computer science.  I do not hold a bachelor or master's degree in computer science.  I do have about 8 years experience in the field of Quality Assurance, and I have developed software in Java and Perl, using what I've learned from various books, on-line tutorials and being as inquisitive as I possibly can.  I am currently using Cuke4Duke and Cucumber to build out several projects.

Many of my former colleagues, continue to work in shops where they have not heard of BDD or if they have, they (the developers, business analysts, product owners, testers) are not too interested in learning what sort of benefits might arise from adopting BDD.  Adoption of BDD, as I see things, is really a dramatic shift in how people work and think and requires the buy-in of the entire company.  And, generally speaking, most (not all of course) people are fearful of change.

A number of the tester's I know, continue to be involved with

• writing Test Plans -- these are usually put together using Word or published to a wiki
• writing Test Cases -- these are usually put together in Excel
• testing the application -- generally, this is a manual process.  Automation is usually an afterthought...that is, until it appears that the sheer number of test cases looks like it will require several testers to complete.  Then, automation becomes a hot-topic issue.

There are several former colleagues of mine who have recently discovered Cucumber. Keep in mind, these teams are struggling to find an open-source solution that is relatively easy to understand and not too difficult to implement.  Cucumber seems to fit this bill.   I realize that Cucumber is not considered a testers tool, but when a tester sees how easy it is to create test cases based on plain text files in a very structured sort of way, Cucumber becomes very appealing.  Many of these testers, create the test scenarios in a way that was mentioned in the article -- i.e., the scenarios read more like test scripts.  Testers are accustom to testing in this manner.  Crack open any test case Excel file, and you will find a series of steps outlined -- for example: (1) open the home page, (2) enter the username, (3) enter the password, etc.

The problem with this approach, as outlined in the article, is that this quickly leads to very brittle and unmaintainable tests.  Such tests, will likely lead to many testers blaming (unfairly) Cucumber for their problems.  The article also points out the idea of abstracting these sorts of test scripts into a more business-level (focused) set of rules.  References are also made to Gojko Adzic's publications, Specification By Example: How Successful Teams Deliver the Right Software and Bridging the Communication Gap: Specification by Example and Agile Acceptance Testing.  Both of which are a great read, and have helped me tremendously to understand the importance of crafting test scenarios at a business-level rather than at an implementation-level.  The unfortunate truth is that a number of testers that I know have based many of their automated tests on the imperative style.

If the option was given to Cucumber users to allow them to take their nested steps and restructure them into a more business-level format (declarative style), I think this may be of some help to the testers I know.  Or, possibly, just more education on the pitfalls of using an imperative style might be of some use.  My apologies if I've rambled a bit.  Obviously, my ramblings are solely based on my experience as a tester, and are not reflective of the entire testing community.

[Matt Wynne]

@Pekka, I'm trying to steal a feature from Robot, I thought you might be interested :)

Here's the (relatively opaque) Robot framework documentation:
http://robotframework.googlecode.com/svn/tags/robotframework-2.5.6/doc/userguide/RobotFrameworkUserGuide.html#creating-user-keywords

In Robot, you can define them in their equivalent of a feature file, which scopes them to that feature. Or you can define them elsewhere and they're global. It would work something like this:

Feature: Wizard
Scenario: Wizard Step 2
Given I am on step 2 of the wizard
When I click "Next"
Then I should be on step 3

Macro: I am on step #{step_number} of the wizard
Given I have visited the wizard start page
And I have clicked "Next" #{step_number} times

The #{} notation allows you to capture a variable in the macro and then use it in the steps.

The nice thing about doing it at the Gherkin level like this, versus doing it in Ruby code, is that it's declarative. So you can look at some features and describe the mapping from high-level step to low-level steps without running the features.

cheers,
Matt

--
Freelance programmer & coach
Founder, http://relishapp.com
+44(0)7974430184 | http://twitter.com/mattwynne

[Matt Wynne]

Thanks for your contribution to the discussion, Robert.

It's worth noting that you can already do this. You can call the #steps method from within a step definition, and pass it arbitrary gherkin steps to try to run:

Given /I am on page 3 of the wizard/ do

   steps %{

     Given I have completed page 1 of the wizard

     And I have completed page 2 of the wizard

   }

end

There are maintainability problems with this approach, which are well documented on this list if you search the archives (for Nested Steps). But as a first step to help move away from imperative steps, it can be a great move.

[Robert]

Thanks, Matt.  This could certainly lessen the troubles of a number of my colleagues.  The example you provided looks like it was meant for Ruby.  Is this feature also supported in Java (i.e., Cuke4Duke) ? And, if so, could you provide an example.

 

[Mike Sassak]

That's a pretty cool feature I would probably never use myself! It
shouldn't be too hard to add a new keyword to Gherkin. Have you run
this past any QAs to see what they think of it? My impression from the
ML here is that it would be welcome, but that's just my impression.

Mike

> cheers,
> Matt
>
> --
> Freelance programmer & coach
> Founder, http://relishapp.com
> +44(0)7974430184 | http://twitter.com/mattwynne
>

[Matt Wynne]

Nope, I'm not coming across many day-to-day at the moment.

Are there any QAs readying this who would like to comment?

Pekka said his impression of Cucumber was that it was much more for developers, wheras Robot has been written by and for testers. I take Andrew's point, but I would like Cucumber to be as useful to as wide an audience as possible, as long as that doesn't mean making it unduly complicated.

> My impression from the
> ML here is that it would be welcome, but that's just my impression.
>
> Mike
>
>> cheers,
>> Matt
>>
>> --
>> Freelance programmer & coach
>> Founder, http://relishapp.com
>> +44(0)7974430184 | http://twitter.com/mattwynne
>>
>> --
>> You received this message because you are subscribed to the Google Groups "Cukes" group.
>> To post to this group, send email to cu...@googlegroups.com.
>> To unsubscribe from this group, send email to cukes+un...@googlegroups.com.
>> For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups "Cukes" group.
> To post to this group, send email to cu...@googlegroups.com.
> To unsubscribe from this group, send email to cukes+un...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
>

cheers,
Matt

ma...@mattwynne.net
07974 430184

[Matt Wynne]

Thanks, Matt.  This could certainly lessen the troubles of a number of my colleagues.  The example you provided looks like it was meant for Ruby.  Is this feature also supported in Java (i.e., Cuke4Duke) ? And, if so, could you provide an example.

I'm afraid I don't know the answer to this. I expect there is a way, but I haven't used Cuke4Duke much. Can anyone help?

[aslak hellesoy]

Here is an example:

https://github.com/cucumber/cuke4duke/blob/master/examples/java/src/test/java/simple/CallingSteps.java

Aslak

 

cheers,

Matt

ma...@mattwynne.net

07974 430184

[Robert]

On Thursday, June 9, 2011 9:01:55 AM UTC-7, Aslak Hellesøy wrote:

On Thu, Jun 9, 2011 at 4:40 PM, Matt Wynne <ma...@mattwynne.net> wrote:

On 8 Jun 2011, at 23:55, Robert wrote:

On Wednesday, June 8, 2011 8:06:59 AM UTC-7, mattwynne wrote:

On 5 Jun 2011, at 20:15, Robert wrote:

be of some help to the testers I know.  Or, possibly, just more education on the pitfalls of using an imperative style might be of some use.  My apologies if I've rambled a bit.  Obviously, my ramblings are solely based on my experience as a tester, and are not reflective of the entire testing community.

Thanks for your contribution to the discussion, Robert.

It's worth noting that you can already do this. You can call the #steps method from within a step definition, and pass it arbitrary gherkin steps to try to run:

Given /I am on page 3 of the wizard/ do

   steps %{

     Given I have completed page 1 of the wizard

     And I have completed page 2 of the wizard

   }

end

There are maintainability problems with this approach, which are well documented on this list if you search the archives (for Nested Steps). But as a first step to help move away from imperative steps, it can be a great move.

Thanks, Matt.  This could certainly lessen the troubles of a number of my colleagues.  The example you provided looks like it was meant for Ruby.  Is this feature also supported in Java (i.e., Cuke4Duke) ? And, if so, could you provide an example.

I'm afraid I don't know the answer to this. I expect there is a way, but I haven't used Cuke4Duke much. Can anyone help?

Here is an example:

https://github.com/cucumber/cuke4duke/blob/master/examples/java/src/test/java/simple/CallingSteps.java

Aslak

 

Thanks, Aslak!

 

[Robert]

I'd like to comment.  I have had no experience with the Robot framework, so I can't really give you any sort of impression from a tester's perspective, I have been working with Cucumber (ala Cuke4Duke) for the past 6 months now.   I disagree with Pekka's impression.  Cucumber is pretty easy to get up and running -- even for a tester :-).  It's language is very expressive for creating acceptance tests, and the fact that it is structured in such a way that each scenario includes a pre-condition (Given) an action (When) and an expected behavior or outcome (Then) lends itself to creating very structured tests that attempt to verify one condition. 

I've introduced it to various colleagues of mine, and they have been pleasantly surprised with the Given-When-Then syntax.  Prior to its use, many of my test cases, which were based on the product requirement, read something like "Verify that a user can login", followed by a series of steps to verify this condition (e.g., enter a valid username, enter a valid password, etc).  With the use of Cucumber, I am forced to consider a pre-condition and action before performing any sort of verification, which, in my humble opinion, is always a good thing -- keeps me asking questions.  Where I think things get a bit tricky with Cucumber is step reuse.  This can occur, at least it has for me, if one creates a number of scenarios in the imperative style.  So, having the option to use a "Macro" feature that abstracts a series of steps to a higher level would be a nice touch for Cucumber.

 

[Richard Paul]

I'm adverse to being able to define macros in a feature file, the
feature files should act as living documentation of the system. Having
a bunch of macros in this documentation seems to create more cruft
that the product owner/stakeholders will have to ignore. Anything that
make the already difficult task of collaboration harder should be
avoided if possible.

Having the macros defined globally would alleviate this problem but it
does seem more complicated. To find the definition of a step you have
to check both the ruby code and the macros. To be able to use a macro,
you still need to write the ruby code for the nested steps... To my
developer eyes macros don't seem to add much value.

Cheers,
Richard

On Jun 8, 4:04 pm, Matt Wynne <m...@mattwynne.net> wrote:
> @Pekka, I'm trying to steal a feature from Robot, I thought you might be interested :)
>
> On 3 Jun 2011, at 15:27, Mike Sassak wrote:
>
>
>
>
>
>
>
>
>

> > On Fri, Jun 3, 2011 at 6:00 AM, Andrew Premdas <aprem...@gmail.com> wrote:

> Here's the (relatively opaque) Robot framework documentation:http://robotframework.googlecode.com/svn/tags/robotframework-2.5.6/do...