Gherkin 3 - the road to a simpler Cucumber

[aslak hellesoy]

Hi all!

I want to kick off Gherkin 3 - a new cross-platform Gherkin parser that will address most of the complexity and bugs in the various Cucumber implementations. I am looking for volunteers to help us implement this in C/Ruby, Java, JavaScript, C# and C/Python (for the various Cucumber implementations out there). It's a fun challenge involving parser generators (Lex/Yacc - Flex/Bison) and various programming languages.

The reason I want to do this is that I have come to the realisation that if the Gherkin library was different, then a lot of the complexity and bugs in Cucumber would go away. (Cucumber's implementation of Background, Scenario Outline, Hooks, JSON reports etc is complicated, and has some nasty bugs that are very hard to fix with the current Gherkin design).

I think Gherkin 3 can solve that. Over the past couple of months I have been experimenting with a new cross-platform parser design that builds an AST [1]. This AST can then be traversed by a visitor to interpret it or transform it. You can find a proof of concept of this design in the bool project [2]. There is nothing novel here, except perhaps for demonstrating how this can be done in a cross-platform fashion.

Gherkin 3 would generate an AST for each Feature file. This AST would then be transformed to a simpler structure that is easier for Cucumber to deal with.

Background steps would be inlined, Scenario Outlines and their Examples tables would be expanded and replaced with regular Scenarios. You could think of this as a sort of compilation phase that happens in Gherkin before handing data over to Cucumber.

The format passed onto Cucumber would just be a list of Units, and each Unit would consist of zero or more Fragments. Typically, a Scenario or Scenario Outline Examples row would become a Unit, and Steps would become a Fragment.

Cucumber would add Fragments to each Unit for Before/After hooks and then execute the whole thing. Here is an example Unit:

    Unit

    +--Fragment (may come from a Before)

    +--Fragment (may come from a Given)

    +--Fragment (may come from a When)

    +--Fragment (may come from a Then)

    +--Fragment (may come from an After)

This structure is much easier to deal with for Cucumber, as it doesn't need to know about the structure of the source. This would allow us to refactor and simplify Cucumber drastically. For Cucumber-JVM's JUnit integration, a Unit would become a JUnit test.

The Gherkin grammar would be the same as now. Users wouldn't have to modify their feature files at all, but they would notice a difference in the reporting. Reports would follow the Unit/Fragment structure instead of the Gherkin source structure.

Line numbers in the reports would still refer to the original line from the gherkin source, so that it's easy to trace the output back to the source.

Interested in helping out? The first step would be to write a working Lex/Yacc [3] grammar for Gherkin (English only) and use a similar project structure to the bool project. I'll set up a new gherkin3 github project shortly.

Cheers,

Aslak

[1] http://en.wikipedia.org/wiki/Abstract_syntax_tree

[2] https://github.com/cucumber/bool

[3] Nobody uses Lex/Yacc anymore. We'll use Jison for JavaScript, Flex/Bison for C/MRI and C/Python, JFlex/Bison for JRuby/Java and GPPG for C#.

[Ilan Pillemer]

Sent from my iPhone

On 31 Jan 2013, at 11:35 PM, aslak hellesoy <aslak.h...@gmail.com> wrote:

> Hi all!
>
> I want to kick off Gherkin 3 - a new cross-platform Gherkin parser that will address most of the complexity and bugs in the various Cucumber implementations. I am looking for volunteers to help us implement this in C/Ruby, Java, JavaScript, C# and C/Python (for the various Cucumber implementations out there).

I would enjoy the challenge and if you want me I would like to volunteer to assist with this!

This means I would like to be involved with this and the eclipse plugin.

--
Ilan

[Andrew Premdas]

Gherkin 3 sounds great, but the line "Reports would follow the Unit/Fragment structure instead of the Gherkin source structure" worries me a little. Could this be detrimental to Cucumber's current ability to let you know what language in your feature file is causing you problems. Cucumber's current ability to let its output follow the original Gherkin source is rather important for situations where you want to use Cucumber to document what your application is currently doing. If you could only have a more granular output, the difference between the text of the source and output would be significant, particularly for people who make extensive use of tables and example groups.

With this in mind I think it would be good to have a requirement that says that it should be easy/feasible to construct existing reports from the new structure as well as more granular ones.

All best

Andrew

--
-- Rules --
 
1) Please prefix the subject with [Ruby], [JVM] or [JS].
2) Please use interleaved answers http://en.wikipedia.org/wiki/Posting_style#Interleaved_style
3) If you have a question, don't reply to an existing message. Start a new topic instead.
 
You received this message because you are subscribed to the Google Groups Cukes group. To post to this group, send email to cu...@googlegroups.com. To unsubscribe from this group, send email to cukes+un...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/cukes?hl=en
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

------------------------

Andrew Premdas

blog.andrew.premdas.org

[aslak hellesoy]

On Fri, Feb 1, 2013 at 1:17 PM, Andrew Premdas <apre...@gmail.com> wrote:

Gherkin 3 sounds great, but the line "Reports would follow the Unit/Fragment structure instead of the Gherkin source structure" worries me a little. Could this be detrimental to Cucumber's current ability to let you know what language in your feature file is causing you problems. Cucumber's current ability to let its output follow the original Gherkin source is rather important for situations where you want to use Cucumber to document what your application is currently doing. If you could only have a more granular output, the difference between the text of the source and output would be significant, particularly for people who make extensive use of tables and example groups.

Good points. Since the results will contain back-references to the original source (aka source map), a report that looks like the original source with colours and extra info can be generated simply by reading the original source and outputting it again with the results.

This is something we can add later.

Aslak

[Rex Hoffman]

Aslak, Are the kinds of bugs that arise gherkin lexing bugs or cucumber-jvm execution or reporting bugs?  I have no knowledge of the difficulties with lexing errors, but think I can speak a bit to cucumber reporting and execution errors....

I am interested in the evolution of gherkin, but I'm not sure I'm convinced that improved stability and loose coupling couldn't be achieved with the existing gherkin implementation  (At least in cucumber-jvm, ruby, js and others I, again, lack knowledge).

While a cleaner implementation of the gherkin parser (and decoupling it from the reporting API's in gherkin itself) would be helpful by requiring minimal effort conversion to work with junit I don't think its necessary to clean cucumber-jvm itself.   In fact, working to loosely couple cucumber-jvm to gherkin (with one quick translator layer) could make gherkin-3 trivially easy to integrate with cucumber-jvm.

For instance, I'm sure we can translate all the output of the current gherkin parser to a Scenarios that that are treated as tests in junit.... with their reporter callbacks to generate whatever kind of report we wish, and in a thread safe way.  I did this while testing out testng integration, before being stopped by reporting issues.

I'd be happy to work on decoupling cucumber-jvm from gherkin, if you have interest in pulling this kind of work in.... but I'm not sure I see the value of re-implementing the lexer, not that I'm a fan of ragel...

My ideal view of the world would be 

parser (maps of features/scenarios/etc...) -> interpreter to scenarios (builds scenarios) -> optional manipulators (multiplex tests, contextualize them, spring, pico, etc)  -> inject listeners for individual reporting (to be aggregated/interpreted down stream) ->converter (scenarios - unit tests) --> executor (works in parallel?) --> report aggregator

My 2 cents.

[Massimo Manca]

Excuse me if I entered this discussion so late, I would like to show you what in practice I am doing to test embedded projects mainly in C and C++ and what I had introduced to write clear requirements using Gherkin-like syntax. I know that there are some constructs not present in Gherkin so considering that Aslak is thinking to change a little bit under the hood Gherkin I would like to know if other engineers should confirm the usefulness of my approach (and also every criticism about it) and eventually ask to implement in the Gherkin 3 release.

This is an example written in Gherkin-style:

# Rel. 0.2 MP & MM 27/11/2012 
# Project Test
# file First.feature
# Brief descriptive text
#
Feature: Device startup
	 To be able to use the system... bla bla bla
         and then... bla bla
	 I must... bla bla bla

	@beforeEveryScenario	
	Background:
		Given	the device is installed in the right place
		And 	is connected to the power source
		And 	is connected to the communication media
		
	@subscenario
	@success
	Scenario: Device startup procedure completed
		Given User powered up the device
		When  Device started and completed self test procedure without errors
		Then  Ready.Led = YELLOW.ON
		And   Device.Status = StartUp

	@subscenario
	@fail
	Scenario Outline: Device startup procedure failed
		Given Device startup procedure completed
		When  Device finished the procedure returning <error>
		Then  Device.Led.color  = <color>
		And   Device.Led.status = <light>
		And   Device.Status = <status>
		And   User switches OFF the power source switch
		And   User repeats the device startup procedure or calls the technical service
	
		Examples: Errors
			| error        | color | light | status  |   comment or reference   |
			| NO POWER     |   -   |   -   |    -    |                          |
			| ROM CRC      |  RED  | FIXED | ROMFLT  |                          |
			| RAM          |  RED  | BLINK | RAMFLT  |                          |
			| RF           | YELL. | FIXED | RFFLT   |                          |
			| NONE RFID    | ORAN. | FIXED | RFIDFLT |                          |
#
# Here I place suggestions, observations and criticisms emerged by discussion with the team
# and with the customer and every other note should be useful to modify the scenario
# it is a temporary block that will be deleted after the modified implementation will be ready
# tested, confirmed and approved (the history is preserved by the source code version control system
#

		
	@subscenario
	@success
	Scenario: User is known
		Given User inserted the user's card in the slot
		And   the card was previously written with data for a normal user
		When  Device knows the user
		Then  Ready.Led = GREEN.ON
		And   Device.Status = Ready

	@subscenario
	@fail
	Scenario: User isn't known
		Given User inserted the user's card in the slot
		And   the card was previously written with data for a wrong user
		When  Device doesn't know the user
		Then  Ready.Led = RED.ON
#
# Here I place suggestions, observations and criticisms emerged by discussion with the team
# and with the customer and every other note should be useful to modify the scenario
# it is a temporary block that will be deleted after the modified implementation will be ready
# tested, confirmed and approved (the history is preserved by the source code version control system
#

		
	@mainscenario
	@success
	Scenario: User logon and device ready to work
		Given Device startup procedure completed
		And   User is known
		When  Device receives the 1st data string from GPS
		Then  Device updates geographic position
		And   searches for the nearest server
		And   connects to the sensor network

	@mainscenario
	@fail
	Scenario: Device not ready to work
		Given Background conditions satisfied
		When  Device startup procedure failed
		Then  Device isn't operational


	@mainscenario
	@fail
	Scenario: Device doesn't recognize the user
		Given Device startup procedure completed
		When  Device doesn't recognize the user
		Then  Device isn't operational


#
# Here I place suggestions, observations and criticisms emerged by discussion with the team
# and with the customer and every other note should be useful to modify the scenario
# it is a temporary block that will be deleted after the modified implementation will be ready
# tested, confirmed and approved (the history is preserved by the source code version control system
#

#END OF THE EXAMPLE


This example is the result of an initial discussion where was realized only the successful @mainscenario and then it was detailed
better with @subscenarios to translate the initial idea to executable test steps that are sufficient to
test the real application.


I know that there is some skepticism about this way to write BDD style testing requirements but in my personal experience it
is very well suited in the embedded field to simply automate acceptance tests (also if it needs some improvements).
	
As you can see it is not so simple to define a main scenario to collect every sub scenario inside.
The best strategy I implemented instead to implement the 3 @mainscenario is to use a Scenario Outline as this: 

	@mainscenario
	Scenario Outline: Device startup 
		Given Device startup procedure completed
		When  Device startup procedure completed returns < startError >
		And   User recognition completed returns < userError >
		Then  Device.Led.color  = <color>
		And   Device.Led.status = <light>
		And   Device.Status = <status>
	
		Examples: Errors
			| startError   | userError | color | light | status  |   comment or reference   |
			| NO POWER     |     -     |   -   |   -   |    -    |                          |
			| ROM CRC      |     -     |  RED  | FIXED | ROMFLT  |                          |
			| RAM          |     -     |  RED  | BLINK | RAMFLT  |                          |
			| RF           |     -     | YELL. | FIXED | RFFLT   |                          |
			| NONE RFID    |     -     | ORAN. | FIXED | RFIDFLT |                          |
...
			| NONE ERROR   |  UNKNOWN  | RED   | FIXED | RFIDFLT |                          |
			| NONE ERROR   |   KNOWN   | GREEN | FIXED |  READY  |                          |

as you can understand in this case is not possible that the system could begin to test the user if the startup failed
but in many other situations I found it is possible so the table grows and is not so simple to maintain it.

So I think Gherkin could support a model where a main scenario should be the result of the composition of
some sub scenarios and sub scenarios should be seen as steps of it so that I could re assume my idea in this way:

	MainScenario: Device startup #the same title of the feature
		Background
		Step(Device startup procedure, @success, @fail) #title and a scenario list with every element comma separated
		Step(User logon, @success, @fail)
		After

we could identify the scenarios with a tag next to @scenario and @subscenario or we can identify them by the title
assuming the same title and @success or @fail tags to identify the purpose of the scenario.
In my opinion this should simplify the translation of the scenarios to executable code and could help to decompose
scenarios in a more detailed scenarios that should be useful in some situation to produce an executable scenario
more near to the implementation to improve the testability.


Another thing that should be very useful is to change the meaning of Background to Before and After, so that should be possible
to have a Before that is common to every scenario in a feature (I mean a block that should be true before every scenario) and
a Before that should be specific for a particular scenario.
The same concept should be used for After.

[Massimo Manca]

Something went wrong with the original post

-------- Messaggio originale --------

Oggetto:	Re: [Cucumber] Gherkin 3 - the road to a simpler Cucumber
Data:	Sat, 02 Feb 2013 15:36:32 +0100
Mittente:	Massimo Manca <massimo....@gmail.com>
A:	cu...@googlegroups.com

[aslak hellesoy]

I'm sorry, you lost me. I think I understand that you want to compose scenarios, but I don't understand why and how. Your proposal sounds very non-intuitive.

Did anyone else understand this? 

[Massimo Manca]

Yes, it is a composition. The problem doesn't arise with successful scenarios but comes with not successful scenarios. Normally the successful scenario is just one scenario but there are more to specify what happens on failures. Not having an OR clause means that I have to write a scenario with only one step that fails and this is good because paths are explicated in this way. Also the scenario decomposition in my opinion is a good way to refine a high level scenario initially written in a too simple way. Actually there isn't a dedicated way/construct to call these scenarios and I use the Given clause with a description or with the scenario title (this is good for understanding but I think is more difficult to automate the tests).

In embedded development I need this things that I don't need in web and PC sw development. I should prefer to use a standard way that could be ok in every situation.

[Matt Wynne]

It sounds to me like the OP wants GivenScenario to come back, or something like that.

cheers,

Matt

--
http://mattwynne.net || https://twitter.com/mattwynne || http://the-cucumber-book.com || http://bddkickstart.com || http://www.relishapp.com

[Tim Walker]

OP?

[Massimo Manca]

It is a try to put in one MainScenario every other Scenario needed to realize the Feature. So to test the feature is necessary to execute the main scenario. The 1st Step means that I want to execute the "Device startup procedure" choosing by the successful scenario (indicated with @success) or one of the unsuccessful scenarios (in this case only one tagged with @fail) of course every unsuccessful scenario should have a different tag. Then the tool that reads this file should execute every possible combination of scenarios (only 1 combination will produce the successful scenario for the feature).

My way to write requirements using Gherkin syntax: initially I write the successful scenario at an high level of abstraction that I ask some questions to understand better its business value and its behavior so normally I detail better the scenario or I decompose the scenario in more scenarios. Then when I finished to understand the successful behavior I start to understand what can go wrong and I add more unsuccessful scenarios.
Hypothetically every successful scenario may have 1 or more unsuccessful scenarios so if the feature is not very simple the application should have to realize more then 1 successful scenario and several unsuccessful scenarios to be tested. For these reasons I would like to have 1 main scenario for every Feature that can "call" sequentially the others to test every "execution path" in terms of Scenarios.

[Matt Wynne]

On 5 Feb 2013, at 16:59, Tim Walker <walk...@gmail.com> wrote:

OP?

I think it stands for Original Poster - the person who first asked the question.

[Tim Walker]

Thank you...always good to SAUNA.

Tim
Stay Abreast Of Unknown Acronyms

[Andrew Premdas]

Think of each line in a feature as being a method call. You can always replace any number of lines in a feature with just a single line by

1. Giving a name to the collection of lines you want to call

2. Creating a new method (line in a feature) with that name and then calling it

All this stuff about MainScenario etc. can already be done by Gherkin. The only restriction is that if you try and have the collection of lines and the scenarios that use the method that aggregates the collection of lines in the same file then things get messy. But why would you want to that, when you have a file hierarchy available to organise things.

So (in super rough pseudo ruby/gherkin)

# feature 1

Scenario:

  foo

  bar

  baz

# feature 2

Scenario:

  qux

# steps

def qux

  foo

  bar

  baz

end

[Massimo Manca]

Il 05/02/2013 19:08, Andrew Premdas ha scritto:

Think of each line in a feature as being a method call. You can always replace any number of lines in a feature with just a single line by

1. Giving a name to the collection of lines you want to call

2. Creating a new method (line in a feature) with that name and then calling it

All this stuff about MainScenario etc. can already be done by Gherkin. The only restriction is that if you try and have the collection of lines and the scenarios that use the method that aggregates the collection of lines in the same file then things get messy. But why would you want to that, when you have a file hierarchy available to organise things.

So (in super rough pseudo ruby/gherkin)

is it possible to express in pure Gherkin? If yes may you help me to express it? It is an interesting possibility to me.

# feature 1

Scenario:

  foo

  bar

  baz

are foo, bar and baz steps like Given, When, Then (eventually also AND)?

[Andrew Premdas]

Yes foo, bar and baz are steps

And qux is a step that encapsulates what foo, bar and baz do

All you have to do is understand well enough what the sequence of foo, bar and baz do to name qux correctly. Perhaps an example would clarify. Notice that the 'Serve Tea' step is the equivalent of including the 'Make tea' scenario before the 'Serve afternoon tea' scenario

Feature: Make tea

Scenario: Make tea

  Boil kettle

  Warm pot

  Add tea

  Brew

  Pour

Feature: Afternoon tes

Scenario: Serve afternoon tea

  Serve tea

  Serve sandwiches

  Serve scones

# make_tea_steps.rb

  module TeaSH

    def boil_kettle ...

    def warm_pot ....

     ...

  end

  World TeaSH

  

  When "Boil kettle" do

    boil_kettle

  end

end

#afternoon_tea_steps.rb

  module AfternoonTeaSH

     def serve_tea

        boil_kettle

        warm_pot

        ...

     end

  end

  World AfternoonTeaSH

  When "Serve tea" do

     serve_tea

  end

end

Finally note that this mechanism is far more expressive than the one you are proposing, for example your can easily make the tea later, or make it twice

Scenario: Serve gross out afternoon tea

  Serve sandwiches

  Serve tea

  Serve sandwiches

  Serve scones

  Serve tea

  

[Tim Walker]

Hi Andrew,

To be clear: nested steps are back in your good grace? Or are you
referring to (in this meta example) SUT code being called from within
the step (as opposed to %Q(boil kettle))?

Thank you,

Tim

[Massimo Manca]

yes this for successful scenarios, for unsuccessful scenario I need to write the same changing 1 scenario step with 1 unsuccessful scenarios.
The problem is: how can I write it in Gherkin? Exactly as you did? Until now I understood that I should write scenarios using Given-When-Then steps; am I wrong?
Consider your example: it is done by 5 Serve steps, supposing that 1st step should have 2 error conditions, it will mean that I have to write 3 scenarios (one for the sucessful scenario and 2 for the 2 unsuccessful scenarios); do you think that I should write only 1 Scenario using a table to consider all the successful/unsuccessful conditions?

[Andrew Premdas]

Absolutely not!! Nested steps are the devils spawn ;)

As far as this conversation is concerned though, thats an implementation detail

All best 

Andrew

[Andrew Premdas]

Referring back to my example:

Feature: Afternoon tea

Scenario: No tea

    Given tea cannot be served

     Then offer coffee

At this level of abstraction it might be best  if we don't care why tea cannot be served. AfternoonTea is dealing with a higher level of abstraction than MakingTea. At this level it would be nice not to care why the tea can't be served. However we want to let the reasons for tea failing seep into this level of abstraction we could always have

Given tea cannot be served because there is no tea

 Then offer coffee

Given tea cannot be served because there is no hot water

Then offer lemonade 

 

In general with Cucumber do the happy path, then use the tools you have created to do the sad paths. 

So another example:

Feature: Sign in

  As a registered user
  I want to sign in
  So I can be recognised by the site

  Background:
    Given I am registered as 'Fred'

  Scenario: Sign in
    When I sign in
    Then I should be signed in

  Scenario: Sign in with bad password
    When I sign in with a bad password
    Then I should be signed out


  Scenario: Sign in with bad email
    When I sign in with a bad email
    Then I should be signed out



Here the happy path is 'sign in' and the sad paths are extensions of signing in with bad data

  

[Massimo Manca]

This is a too simple example, I don't need any extension to Gherkin to mange similar cases. When you write a scenario using several And steps potentially you have to write a number of unsuccessful scenarios higher or equal then the number of steps.
This is a simple example to register a user:

Scenario Register a new user
    Given the user is not already registered
    When the user inserts all data correctly
    Then the application saves his data
    And   creates a new username and password.

to implement the Given row you should search your user's database so if you would explicit it you should write a detailed scenario:

Scenario system returns that the user is not already registered
    When a user insert his data
    Then  the system search for them in the database
    And returns that he is an unknown user

so a better original scenario should be

Scenario Register a new user
    Given the user is not already registered
    When the user inserts his data correctly
    Then the system search them in the database
    And   doesn't find them
    And creates a new username and password
    And   saves all the user data in the database

and in my opinion this is better:

Scenario Register a new user
    Given the user is not already registered
    When the user inserts his data correctly
    Then   system returns that the user is not already registered
    And creates a new username and password
    And   saves all the user data in the database

but you know there is no reference to the Scenario that documents "system returns that the user is not already registered" so that should be dangerous to change a scenario title during requirements gathering and normally this means to add a tag or to add a unique label/number inside the scenario title. May be this a solution?

@Register_a_new_user
Scenario Register a new user
    Given the user is not already registered
    When the user inserts all data correctly
    Then the application saves his data
    And   creates a new username and password.

Scenario Register a new user
    Given the user is not already registered
    When the user inserts his data correctly
    Then   system returns that the user is not already registered   [implements(@Register_a_new_user)]
    And creates a new username and password
    And   saves all the user data in the database

My idea and yours are not one better and the other worst, we simply have different need to achieve the same goal: develop an application using behavior driven development way.

One of my goal is that I haven't to see the test code to understand the behavior of the test code, I should just read the Feature and its Scenarios written in Gherkin. Another goal is that I shouldn't (normally) write comments in Gherkin to understand the behavior of the test, (may be that some comments to describe the system are needed).

I can say that when I write a requirement for an embedded application I normally need more "And" in the steps and 3 to 5 times more unsuccessful scenarios then in a PC application. In this way translate requirements to tests and write the code to fulfill the requirements is a lot simpler then using other methodologies. So, considering that Aslak should write a new parser I think should be a good idea to ask to every engineer in the community to send proposals, ask for improvements and so on. Another possibility should be to make possible that a user can extend Gherkin to recognize some user syntax and constructs.

Just to be clear an embedded system must detect different error conditions then a web application or a PC application and we need to write specific requirements to test all error conditions. Unit testing, simulations and mocks are very good to test dangerous conditions difficult or dangerous to test when you aren't sure if your source code isn't correct and they have to be realized in code of course and not in Gherkin.
But until now Gherkin is the best simple syntax I found to describe functional requirements.

And just to be clear the 95% of every microprocessor/microcontroller sold is not used in a PC or general type of calculators but in embedded devices so I think Gherkin should address also embedded applications needs.

[Andrew Premdas]

Your goals are misguided. You said

 

    One of my goal is that I haven't to see the test code to understand the behaviour of the test code, I should just read the Feature and its Scenarios written in Gherkin

This clearly is not achievable. Any Gherkin feature is by definition a higher level abstract representation of what is actually happening. The whole point of Gherkin is to allow people to use the power of natural language to present a representation of what is going on underneath that can convey something that can be usefully understood by the reader without overwhelming them with every aspect of the implementation. You need an implementation to make your Gherkin features work. For each feature there are a large (infinite) number of implementations that can make your features go green, and a subset (still large, still infinite?) of implementations that can actually work with your system and suggest its doing what you want it to do.

This means is no way you can understand the behaviour of the test code from the gherkin. The best you can hope to achieve is to clearly describe what the behaviour is and why is it important. Perhaps this is the reason you feel you have to put so much of the 'how' in your features.

Doing Behaviour Driven Development doesn't mean that you have to express every detail of how something behaves in your features. Quite the opposite. Features were designed to hold the essence of the behaviour, the what and why, not to include the how. You feel that because you are working with embedded systems you have to include much more of the how in your features and that gherkin should change to accomodate this. The way I see things embedded systems are no different from web applications, or web services or api's. They are all things which have behaviour which can be described at various levels of abstraction. In each case gherkin gives you the power to choose what level of abstraction you wish to work at. If you choose wisely you can make development much simpler and allow your features to communicate just enough information to explain what is being done and why, without overwhelming the reader with too much information. 

However if you don't understand how to work with multiple levels of abstraction, or you don't have the language skills to express yourself fluently at different levels of abstraction, or if you think gherkin should explain how the test works then you will find gherkin limiting, and will want to add extra constructs to it. You are not the first person to have asked for these additional constructs in gherkin, and I am sure you will not be the last. Your problems are not unique, your requests are not unique, but your understanding (like every human beings) is unique. Perhaps you can solve your problems best by developing/changing that.

You said: 

  "Just to be clear an embedded system must detect different error conditions then a web application or a PC application and we need to write specific requirements to test all error conditions". 

This seems to be a pretty clear description of exhaustive testing. Cucumber and Gherkin are not suitable tools for this. If you want to do exhaustive testing you should use unit test tools to "test all error conditions". You can't do everything with one tool, and you can't test everything with one test.

Gherkin already addresses the needs of embedded systems. You just have to choose the correct level of abstraction for your particular context. And yes this will mean making compromises on what your features contain, but that is the nature of writing features. Your current features are telling you that you haven't yet found a suitable level of abstraction, but your not listening to them. You don't need to change Gherkin, you just need to change your features.

All best

Andrew

[Massimo Manca]

There is a basic misunderstanding probably because I wasn't able to explain my situation. I try to compare similar procedures in a PC and embedded application. To describe a data entry in a window I think we don't need to specify what buttons the user has to click with a mouse because we can design the window only with buttons needed for that window. In an embedded application frequently you have real buttons/switches that can have multiple meanings so you should specify what button/s the user has to press to perform an action. It doesn't matter how the sw reads the buttons but the sequence matters so that a "When" for a PC application should be:

When user insert rate and currency data

in an embedded device should be:

When user press button "RATE"
And    insert a valid amount using only the numeric and the "." keys
And   select the currency using "<" and ">" keys
And   user confirms data pressing "ENTER"

because also the input sequence should matter.

This means is no way you can understand the behaviour of the test code from the gherkin.

It is not totally true. A Given-When-Then may be translate to an automata (finite state machine).

The best you can hope to achieve is to clearly describe what the behaviour is and why is it important. Perhaps this is the reason you feel you have to put so much of the 'how' in your features.

Doing Behaviour Driven Development doesn't mean that you have to express every detail of how something behaves in your features.

The problem is that the meaning of "detail" is different in different contests so in web and PC application I agree that "details" means what you say but your details in an embedded device aren't details but high/middle level descriptions.

Quite the opposite. Features were designed to hold the essence of the behaviour, the what and why, not to include the how. You feel that because you are working with embedded systems you have to include much more of the how in your features and that gherkin should change to accomodate this. The way I see things embedded systems are no different from web applications, or web services or api's.

No this is not true. Don't think to a mobile phone, think to an ABS ecu, to a thermostat and need to describe the data acquisition and regulation.

They are all things which have behaviour which can be described at various levels of abstraction. In each case gherkin gives you the power to choose what level of abstraction you wish to work at. If you choose wisely you can make development much simpler and allow your features to communicate just enough information to explain what is being done and why, without overwhelming the reader with too much information.

Exactly, if you can use Gherkin at every level you can decide when specify low or high level of details

However if you don't understand how to work with multiple levels of abstraction, or you don't have the language skills to express yourself fluently at different levels of abstraction, or if you think gherkin should explain how the test works then you will find gherkin limiting, and will want to add extra constructs to it. You are not the first person to have asked for these additional constructs in gherkin, and I am sure you will not be the last. Your problems are not unique, your requests are not unique, but your understanding (like every human beings) is unique. Perhaps you can solve your problems best by developing/changing that.

You said: 

  "Just to be clear an embedded system must detect different error conditions then a web application or a PC application and we need to write specific requirements to test all error conditions". 

This seems to be a pretty clear description of exhaustive testing. Cucumber and Gherkin are not suitable tools for this. If you want to do exhaustive testing you should use unit test tools to "test all error conditions". You can't do everything with one tool, and you can't test everything with one test.

Ok... I am on the partie that thinks this: TDD unit testing are good to validate the application units/modules from the point of view of the developer. Instead BDD using Gherkin as ubiquitous language permits to write requirements from the customer/user point of view so to validate the application behavior. In my development cycle I use both and I am pretty happy.

Gherkin already addresses the needs of embedded systems. You just have to choose the correct level of abstraction for your particular context. And yes this will mean making compromises on what your features contain, but that is the nature of writing features. Your current features are telling you that you haven't yet found a suitable level of abstraction, but your not listening to them. You don't need to change Gherkin, you just need to change your features.

In fact it addresses many things but if a lot of engineers ask to have some other functionalities I think they found some problems or aspects that aren't well  addressed so why not talking about a way to modify/extend Gherkin that could be more useful for all.

All best

Andrew

--

------------------------

Andrew Premdas

blog.andrew.premdas.org

[Johannes Fahrenkrug]

Hi Aslak,

I know this is a really old post. I am curious what the roadmap for Gherkin 3 is. Is it even still a thing? I've seen the gherkin3 repo but it doesn't seem very active. Could you shed some light on the status and future of Gherkin? Thank you!

- Johannes

[aslak hellesoy]

On Thu, Feb 26, 2015 at 9:44 PM, Johannes Fahrenkrug <jfahr...@gmail.com> wrote:

Hi Aslak,

I know this is a really old post. I am curious what the roadmap for Gherkin 3 is. Is it even still a thing? I've seen the gherkin3 repo but it doesn't seem very active. Could you shed some light on the status and future of Gherkin? Thank you!

It is indeed a thing. Steve Tooke and I started on the Ruby implementation today, and I also started the JavaScript implementation. The C# implementation (which is the new reference impl) is currently being integrated into SpecFlow. And the Java implementation is getting there too!

We could definitely use some help to move these forward faster - so if you want some fun coding - come and join us! It's essentially a matter of porting the C# code over to Ruby/JavaScript/Java

Aslak

 

--
Posting rules: http://cukes.info/posting-rules.html

---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+un...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Johannes Fahrenkrug]

Hi Aslak, 

Thanks for your quick reply. That's really interesting. So would the first step be to port berp to Ruby/JS/Java? I'm interested in helping out and I'm also interested in a Swift/ObjC port. My time is very limited, though (as everyone else's is too :)).

- Johannes

[aslak hellesoy]

On Fri, Feb 27, 2015 at 8:45 PM, Johannes Fahrenkrug <jfahr...@gmail.com> wrote:

Hi Aslak, 

Thanks for your quick reply. That's really interesting. So would the first step be to port berp to Ruby/JS/Java?

No. Berp is a parser generator. It happens to be implemented in C#, but that is irrelevant since it can output parsers in any language. Berp is used to generate parsers for all Gherkin implementations. It takes a grammar file (gherkin.berp) and a template (gherkin-X.razor) and outputs a parser in language X. The lexer and Ast code is hand-written.

 

I'm interested in helping out and I'm also interested in a Swift/ObjC port. My time is very limited, though (as everyone else's is too :)).

That would be awesome. I guess we need to write up some docs about how to contribute!

Aslak

[Matt Wynne]

On 27 Feb 2015, at 23:32, aslak hellesoy <aslak.h...@gmail.com> wrote:

On Fri, Feb 27, 2015 at 8:45 PM, Johannes Fahrenkrug <jfahr...@gmail.com> wrote:

I'm interested in helping out and I'm also interested in a Swift/ObjC port. My time is very limited, though (as everyone else's is too :)).

That would be awesome.

+1000 <3 <3 <3

cheers,

Matt

---

ma...@mattwynne.net

[Johannes Fahrenkrug]

Hi Aslak,

No. Berp is a parser generator. It happens to be implemented in C#, but that is irrelevant since it can output parsers in any language. Berp is used to generate parsers for all Gherkin implementations. It takes a grammar file (gherkin.berp) and a template (gherkin-X.razor) and outputs a parser in language X. The lexer and Ast code is hand-written.

Ah OK! Got it! So it's a Ragel replacement, got it.

 

 

I'm interested in helping out and I'm also interested in a Swift/ObjC port. My time is very limited, though (as everyone else's is too :)).

That would be awesome. I guess we need to write up some docs about how to contribute!

That, good Sir, would be much appreciated :)

- Johannes