"Please don't use Cucumber"
296 views
Skip to first unread message
Julien Biezemans
Jun 1, 2012, 6:25:26 AM6/1/12
to cu...@googlegroups.com
Joseph Wilk
Jun 1, 2012, 6:56:40 AM6/1/12
to cu...@googlegroups.com
On 01/06/2012 11:25, Julien Biezemans wrote:
Any thoughts/reactions to this? http://jimmycuadra.com/posts/please-don-t-use-cucumber
**First premise is:
" Cucumber has almost no practical benefit over acceptance testing in pure Ruby with Capybara"
Evidence:
"I have never seen them matter or be remotely applicable in the real world"
This statement is relative to the persons experience but it is expressed as generalisation. A single persons experience does not provide sufficient evidence of the stated premise.
"Gherkin is really just glorified comments."
This point is a miss classification. Comments do not have structure or grammar rules and they are not executed. Comments are removed by the compiler.
If you simply write free form comments above Capybara scenarios, you can convey the same high level information about what the test is doing and what the acceptance criteria are "
Correct. You could replicate Cucumber in comments.
"without any of the overhead, maintenance cost, and general technical debt of Cucumber"
Correct, but that does not exclude maintaining costs for keeping the comments up to date, using consistent language etc etc.
but in my experience, developers tend to avoid the test-first approach with Cucumber simply because it's so painful to use
Again a good point and I'm sure that could be the case with some developers. Don't use a tool that you find painful. But again its an extrapolate to generalize this.
**Second premise:
product managers don't really care about Gherkin
Evidence: (there does not seem to be any)
Their time is better spent brainstorming all the various use cases for a feature and communicating this either verbally or in free form text.
Reading (and especially writing) Gherkin is a waste of their time, because Gherkin is not English.
Maybe they could better use their time but that does not provide any evidence of the premise.
**Third premise:
Gherkin is not English.
Correct Gerkin is not pure, free flowing English, it is a subset of English.
It's extremely inflexible Ruby disguised as English.
Technically incorrect, Gherkin has no tie to Ruby (and in fact uses a python syntax for multiline strings)
The more naturally it reads, the more difficult it is to translate it into reusable code via step definitions.
A good point, this is a challenge a lot of people face and its a difficult one. I've seen people solve this but that does not discount the point.
**Fourth premise (and his conclusion)
Whenever someone writes a criticism of a particular piece of software, there is always a group of people who respond by saying, "It's just a tool. If it works for you, use it. If it doesn't, don't. While I agree in theory, this is where the effect of the cargo cult becomes real and damaging.
I personally find this one difficult to understand. Tell people to think and evaluate their tools and why they use them feels like the opposite of cargo culting. I would like to hear more about how this leads to cargo culting.
Joe
" Cucumber has almost no practical benefit over acceptance testing in pure Ruby with Capybara"
Evidence:
"I have never seen them matter or be remotely applicable in the real world"
This statement is relative to the persons experience but it is expressed as generalisation. A single persons experience does not provide sufficient evidence of the stated premise.
"Gherkin is really just glorified comments."
This point is a miss classification. Comments do not have structure or grammar rules and they are not executed. Comments are removed by the compiler.
If you simply write free form comments above Capybara scenarios, you can convey the same high level information about what the test is doing and what the acceptance criteria are "
Correct. You could replicate Cucumber in comments.
"without any of the overhead, maintenance cost, and general technical debt of Cucumber"
Correct, but that does not exclude maintaining costs for keeping the comments up to date, using consistent language etc etc.
but in my experience, developers tend to avoid the test-first approach with Cucumber simply because it's so painful to use
Again a good point and I'm sure that could be the case with some developers. Don't use a tool that you find painful. But again its an extrapolate to generalize this.
**Second premise:
product managers don't really care about Gherkin
Evidence: (there does not seem to be any)
Their time is better spent brainstorming all the various use cases for a feature and communicating this either verbally or in free form text.
Reading (and especially writing) Gherkin is a waste of their time, because Gherkin is not English.
Maybe they could better use their time but that does not provide any evidence of the premise.
**Third premise:
Gherkin is not English.
Correct Gerkin is not pure, free flowing English, it is a subset of English.
It's extremely inflexible Ruby disguised as English.
Technically incorrect, Gherkin has no tie to Ruby (and in fact uses a python syntax for multiline strings)
The more naturally it reads, the more difficult it is to translate it into reusable code via step definitions.
A good point, this is a challenge a lot of people face and its a difficult one. I've seen people solve this but that does not discount the point.
**Fourth premise (and his conclusion)
Whenever someone writes a criticism of a particular piece of software, there is always a group of people who respond by saying, "It's just a tool. If it works for you, use it. If it doesn't, don't. While I agree in theory, this is where the effect of the cargo cult becomes real and damaging.
I personally find this one difficult to understand. Tell people to think and evaluate their tools and why they use them feels like the opposite of cargo culting. I would like to hear more about how this leads to cargo culting.
Joe
-- Julien. -- There are two rules: 1) Please prefix the subject with [Ruby], [JVM] or [JS]. This allows people to filter messages. 2) Please use interleaved answers http://en.wikipedia.org/wiki/Posting_style#Interleaved_style You received this message because you are subscribed to the Google Groups Cukes group. To post to this group, send email to cu...@googlegroups.com. To unsubscribe from this group, send email to cukes+un...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/cukes?hl=en
-- Joseph Wilk http://blog.josephwilk.net http://www.songkick.com +44 (0)7812 816431 | http://twitter.com/josephwilk
Joseph Wilk
Jun 1, 2012, 7:17:21 AM6/1/12
to cu...@googlegroups.com
On 01/06/2012 11:25, Julien Biezemans wrote:
> Any thoughts/reactions to this?
So my concluding remarks would be:
> Any thoughts/reactions to this?
* Some people don't see the value in Cucumber.
* Some people do see the value in Cucumber.
And as a guide for people finding pain in Cucumber:
1. Find the reason why its painful.
2. Is the problem the tool or something else.
3. If its the tool, try something else.
4. Is the pain better. If yes great! If no see 2.
Joe
>
> http://jimmycuadra.com/posts/please-don-t-use-cucumber
Matt Wynne
Jun 1, 2012, 10:46:55 AM6/1/12
to cu...@googlegroups.com
On 1 Jun 2012, at 11:25, Julien Biezemans wrote:
Same old same old.
Is this post written someone important that lots of people listen to? I don't know who he is, but then I don't get out much.
cheers,
Matt
--
Freelance programmer & coach
Author, http://pragprog.com/book/hwcuc/the-cucumber-book
Founder, http://www.relishapp.com/
Twitter, https://twitter.com/mattwynne
Freelance programmer & coach
Author, http://pragprog.com/book/hwcuc/the-cucumber-book
Founder, http://www.relishapp.com/
Twitter, https://twitter.com/mattwynne
Stephen Abrams
Jun 1, 2012, 12:22:42 PM6/1/12
to cu...@googlegroups.com
On Fri, Jun 1, 2012 at 6:25 AM, Julien Biezemans <j...@jbpros.com> wrote:
> Any thoughts/reactions to this?
>
> http://jimmycuadra.com/posts/please-don-t-use-cucumber
>
About this:
> Any thoughts/reactions to this?
>
> http://jimmycuadra.com/posts/please-don-t-use-cucumber
>
"Reading (and especially writing) Gherkin is a waste of their time,
because Gherkin is not English. It's extremely inflexible Ruby
disguised as English. The more naturally it reads, the more difficult
it is to translate it into reusable code via step definitions."
Technically, Gherkin in not English or any other spoken language. Its
a constraint placed upon any spoken language. Given/When/Then (a
subset of Gherkin) require a stakeholder to express a use case as an
_action_ within an explicit _context_ that has a particular _result_.
Its drives specifications just as TDD drives development, in that by
working with these constraints (which are how most of our software
test frameworks are setup), we discover more about what the
specification should be.
I don't believe that the constraints are due primarily to the
challenges of implementing a domain language for specifications (DSL),
be it in Ruby or any other language. Rather, it is a set of
constraints applied in the same way that REST is applied to the web:
by limiting ourselves in how we speak about features, we end up with a
practical approach to designing, refactoring, communicating, and
(let's not forget) testing them.
Even as an engineer working with specifications that originally came
to me in this format, I personally have found that re-expressing them
in this context has been worthwhile (although challenging and time
consuming), just as I have with TDD.
--
Steve
Stephen Abrams
Jun 1, 2012, 12:24:58 PM6/1/12
to cu...@googlegroups.com
specifications that originally came
to me NOT in this format"
--
Steve
Massimo Manca
Jun 1, 2012, 12:47:30 PM6/1/12
to cu...@googlegroups.com
Il 01/06/2012 18:22, Stephen Abrams ha scritto:
> On Fri, Jun 1, 2012 at 6:25 AM, Julien Biezemans <j...@jbpros.com> wrote:
>> Any thoughts/reactions to this?
>>
>> http://jimmycuadra.com/posts/please-don-t-use-cucumber
>>
> About this:
> "Reading (and especially writing) Gherkin is a waste of their time,
> because Gherkin is not English. It's extremely inflexible Ruby
> disguised as English. The more naturally it reads, the more difficult
> it is to translate it into reusable code via step definitions."
>
> Technically, Gherkin in not English or any other spoken language. Its
> a constraint placed upon any spoken language. Given/When/Then (a
> subset of Gherkin) require a stakeholder to express a use case as an
> _action_ within an explicit _context_ that has a particular _result_.
> Its drives specifications just as TDD drives development, in that by
> working with these constraints (which are how most of our software
> test frameworks are setup), we discover more about what the
> specification should be.
The base idea should be to use a user friendly language to express
> On Fri, Jun 1, 2012 at 6:25 AM, Julien Biezemans <j...@jbpros.com> wrote:
>> Any thoughts/reactions to this?
>>
>> http://jimmycuadra.com/posts/please-don-t-use-cucumber
>>
> About this:
> "Reading (and especially writing) Gherkin is a waste of their time,
> because Gherkin is not English. It's extremely inflexible Ruby
> disguised as English. The more naturally it reads, the more difficult
> it is to translate it into reusable code via step definitions."
>
> Technically, Gherkin in not English or any other spoken language. Its
> a constraint placed upon any spoken language. Given/When/Then (a
> subset of Gherkin) require a stakeholder to express a use case as an
> _action_ within an explicit _context_ that has a particular _result_.
> Its drives specifications just as TDD drives development, in that by
> working with these constraints (which are how most of our software
> test frameworks are setup), we discover more about what the
> specification should be.
requirements; it should be understandable for the customer (without any
learning phase) and for the development team. Also it has to be unambiguous.
So, using English, German or every other language in a free format it is
very difficult to achieve both goals. For the same reason we have
programming languages: we need a formal syntax.
I think that Gherkin is quite good with some exceptions as the table
format to express scenario outlines that is more developer oriented.
And in my opinion one of the best documentations I see about Gherkin is
that on Behat (php framework for bdd).
Tim Walker
Jun 1, 2012, 12:54:06 PM6/1/12
to cu...@googlegroups.com
We are on a journey to find the best way to develop software. I've been at it since 1976 and like The Big Muddy, the way we do it just keeps moving along. Executable Specification is a concept that I personally believe strongly in: there is no reason for there to be a difference between a requirement and a test that proves it. Further, I feel, it is the very salvation for agile and lean software development.
Cargo cult? Maybe, but certainly not for me. I developed and delivered the class "Executable Requirements with FitNesse: to many, many teams small and very large. It was while teaching a class in North Carolina that the architect asked me if I'd seen what the Ruby on Rails guys were doing (the state of the art in FitNesse was the Domain Fixture) with Story Runner. I was very intrigued by the thought: What if you could simply execute a User Story directly? In terms of "maximizing the amount of work not done", this is huge: No duplicated or out of date requirements, tests, documentation and code (to say nothing of specificity of examples, know requirements are met - and stay met). I wanted this badly: so, I learned Ruby on Rails, went to the pragmatic camps, developed some apps and learned the pedigree of Cucumber, FIT, JBehave, RBehave, Story Runner. A long history of excellence.
Then, four years ago or so I decided to leave education and coaching on the opportunity to apply BDD to a greenfield, mission critical, application. I've lived, breathed and suffered the worst of it using Cucumber. It's been extremely hard and has taken a long time but, when we see some problem, like a business rule that is not documented, a mis-understood requirement, managing manual testing with automated testing, defining precise acceptance and many, many other things too numerous to name the answer is always "this is solved by Cucumber, we just need to do it better". So the BA's and product owners ARE getting up to speed on Gherkin and Cucumber. Manual testers ARE generating test cases in Cucumber and no longer in a separate system. Test cases that are checked right in along with the code that it tests.
Something that this article completely fails to mention is the use of Cucumber for non-capybara based acceptance testing. Capybara is just one part: Testing through the UI. Keeping in mind the testing pyramid, and how *vital* it is to understand: you do not want most of you business rules tested through the UI! I feel this article missed that pretty important and basic point.
Saying that Gherkin is just comments is baffling to me and seems to ignore completely the power inherent in the approaches and the speed and accuracy we get by the reuse of tested steps, especially around building up of a known state. Stating that Gherkin is "executable comments" would be closer. Still makes no sense. Comments go out of date when the code changes. A class of problem removed by Gherkin.
We can (and should) use Rspec (keeping in mind that there are RSpec haters too!), and we should, but that is a developer facing language and does not serve the needs of testing in Quadrant 2 and best used as part of the outside in approach.As a way of driving that last point home I'll offer only this link: https://www.relishapp.com/rspec/rspec-mocks/docs Please do read between the lines.
I could go on and on like this, mostly about critical and subtle points and second-orders the author just seems to ignore. For example, I am a huge fan of Eric Evans and managing complexity in our projects by addressing a consistency or our domain and the language that expresses it and feel Cucumber tackles that head on. I do not understand why comments in Capybara are better for testing in quadrant 2 or why this is preferable to Gherkin for a product owner or business analyst? Just makes no sense.
About the only thing I agree with the author on is that this is hard and that developers balk at ATDD driven by Cucumber or FIT but, as I have seen over and over and over, this is only because (in my opinion) they have not broken through to that place where the lights go on and not seeing the benefits to people outside of development.
Just my opinion.
Tim
On Fri, Jun 1, 2012 at 5:17 AM, Joseph Wilk <j...@josephwilk.net> wrote:
On 01/06/2012 11:25, Julien Biezemans wrote:So my concluding remarks would be:
Any thoughts/reactions to this?
* Some people don't see the value in Cucumber.
* Some people do see the value in Cucumber.
And as a guide for people finding pain in Cucumber:
1. Find the reason why its painful.
2. Is the problem the tool or something else.
3. If its the tool, try something else.
4. Is the pain better. If yes great! If no see 2.
Joe
http://jimmycuadra.com/posts/please-don-t-use-cucumber
-- Julien.
-- There are two rules:
1) Please prefix the subject with [Ruby], [JVM] or [JS]. This allows people to filter messages.
2) Please use interleaved answers http://en.wikipedia.org/wiki/Posting_style#Interleaved_style
You received this message because you are subscribed to the Google Groups Cukes group. To post to this group, send email to cu...@googlegroups.com. To unsubscribe from this group, send email to cukes+unsubscribe@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/cukes?hl=en
-- There are two rules:
1) Please prefix the subject with [Ruby], [JVM] or [JS]. This allows people to filter messages.
2) Please use interleaved answers http://en.wikipedia.org/wiki/Posting_style#Interleaved_style
You received this message because you are subscribed to the Google Groups Cukes group. To post to this group, send email to cu...@googlegroups.com. To unsubscribe from this group, send email to cukes+unsubscribe@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/cukes?hl=en
Rob Crawford
Jun 1, 2012, 1:02:35 PM6/1/12
to cu...@googlegroups.com
On Fri, Jun 1, 2012 at 12:54 PM, Tim Walker <walk...@gmail.com> wrote:
*snip*
Something that this article completely fails to mention is the use of Cucumber for non-capybara based acceptance testing. Capybara is just one part: Testing through the UI. Keeping in mind the testing pyramid, and how *vital* it is to understand: you do not want most of you business rules tested through the UI! I feel this article missed that pretty important and basic point.
I'm working on an event-driven, rules-based system and we're looking to use the same Cucumber scenarios to test our rules at the unit level (calling the rules engine directly from the test harness)
*and* at the integration level (the test harness generates events and submits them as if they came from the field). Organizational restraints have kept us from getting user involvement in the scenarios (which are simple at this point), but I'd feel comfortable having them review them.
I don't know of another technology that lets us do all that, but it could be I'm just ignorant.
George Dinwiddie
Jun 1, 2012, 1:39:37 PM6/1/12
to cu...@googlegroups.com
Tim,
On 6/1/12 12:54 PM, Tim Walker wrote:
> We are on a journey to find the best way to develop software. I've been
> at it since 1976 and like The Big Muddy, the way we do it just keeps
> moving along. Executable Specification is a concept that I personally
> believe strongly in: there is no reason for there to be a difference
> between a requirement and a test that proves it. Further, I feel, it is
> the very salvation for agile and lean software development.
>
> Cargo cult? Maybe, but certainly not for me. I developed and delivered
> the class "Executable Requirements with FitNesse: to many, many teams
> small and very large. It was while teaching a class in North Carolina
> that the architect asked me if I'd seen what the Ruby on Rails guys were
> doing (the state of the art in FitNesse was the Domain Fixture) with
> Story Runner. I was very intrigued by the thought: *What if you could
> simply execute a User Story directly?* In terms of "maximizing the
> rules tested through the UI!* I feel this article missed that pretty
- George
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------
On 6/1/12 12:54 PM, Tim Walker wrote:
> We are on a journey to find the best way to develop software. I've been
> at it since 1976 and like The Big Muddy, the way we do it just keeps
> moving along. Executable Specification is a concept that I personally
> believe strongly in: there is no reason for there to be a difference
> between a requirement and a test that proves it. Further, I feel, it is
> the very salvation for agile and lean software development.
>
> Cargo cult? Maybe, but certainly not for me. I developed and delivered
> the class "Executable Requirements with FitNesse: to many, many teams
> small and very large. It was while teaching a class in North Carolina
> that the architect asked me if I'd seen what the Ruby on Rails guys were
> doing (the state of the art in FitNesse was the Domain Fixture) with
> simply execute a User Story directly?* In terms of "maximizing the
> amount of work not done", this is huge: No duplicated or out of date
> requirements, tests, documentation and code (to say nothing of
> specificity of examples, know requirements are met - and stay met). I
> wanted this badly: so, I learned Ruby on Rails, went to the pragmatic
> camps, developed some apps and learned the pedigree of Cucumber, FIT,
> JBehave, RBehave, Story Runner. A long history of excellence.
>
> Then, four years ago or so I decided to leave education and coaching on
> the opportunity to apply BDD to a greenfield, mission critical,
> application. I've lived, breathed and suffered the worst of it using
> Cucumber. It's been /extremely/ hard and has taken a long time but, when
> requirements, tests, documentation and code (to say nothing of
> specificity of examples, know requirements are met - and stay met). I
> wanted this badly: so, I learned Ruby on Rails, went to the pragmatic
> camps, developed some apps and learned the pedigree of Cucumber, FIT,
> JBehave, RBehave, Story Runner. A long history of excellence.
>
> Then, four years ago or so I decided to leave education and coaching on
> the opportunity to apply BDD to a greenfield, mission critical,
> application. I've lived, breathed and suffered the worst of it using
> we see some problem, like a business rule that is not documented, a
> mis-understood requirement, managing manual testing with automated
> testing, defining precise acceptance and many, many other things too
> numerous to name the answer is always "this is solved by Cucumber, we
> just need to do it better". So the BA's and product owners ARE getting
> up to speed on Gherkin and Cucumber. Manual testers ARE generating test
> cases in Cucumber and no longer in a separate system. Test cases that
> are checked right in along with the code that it tests.
>
> Something that this article completely fails to mention is the use of
> Cucumber for non-capybara based acceptance testing. Capybara is just one
> part: Testing through the UI. Keeping in mind the testing pyramid, and
> how *vital* it is to understand: *you do not want most of you business
> mis-understood requirement, managing manual testing with automated
> testing, defining precise acceptance and many, many other things too
> numerous to name the answer is always "this is solved by Cucumber, we
> just need to do it better". So the BA's and product owners ARE getting
> up to speed on Gherkin and Cucumber. Manual testers ARE generating test
> cases in Cucumber and no longer in a separate system. Test cases that
> are checked right in along with the code that it tests.
>
> Something that this article completely fails to mention is the use of
> Cucumber for non-capybara based acceptance testing. Capybara is just one
> part: Testing through the UI. Keeping in mind the testing pyramid, and
> rules tested through the UI!* I feel this article missed that pretty
> important and basic point.
>
> Saying that Gherkin is just comments is baffling to me and seems to
> ignore completely the power inherent in the approaches and the speed and
> accuracy we get by the reuse of tested steps, especially around building
> up of a known state. Stating that Gherkin is "executable comments" would
> be closer. Still makes no sense. Comments go out of date when the code
> changes. A class of problem removed by Gherkin.
>
> We can (and should) use Rspec (keeping in mind that there are RSpec
> haters too!), and we should, but that is a developer facing language and
> does not serve the needs of testing in Quadrant 2 and best used as part
> of the outside in approach.As a way of /driving that last point home/
>
> Saying that Gherkin is just comments is baffling to me and seems to
> ignore completely the power inherent in the approaches and the speed and
> accuracy we get by the reuse of tested steps, especially around building
> up of a known state. Stating that Gherkin is "executable comments" would
> be closer. Still makes no sense. Comments go out of date when the code
> changes. A class of problem removed by Gherkin.
>
> We can (and should) use Rspec (keeping in mind that there are RSpec
> haters too!), and we should, but that is a developer facing language and
> does not serve the needs of testing in Quadrant 2 and best used as part
> I'll offer only this link:
> https://www.relishapp.com/rspec/rspec-mocks/docs Please do read between
> the lines.
>
> I could go on and on like this, mostly about critical and subtle points
> and second-orders the author just seems to ignore. For example, I am a
> huge fan of Eric Evans and managing complexity in our projects by
> addressing a consistency or our domain and the language that expresses
> it and feel Cucumber tackles that head on. I do not understand why
> comments in Capybara are better for testing in quadrant 2 or why this is
> preferable to Gherkin for a product owner or business analyst? Just
> makes no sense.
>
> About the only thing I agree with the author on is that this is hard and
> that developers balk at ATDD driven by Cucumber or FIT but, as I have
> seen over and over and over, this is only because (in my opinion) they
> have not broken through to that place where the lights go on and not
> seeing the benefits to people outside of development.
>
> Just my opinion.
What a great post! This should be published somewhere.
> https://www.relishapp.com/rspec/rspec-mocks/docs Please do read between
> the lines.
>
> I could go on and on like this, mostly about critical and subtle points
> and second-orders the author just seems to ignore. For example, I am a
> huge fan of Eric Evans and managing complexity in our projects by
> addressing a consistency or our domain and the language that expresses
> it and feel Cucumber tackles that head on. I do not understand why
> comments in Capybara are better for testing in quadrant 2 or why this is
> preferable to Gherkin for a product owner or business analyst? Just
> makes no sense.
>
> About the only thing I agree with the author on is that this is hard and
> that developers balk at ATDD driven by Cucumber or FIT but, as I have
> seen over and over and over, this is only because (in my opinion) they
> have not broken through to that place where the lights go on and not
> seeing the benefits to people outside of development.
>
> Just my opinion.
- George
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------
Reply all
Reply to author
Forward
0 new messages