GWTW
George Dinwiddie
generate XML test files for an existing tool that tests an asynchronous
web service. This tool waits for the response from the system under
test, and then evaluates XPATH expectations contained in the original
request (which are returned unchanged in the response). Our "Then"
steps to insert expectations into the XML.
But we've ended up with a pattern like
Given ...
When ...
Then we expect ...
When the request is processed
The final "When" step writes the file that gets submitted by the
existing tool.
Comments?
- George
--
Dec. 14 - Agile Richmond in Glen Allen, VA
http://georgedinwiddie.eventbrite.com/
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------
David Chelimsky
On Dec 8, 2010, at 3:28 PM, George Dinwiddie wrote:
> Ran into an interesting situation, today. We're using cucumber to generate XML test files for an existing tool that tests an asynchronous web service. This tool waits for the response from the system under test, and then evaluates XPATH expectations contained in the original request (which are returned unchanged in the response). Our "Then" steps to insert expectations into the XML.
>
> But we've ended up with a pattern like
>
> Given ...
> When ...
> Then we expect ...
> When the request is processed
>
> The final "When" step writes the file that gets submitted by the existing tool.
>
> Comments?
Better not let the AAA crowd see this :)
>
> - George
>
> --
> Dec. 14 - Agile Richmond in Glen Allen, VA
> http://georgedinwiddie.eventbrite.com/
> ----------------------------------------------------------------------
> * George Dinwiddie * http://blog.gdinwiddie.com
> Software Development http://www.idiacomputing.com
> Consultant and Coach http://www.agilemaryland.org
> ----------------------------------------------------------------------
>
> --
> You received this message because you are subscribed to the Google Groups "Cukes" group.
> To post to this group, send email to cu...@googlegroups.com.
> To unsubscribe from this group, send email to cukes+un...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
>
Mike Sassak
<li...@idiacomputing.com> wrote:
> Ran into an interesting situation, today. We're using cucumber to generate
> XML test files for an existing tool that tests an asynchronous web service.
> This tool waits for the response from the system under test, and then
> evaluates XPATH expectations contained in the original request (which are
> returned unchanged in the response). Our "Then" steps to insert
> expectations into the XML.
>
> But we've ended up with a pattern like
>
> Given ...
> When ...
> Then we expect ...
> When the request is processed
>
> The final "When" step writes the file that gets submitted by the existing
> tool.
>
> Comments?
>
I've done this before, too. Usually I end up rewording the steps to
maintain the GWT order, but that's more because I think maintaining
that consistency lends some weight to the meaning of GWT that Cucumber
itself doesn't. I like the finality of "Then", but if the meaning is
clear I don't think it matters too much. I suspect this might lead to
expressing conditionals, e.g. "When W Then X But when Y Then Z", but
that's the only danger I see, and I think a pretty remote one.
Mike
> - George
>
> --
> Dec. 14 - Agile Richmond in Glen Allen, VA
> http://georgedinwiddie.eventbrite.com/
> ----------------------------------------------------------------------
> * George Dinwiddie * http://blog.gdinwiddie.com
> Software Development http://www.idiacomputing.com
> Consultant and Coach http://www.agilemaryland.org
> ----------------------------------------------------------------------
>
Jonas Nicklas
any way I choose. A lot of the scenarios I write seem to have some
kind of intermediate "plateau" where it makes sense to assert on a
couple of things that happened in between before taking the final
action. For example, simplified:
When I try to create a post with invalid data
Then I should see a form with errors
When I change the fields with errors
And I submit the form again
Then I should see the new post
(Note that I wouldn't write the scenario this declaratively, it's just
an example)
I've yet to see any kind of drawback coming from this.
/Jonas
aslak hellesoy
> Personally I've never had a problem with mixing up the order of GWT in
> any way I choose. A lot of the scenarios I write seem to have some
> kind of intermediate "plateau" where it makes sense to assert on a
> couple of things that happened in between before taking the final
> action. For example, simplified:
>
> When I try to create a post with invalid data
> Then I should see a form with errors
> When I change the fields with errors
> And I submit the form again
> Then I should see the new post
>
> (Note that I wouldn't write the scenario this declaratively, it's just
> an example)
>
> I've yet to see any kind of drawback coming from this.
>
I have called the BDD police. They should be running down your door
any second now. Bet you think that's funny!
George Dinwiddie
Jonas instead of my client and me. :)
On 12/8/10 9:26 PM, Mike Sassak wrote:
> On Wed, Dec 8, 2010 at 3:28 PM, George Dinwiddie
> <li...@idiacomputing.com> wrote:
>> But we've ended up with a pattern like
>>
>> Given ...
>> When ...
>> Then we expect ...
>> When the request is processed
>>
>> The final "When" step writes the file that gets submitted by the existing
>> tool.
>>
>> Comments?
>>
>
> I've done this before, too. Usually I end up rewording the steps to
> maintain the GWT order, but that's more because I think maintaining
> that consistency lends some weight to the meaning of GWT that Cucumber
> itself doesn't. I like the finality of "Then", but if the meaning is
> clear I don't think it matters too much. I suspect this might lead to
> expressing conditionals, e.g. "When W Then X But when Y Then Z", but
> that's the only danger I see, and I think a pretty remote one.
Mike, We initially tried straight GWT (and without the "we expect") but
that gave us
Given ...
When ...
Then <some condition is true>
And the request is processed
I was uncomfortable with this because the condition isn't true until
after the request is processed. It seemed that the scenario was lying
to me about what was happening, and I thought that was going to cause
recurring problems in understanding the test.
Jon Kern
"When the request is processed" sounds a lot like "after the method
returns."
Isn't that implied that you evaluate the "then" expectations after you
have something in your hands (async notwithstanding)?
Or put another way,
Given I set everything up
When I execute the Web Service
Then I expect the ...
I would tend to keep the GWT flavor.
jon
blog: http://technicaldebt.com
twitter: http://twitter.com/JonKernPA
George Dinwiddie said the following on 12/8/10 3:28 PM:
Jon Kern
did <g>.
I would refactor this test... into two.
The error case is simply one scenario... You expect to see the errors in
the form. Done
The case of entering fields and it is a successful post is a second
scenario. Not part of the first one.
Putting the two together is not as effective at communicating what you
are building. And possibly, your scenario below repeats an existing case.
jon
blog: http://technicaldebt.com
twitter: http://twitter.com/JonKernPA
Jonas Nicklas said the following on 12/9/10 2:32 AM:
Jon Kern
jon
blog: http://technicaldebt.com
twitter: http://twitter.com/JonKernPA
aslak hellesoy said the following on 12/9/10 3:01 AM:
George Dinwiddie
On 12/9/10 8:21 AM, Jon Kern wrote:
> Is the last then really needed?
>
> "When the request is processed" sounds a lot like "after the method
> returns."
>
> Isn't that implied that you evaluate the "then" expectations after you
> have something in your hands (async notwithstanding)?
I can't really ignore the asynchronous aspect without making every Then
step a one-time use. I have to have some way to trigger the actual
submission. The expectations have to be expressed before that
submission, though they are evaluated after.
The point of using Cucumber in this case is to express the expectations
in Gherkin rather than XML. The evaluation is still being done by the
other tool because I don't know how to integrate this asynchronous call
into Cuke.
> Or put another way,
> Given I set everything up
> When I execute the Web Service
> Then I expect the ...
There would be no Then if I do this, as my expectations would be in the
Given.
Matt Wynne
On 9 Dec 2010, at 16:11, George Dinwiddie wrote:
> Jon,
>
> On 12/9/10 8:21 AM, Jon Kern wrote:
>> Is the last then really needed?
>>
>> "When the request is processed" sounds a lot like "after the method
>> returns."
>>
>> Isn't that implied that you evaluate the "then" expectations after you
>> have something in your hands (async notwithstanding)?
>
> I can't really ignore the asynchronous aspect without making every Then step a one-time use. I have to have some way to trigger the actual submission. The expectations have to be expressed before that submission, though they are evaluated after.
>
> The point of using Cucumber in this case is to express the expectations in Gherkin rather than XML. The evaluation is still being done by the other tool because I don't know how to integrate this asynchronous call into Cuke.
>
>> Or put another way,
>> Given I set everything up
>> When I execute the Web Service
>> Then I expect the ...
>
> There would be no Then if I do this, as my expectations would be in the Given.
I think I might have misunderstood what's going on in your scenario George, but maybe you could get rid of the When step, and have the request processing just happen implicitly inside the Then step (which also asserts the expectations)?
Anyway as others have said, I don't think I would obsess about it too much, unless I thought it was going to confuse people.
cheers,
Matt
ma...@mattwynne.net
07974 430184
George Dinwiddie
I want to be able to have multiple Then steps (e.g. Then the request
should be successful and this field should be that value).
> Anyway as others have said, I don't think I would obsess about it too
> much, unless I thought it was going to confuse people.
Yeah, that's why I wanted to mention it here for wider feedback.
Jonas Nicklas
back and forth between given–when–then a couple of times (ya, sue me).
There's also a performance implication do this. If I would split them
into two scenarios, it would be *way* slower, since I would have to
repeat all of the setup, just to get to the same point, a point which
in of itself wasn't particularly interesting.
I'm curious though, obviously a lot of people feel very strongly about
this, what are your arguments as to why the GWT order should be
adhered to?
/Jonas
Matt Wynne
On 9 Dec 2010, at 21:41, Jonas Nicklas wrote:
> The case I posted was just a simple example. I have scenarios which go
> back and forth between given–when–then a couple of times (ya, sue me).
> There's also a performance implication do this. If I would split them
> into two scenarios, it would be *way* slower, since I would have to
> repeat all of the setup, just to get to the same point, a point which
> in of itself wasn't particularly interesting.
>
> I'm curious though, obviously a lot of people feel very strongly about
> this, what are your arguments as to why the GWT order should be
> adhered to?
For me, the reason I try hard to stick to GWT (and like you, I don't do it religiously) is that I think of a failing test case like an alarm - and when the alarm rings I like it to be obvious what I need to do. If there are multiple assertions stacked up in the same test case, then when the first one fails I don't know what the state of the other ones is until I fix that one. So normally I'll try to split the assertions into different test cases when that makes sense. The cost of a slower test run is paid off for me by the benefit of tests that give me clear messages when they fail.
The antipattern is what I call a "Rambling Scenario" where you amble around the whole system in a single, really long scenario. Concordion tests can look a lot like this, if you've ever seen them.
cheers,
Matt
ma...@mattwynne.net
07974 430184
Jon Kern
Jon Kern
I'll be the first to say "be pragmatic, not dogmatic." And that exceptions to the rule are not uncommon -- be agile!
However, pragmatic implies I have a really good reason to deviate from the dogma :-p
So, for me, I have found that I can generate concise and clear acceptance criteria when I try hard to stick with the GWT and the Single Responsibility Principle or DRY concept.
I often return to features that I wrote a month ago, read them now (usually because I am extending the app in that area), and realize I could write that feature in a more concise fashion. So I have become the GWT police in my own work and found it to be useful to struggle a bit and refactor those tests that aren't are clean and concise as they could be. Funny thing is, refactoring the tests is often quite simple, as it usually is just "rearranging" steps and grouping for context, and little else.
So, to expose my thought process on your original simple example:
>> When I try to create a post with invalid data >> Then I should see a form with errors >> When I change the fields with errors >> And I submit the form again >> Then I should see the new post
By contrast, when both of these "form submission modes" are presented as separate scenarios, I feel
- there is a greater chance that the system functionality will
be more obvious and recognized for what it is:
- normal mode and
- error handling mode
- there is less likely a chance that someone would re-test the
"normal mode" when they discover the need for a third way the
form should work.
And the only reason I pointed it out was because it could be taken out of context as a good way to write GWT, and I thought it made a nice case for a feature refactoring example.
Of course, these are simply my opinions and how I approach GWT... I might be misguided.