> I would prefer to merge it into RSpec whenever it's ready. One of the > strong points about RSpec is that it's a complete BDD toolkit: example > framework for developer testing, feature runner for customer test, and > the mock objects framework. People can get started with one part of > the toolkit, and gradually use the other pieces as their wants and
I agree. These are the key success factors of Ruby on Rails.
-Which consists of 5 or so gems
Aslak
> needs develop. I think it was a big win when RBehave merged into > RSpec. If Cucumber represents significant improvements over the > current Story Runner, I'd like to see those improvements make it into > RSpec.
>> Right now the scope of Before/After is global - i.e. all Before and >> After blocks will run before and after each scenario. I have >> deliberately postponed adding scoping of this until I better >> understand how people want to define this scoping. >> Here is one ideas: >> class MySteps < Cucumber::Steps # Same as Story runnner >> steps_for(:my) >> Before do >> # This block will run before all scenarios that use steps from >> MySteps >> # Any scenarios not using any of these will not have this block >> run >> end >> Given /bla bla/ do >> end >> end >> I'm open to other ideas too.
> I'd like to have a before story and after story as well as a before > scenario and after scenario.
> Right now I do global setup for a story, and cleanup when the story > is done. I also do a setup before each scenario as well.
> It would be nice to have the ability to do both.
Can you give me a code example? (In Cucumber it's Feature, not Story)
On Wed, Aug 20, 2008 at 8:32 AM, Jim Morris <m...@e4net.com> wrote: > Aslak Hellesøy wrote:
>>> I'd like to have a before story and after story as well as a before >>> scenario and after scenario.
>>> Right now I do global setup for a story, and cleanup when the story is >>> done. I also do a setup before each scenario as well.
>>> It would be nice to have the ability to do both.
>> Can you give me a code example? (In Cucumber it's Feature, not Story)
> An example of how I would use it or an example of how to do it?
Of how you would use it. Assume that you have something like this for scenario-level before/after:
class MySteps < Cucumber::Steps # Same as Story runnner steps_for(:my) Before do # This block will run before all scenarios that use steps from MySteps # Any scenarios not using any of these will not have this block run end
Given /bla bla/ do end end
Aslak
> Right now I use the listeners to call a before_story and after_story and > before_scenario.
> My Stories (Features) test the database as well as the return values of > various calls, so the before_story sets up the database accessors via > Sequel, putting them in global variables so I can easily access them from > anywhere.
> In the before_scenario I for instance make sure the user is logged out and > the database is cleared.
On Aug 20, 2008, at 2:20 AM, Aslak Hellesøy wrote:
> (In Cucumber it's Feature, not Story)
no offense, but while you're being picky about names, I dont see too much difference between 'story' and 'feature' but 'cucumber' is a really random meaningless name
> On Aug 20, 2008, at 2:20 AM, Aslak Hellesøy wrote:
>> (In Cucumber it's Feature, not Story)
> no offense, but while you're being picky about names, I dont see too much > difference between 'story' and 'feature'
I see them as very different.
User Stories are inputs to a development process and Features are the outputs.
When I've worked with FitNesse, we had stories on cards and a suite of FitNesse tests. Sometimes a Story would come up in an iteration that was an enhancement of an existing feature. In those cases we did not add new FitNesse tests, but simply enhanced the existing ones instead. So the FitNesse test suite grew to represent executable documentation of an existing system, not a tracking system for stories over the course of iterations.
Had we grouped all the FitNesse tests by the stories as they came in, in which iteration, etc, etc, they would have been much more difficult to navigate and maintain.
Automated scenarios live in the same place that FitNesse tests do. Ideally, we would group them by Story going into an iteration and then by Feature coming out, but we don't really have a good way of doing that.
Maybe the right approach would be to group them by neither Story nor Feature, but rather by execution context. For example, right now I've got stories that run one happy path scenario and one error path scenario that run in-browser, accompanied by a more exhaustive set of error path scenarios that run in-memory. These live in separate Story files and are only coupled together by their names.
It would be nice if I could have a suite of in-browser scenarios, a suite of in-memory scenarios that touch the full stack (in rails), and possibly a suite of scenarios that touch only a given model. Then each scenario can be tagged to a Story and a Feature, and the runner could support running everything by Story, Feature or Execution Context, thus supporting readability, navigability, etc, from a number of useful perspectives.
WDYT?
> but 'cucumber' is a really random meaningless name
On Wed, Aug 20, 2008 at 4:20 PM, David Chelimsky <dchelim...@gmail.com> wrote: > On Wed, Aug 20, 2008 at 9:04 AM, Jonathan Linowes > <jonat...@parkerhill.com> wrote:
>> On Aug 20, 2008, at 2:20 AM, Aslak Hellesøy wrote:
>>> (In Cucumber it's Feature, not Story)
>> no offense, but while you're being picky about names, I dont see too much >> difference between 'story' and 'feature'
> I see them as very different.
> User Stories are inputs to a development process and Features are the outputs.
I would add to that that the features, which are the outputs of the development process, are the inputs to the outcomes. AKA Business value. These two posts complement that aspect nicely:
Regarding Cucumber - yes, the name is completely meaningless, but a couple of chicks who attended my presentation about it at Agile 2008 told me afterwards it made their mind drift... That's good enough for me.
> When I've worked with FitNesse, we had stories on cards and a suite of > FitNesse tests. Sometimes a Story would come up in an iteration that > was an enhancement of an existing feature. In those cases we did not > add new FitNesse tests, but simply enhanced the existing ones instead. > So the FitNesse test suite grew to represent executable documentation > of an existing system, not a tracking system for stories over the > course of iterations.
> Had we grouped all the FitNesse tests by the stories as they came in, > in which iteration, etc, etc, they would have been much more difficult > to navigate and maintain.
> Automated scenarios live in the same place that FitNesse tests do. > Ideally, we would group them by Story going into an iteration and then > by Feature coming out, but we don't really have a good way of doing > that.
> Maybe the right approach would be to group them by neither Story nor > Feature, but rather by execution context. For example, right now I've > got stories that run one happy path scenario and one error path > scenario that run in-browser, accompanied by a more exhaustive set of > error path scenarios that run in-memory. These live in separate Story > files and are only coupled together by their names.
> It would be nice if I could have a suite of in-browser scenarios, a > suite of in-memory scenarios that touch the full stack (in rails), and > possibly a suite of scenarios that touch only a given model. Then each > scenario can be tagged to a Story and a Feature, and the runner could > support running everything by Story, Feature or Execution Context, > thus supporting readability, navigability, etc, from a number of > useful perspectives.
> WDYT?
>> but 'cucumber' is a really random meaningless name
On Wed, Aug 20, 2008 at 7:49 AM, aslak hellesoy <aslak.helle...@gmail.com>wrote:
> Regarding Cucumber - yes, the name is completely meaningless, but a > couple of chicks who attended my presentation about it at Agile 2008 > told me afterwards it made their mind drift... That's good enough for > me.
Aslak, please feel free to use whatever verbiage you choose. I just wanted to mention that for some (including myself), such comments don't reflect well on the speaker. Maybe I'm just showing my age. :)
On Wed, Aug 20, 2008 at 5:32 PM, Mark Wilden <m...@mwilden.com> wrote: > On Wed, Aug 20, 2008 at 7:49 AM, aslak hellesoy <aslak.helle...@gmail.com> > wrote:
>> Regarding Cucumber - yes, the name is completely meaningless, but a >> couple of chicks who attended my presentation about it at Agile 2008 >> told me afterwards it made their mind drift... That's good enough for >> me.
> Aslak, please feel free to use whatever verbiage you choose. I just wanted > to mention that for some (including myself), such comments don't reflect > well on the speaker. Maybe I'm just showing my age. :)
> The RSpec Story runner will continue to exist for a while, I just > wanted to let you know that future development and documentation > efforts will be focused on Cucumber. Cucumber also has some > documentation and examples on Github, more specifically:
> * How to use without Rails > * How to use with Rails+Webrat > * How to use with Watir
> Please let me know if you'd like to help out documenting Cucumber. One > thing that is missing is a migration guide from RSpec Story runner.
Do you prefer to keep such documentation in the README or would a github wiki be better?
I agree with Pat also. But I think importantly we need a little tool/guide on converting the old stories to cucumber before any movement.
I'm currently evaluating moving our stories to cucumber. I'll see if I can put something together that might help others (if its not already available?).
Also the word 'feature' resolves so many issues I had with stories and how they mapped into my testing framework. Thanks Aslak!
Josh Knowles wrote: > On 8/19/08, Pat Maddox <perg...@gmail.com> wrote: >> I would prefer to merge it into RSpec whenever it's ready.
> For whatever its worth I agree with Pat. Most of the short-comings of > the existing story runner that Aslak describes in the Cucumber README > are things that we at weplay have had to hack together. Having some > standardization for running a single scenario, or all scenarios, would > make developing tools on-top of the framework that much easier.
> Of how you would use it. Assume that you have something like this for > scenario-level before/after:
> class MySteps < Cucumber::Steps # Same as Story runnner steps_for(:my) > Before do > # This block will run before all scenarios that use steps from MySteps > # Any scenarios not using any of these will not have this block run > end
> Given /bla bla/ do > end > end
> Aslak
Ok here is an example from one of my current stories.... Modified to be Cucumber-like, although I see you put the Before and after in the steps, I'd p[refer them to be global, but could live with them in the steps class. Of course where would you put the scenarios before and after? They probably need to be global?
(Note I am not using Rails,this is an integration test against a Web Application testing the full stack using net/http and some helpers to wrap that. It can test remote servers, It also uses Sequel to test that the database gets setup properly, again a bunch of helpers lets me inject data directly into the database and test afterwards).
# setup several users each with their own web context # setup global database accessor BeforeFeature do # actual user names to login as user_names= %w(user50 user51 user52)
$umap= {} $wr= {} user_names.each_with_index do |e, i| n= "user#{i+1}" $umap[n]= e $wr[n]= WebRequest.new end
$users= {} $loggedin= {}
$db= MyDBHelper.new('local', false) end
# logout the users AfterFeature do $wr.each_value { |w| w.logout } end
# called before each scenario # to clean up any state conatianing globals used to communicate between steps # and log out any user that was logged in during the previous Scenario BeforeScenario do $wr.each_key do |u| $wr[u].logout if $loggedin[u] end
On Wed, Aug 20, 2008 at 6:24 PM, Ben Mabey <b...@benmabey.com> wrote: > aslak hellesoy wrote: >> The RSpec Story runner is likely to be deprecated in favour of the new >> feature runner (temporarily called Cucumber). >> http://www.nabble.com/-ANN--Cucumber-td18876816.html
>> The RSpec Story runner will continue to exist for a while, I just >> wanted to let you know that future development and documentation >> efforts will be focused on Cucumber. Cucumber also has some >> documentation and examples on Github, more specifically:
>> * How to use without Rails >> * How to use with Rails+Webrat >> * How to use with Watir
>> Please let me know if you'd like to help out documenting Cucumber. One >> thing that is missing is a migration guide from RSpec Story runner.
> Do you prefer to keep such documentation in the README or would a github > wiki be better?
Everyone here is welcome to edit it. If you have concrete experience (or wishes) about how to migrate existing stories to Cucumber this would be the place to do it.
I've been looking through the cucumber documentation and have a couple of questions.
I'm curious which of the disadvantages you list would be impossible/very difficult in the classic story runner. I'm just trying to envisage if Cucumber and the classic story runner where to co-exist what would the Cucumber plugin be able to do that the classic story runner could never realistically hope to achieve.
Also looking at one of your disadvantages: * 'Limited colouring of output' I've been playing around with patches here and there to improve the colour of the classic story runner formatters. Do you still see limitations in this as it is in edge?
My final question is about the Rspec book. I've no idea when this will be released or what pressures there are on publish deadlines. How much would this effect a move to Cucumber (avoiding having redundant story examples in the book when we all use Cucumber)?
> Everyone here is welcome to edit it. If you have concrete experience > (or wishes) about how to migrate existing stories to Cucumber this > would be the place to do it.
On Aug 20, 2008, at 10:20 AM, David Chelimsky wrote:
> I see them as very different.
> User Stories are inputs to a development process and Features are > the outputs.
I decided to churn on this for a few days before responding. Actually I was going to let it drop, but kept thinking about it.
Given your observation (above) I see an analogy between Stores -> Features, and Specs -> Tests, that is, I write specs as input, I develop the code, and tests are the outputs. Same code, different purpose.
You (or someone) named this framework rspec perhaps because 'spec' is more descriptive and accurate than 'test', but more importantly because there is a large body of work and history with conventional QA and testing, and it was important to coin a new term (spec) to distinguish BDD from conventional testing. However, you can use test tools to do BDD (e.g. as many people has chosen to continue using Test Unit and shoulda rather than rspec for whatever reasons).
But with regard to stories vs features, there really isnt a legacy and baggage about the word 'story', so changing the name to 'feature' is mostly semantics.
Personally, I do not see a feature as separate from a story, because a feature means nothing without some context. Stories provide that context. Scenarios provide the specifics. I might write and organize my stories by individual feature, but theres other ways as well: workflows, goals/objectives, different starting setups, user roles, and so on, all involving one or more sets of features.
I'm not trying to be argumentative, and I'm wholeheartedly appreciative of the work you and Alask and everyone does on this project. If anything, I'm just trying to sort all this out for myself.
> On Aug 20, 2008, at 10:20 AM, David Chelimsky wrote:
>> I see them as very different.
>> User Stories are inputs to a development process and Features are the >> outputs.
> I decided to churn on this for a few days before responding. Actually I was > going to let it drop, but kept thinking about it.
> Given your observation (above) I see an analogy between Stores -> Features, > and Specs -> Tests, that is, I write specs as input, I develop the code, > and tests are the outputs. Same code, different purpose.
> You (or someone) named this framework rspec perhaps because 'spec' is more > descriptive and accurate than 'test', but more importantly because there is > a large body of work and history with conventional QA and testing, and it > was important to coin a new term (spec) to distinguish BDD from conventional > testing.
Steven Baker named the framework rspec. I believe his motivation was Uncle Bob saying "specify, don't verify" when talking about what TDD and ATDD is about.
Sadly, "spec" has just as much baggage, if not more, as "test" does. These days we're calling these things "code examples," (tongue pressing into cheek) so maybe we should change the name to rcodeexample?
> However, you can use test tools to do BDD (e.g. as many people has > chosen to continue using Test Unit and shoulda rather than rspec for > whatever reasons).
Agreed. Tools is tools. Process is process. (boat is boat ....)
> But with regard to stories vs features, there really isnt a legacy and > baggage about the word 'story', so changing the name to 'feature' is mostly > semantics.
User Stories have been around since the late 90's. I'd say that qualifies them for legacy/baggage.
> Personally, I do not see a feature as separate from a story, because a > feature means nothing without some context. Stories provide that context. > Scenarios provide the specifics. I might write and organize my stories by > individual feature, but theres other ways as well: workflows, > goals/objectives, different starting setups, user roles, and so on, all > involving one or more sets of features.
> I'm not trying to be argumentative, and I'm wholeheartedly appreciative of > the work you and Alask and everyone does on this project. If anything, I'm > just trying to sort all this out for myself.
This is something that's bugged me since rbehave first appeared (even before we merged it into rspec). I've discussed this w/ Dan North a few times and we've agreed on some aspects of this but not all. I won't speak for Dan here, but my experience tells me that there is not a clean mapping of stories (the things that drive development) to features (the things that exist in the system because they have already been developed).
For example, imagine we're working on a conference organization/registration tool and way back in iteration 1 we had a story about the organizer being able to see a report of conference registrations (this may sound familiar if you saw my talk at RailsConf).
Imagine that one of the conference organizers noted that a given conference was reporting 100% full even though only 499 of the goal of 500 had registered. So a new story gets added suggesting that it should not be rounded up to 100%.
So now we have two choices. We can add a new story file with a new narrative and a single scenario, or we can crack open the existing story file and add a single scenario.
In terms of the feature (which is the report), I see this as just another scenario.
In terms of driving development and estimating effort, I see this as a new User Story.
OT, reminds me of when, a while back, I was developing CAD software, and had a potential customer call who told me (with a strong Boston accent) their company designs pots. "Oh, neat, what kind of pots, you mean cooking pots?" I asked. "No, no," she replied, "machine 'pots' " Needless to say, I didnt win the client :)
On Thu, Aug 21, 2008 at 6:08 PM, Joseph Wilk <li...@ruby-forum.com> wrote: > Hello,
> I've been looking through the cucumber documentation and have a couple > of questions.
Hi, Sorry for the late reply,
> I'm curious which of the disadvantages you list would be impossible/very > difficult in the classic story runner. I'm just trying to envisage if >From http://github.com/aslakhellesoy/cucumber/wikis - 10 means big
effort, 1 small. Relatively. Your opinion may differ.
5: Hard to get started with. A special "all.rb" file must be written before it can be used. 3: No out of the box Rake support, which puts a lot of people off. 3: No i18n, so if you want to write stories in a different language than English you're out of luck. 8: Poor error reporting. No way to know on what line a plain text story failed during execution or parsing. 1: Limited colouring of output. 5: No simple way to execute only one scenario. 8: No command line tool to run stories. 2: No easy before or after hooks. 5: No highlighting of step parameters in the output 2: No detection of ambiguous step names
> Cucumber and the classic story runner where to co-exist what would the > Cucumber plugin be able to do that the classic story runner could never
I assume that by plugin you mean Rails plugin? (Cucumber is a standalone library that *may* be used as a Rails plugin).
> realistically hope to achieve.
The Story runner could achieve this if someone put enough effort into it. That would have to be someone other than me, because I don't have the time (or desire) to do it. Let's say it's up for grabs.
> Also looking at one of your disadvantages: > * 'Limited colouring of output' > I've been playing around with patches here and there to improve the > colour of the classic story runner formatters. Do you still see > limitations in this as it is in edge?
That's one of the easiest things to fix, and also one of the smallest disadvantages IMO.
> My final question is about the Rspec book. I've no idea when this will > be released or what pressures there are on publish deadlines. How much > would this effect a move to Cucumber (avoiding having redundant story > examples in the book when we all use Cucumber)?
Regarding the book - we haven't decided whether or not to cover Cucumber.
In any case, *writing* stories/features for the Story runner will be almost identical to Cucumber features.
On Tuesday I ported one of our projects at work over to Cucumber. Over 1000 steps. I ended up having to change almost nothing in the text or step defs (except for resolving some duplicates and ambiguities that Cucumber complains about where RSR says nothing). I have written up what I did and will post it to the Cucumber wiki next week when I have some time to proofread it.
> Thanks, > -- > Joseph Wilk > http://www.joesniff.co.uk > aslak hellesoy wrote: >> On Wed, Aug 20, 2008 at 6:24 PM, Ben Mabey <b...@benmabey.com> wrote: >>>> * How to use without Rails >>> wiki be better?
>> Everyone here is welcome to edit it. If you have concrete experience >> (or wishes) about how to migrate existing stories to Cucumber this >> would be the place to do it.
As the author of the original scenario runner, if Aslak has come up with a nicer implementation - both in terms of design and hackability - then I say chuck my one out and use his :)
As long as it is an easy adjustment (i.e. transparent or with an easy migration) for users of the current scenario runner then I think we should ship it in.
> I would prefer to merge it into RSpec whenever it's ready. One of the > strong points about RSpec is that it's a complete BDD toolkit: example > framework for developer testing, feature runner for customer test, and > the mock objects framework. People can get started with one part of > the toolkit, and gradually use the other pieces as their wants and > needs develop. I think it was a big win when RBehave merged into > RSpec. If Cucumber represents significant improvements over the > current Story Runner, I'd like to see those improvements make it into > RSpec.
2008/8/24 David Chelimsky <dchelim...@gmail.com> [...]
> Sadly, "spec" has just as much baggage, if not more, as "test" does. > These days we're calling these things "code examples," (tongue > pressing into cheek) so maybe we should change the name to > rcodeexample?
Or rbehave?
The rbehave.org domain is available (I registered it some time ago), and rspec has naturally evolved from its original goal of code-level specs to become a full-stack behaviour description framework.
Just a thought.
With regard to the stories and features thing, I see a BDD-shaped story as providing a context - and justification - for a feature:
As a [stakeholder] *I want [a feature] *So that [I get some benefit]
Before we started using this structure, a "story" would often just be the middle line, so it wasn't immediately obvious who the stakeholder was or why they wanted the feature, which in turn would often lead to over-work, under-work or just plain wrong-work. Of course the word "story" has its own baggage. In XP a story is "a placeholder/promise for a conversation", and as such could just be a title scribbled on a card. I wrote the story article<http://dannorth.net/whats-in-a-story>to put this all in context - if you ask 5 agile folks what a story is, you will likely get 6 answers.
I agree that *the feature is the interesting thing*, and also that there may be several stories about the same feature in different broad contexts. In any event the scenarios provide the definition of "Done" for the feature, which is kind of the whole point. So I guess I'm saying I'm ambivalent about the story/feature distinction. I don't look at stories as work units as much as a more formal description of (some aspect of) a feature.
After speaking with Aslak - and some FDD folks I met at Agile 2008 - I can fully agree with organising stories by feature. In fact in Peter Coad's FDD they have features within feature sets, within subject areas, which might well map to stories within features within [not sure - subject areas? themes? something broader anyway]. FDD features seem to be "thinner" than what I understand Aslak's description of features to be.
One thing that makes me happy is that we seem to have consensus around the word "scenario" - which is where the outside-in work really starts.
> 2008/8/24 David Chelimsky <dchelim...@gmail.com> > [...] > Sadly, "spec" has just as much baggage, if not more, as "test" does. > These days we're calling these things "code examples," (tongue > pressing into cheek) so maybe we should change the name to > rcodeexample?
> Or rbehave?
> The rbehave.org domain is available (I registered it some time > ago), and rspec has naturally evolved from its original goal of > code-level specs to become a full-stack behaviour description > framework.
or RubyDD
or RuBehave
:)
I actually really like calling them specs rather than tests, at a unit-testing level. It makes a real difference to me that I'm expressing a *specification* for the class I'm about to code - it makes it much more natural to do it before you write the implementation when it's a spec rather than a test.
