BDD Terms

[Ray Houston]

Anyone want to take the first crack in defining these? Add some new ones?

Story -
Scenario -
Feature -
Specification -
Test -
Acceptance Criteria -
Acceptance Test -

[colinjack]

Just to try my understanding, so that everyone can disagree:

Story - A user story, high level description and related conversation
(though I normally just use the term user story even when I'm talking
about the associated acceptance tests). http://www.extremeprogramming.org/rules/userstories.html
Scenario - Not really clear on this one, don't use the term. I'd guess
its an example of a real usage that is then generalized into an
acceptance criteria (not sure though).
Epic (Feature) - A large user story, too large to be used for planning
etc. so is likely to be split into other user stories before planning
and assigning them to iterations (thats my experience anyway).
Specification - A specification of a behavior against a part of the
design, in TDD terms a single test.
Test -
Acceptance Criteria - Associated with a user story, allow you to judge
when the user story will be complete. Worked out by developers/users
during the detailed discussions.
Acceptance Test - High level tests that ensure that acceptance
criteria are met by the system. Black box system tests (http://
www.extremeprogramming.org/rules/functionaltests.html).

[Scott Bellware]

My 2 cents:

Story: Shorthand for User Story; An expression of a requirement from a
contextual and user-centric perspective using the form: As a <role>, I
want to <goal> so that <motivation>.

Scenario: An execution path through a finite extent of the system.

Feature: A capability of the system. Feature has no natural
correlation with user stories and scenarios. Features can be cross-
cutting concerns, or can be represented as a story, or anything in
between.

Specification: A description of an expectation of a system's behavior
or geometry, or a construct that represents a description of an
expectation of a system's behavior or geometry.

Test: A verification of an expectation of a system's behavior or
geometry, or a construct that represents a verification of an
expectation of a system's behavior or geometry.

Acceptance Criteria: Conditions that constrain the system's
acceptability to the system's specifications.

Acceptance Test: A verification of the conditions that constrain the
system's acceptability to the system's specifications.

-Scott

[colinjack]

> Feature: A capability of the system. Feature has no natural
> correlation with user stories and scenarios. Features can be cross-
> cutting concerns, or can be represented as a story, or anything in
> between.

Just wondering how you represent your features?

We drive everything from user stories, some of which are epics so we
don't have this additional layer. Happy to use the term but I still
don't fully understand it.

[Jimmy Bogard]

This may come from Scrum too, which recommends that everything in the backlog is an amorphous "feature".  IMO they intentionally leave a specific definition out, for individual teams to decide what actually goes in there.

[Derick Bailey]

BDD is all about the process of creating software, not the software
that has already been created. A feature is language that talks about
what a piece of software does, not how it's created.

...

BDD
Story: As a mechanic, I want to log hours against a work order, so
that the customer gets billed appropriately.
Acceptance Criteria: When a mechanic logs hours against a work order
for a specific date, the totals hours for that mechanic on that work
order on the specified date cannot exceed 24.

Features:
* Work Order Management
* Work Order Based Billing
* Employee Time Entry System
* etc.

[Gregory Long]

I am still learning about BDD (and therefore probably not doing things 'right') so consider the source . . .

 

Story - a tale describing (and bounding) a user's desires for a task.  I tend to follow the As a <role>/I want <goal>/So that <benefit> syntax which limits the 'plot' of the story but I'm not completely sold this is a good thing. 

Scenario - a specific telling of a Story that provides [relatively] loosely bounded expectations.  I strive to provide adequate context to bound a conversation about acceptable behavior in context.

Feature - a desired capability expressed in non-specific terms.  It may bound one or more stories.

Specification - a tightly bounded behavior within a given context

Test - a verification (or a mechanism for verification) of a single specification

Acceptance Criteria - a tightly bound expectation for a behavior within a given context.  I believe Specifications and Acceptance Criteria should arrive at the same conclusion - although not necessarily with the same exact language (the tricky part).

 

Conclusion (you didn't ask but I added) - a shared understanding between users and developers about what constitutes acceptable behavior from an application.

Acceptance Test - a verification (or a mechanism for verification) of a single Acceptance Criteria

 

@Scott - you posted in another thread about how your journey from development 'speak' to english has contributed to your interest in BDD (among other pursuits).  Being an English Literature major - literary theory to be precise - I was struck by the irony of our two situations.  I've spent my entire career trying to understand developent 'speak' and railing against the language.  I've gravitated towards OO and now BDD/DDD because I find the 'translation' between my conversations with people and my conversations with a compiler to be easier.  This is partly why I gravitate towards definitions and usage of BDD terms that more closely map to the 'literal' english.  While I believe I understand and appreciate the specificity of langauge like

"a construct that represents a verification of an
expectation of a system's behavior or geometry"

I find the opportunity for mis-understanding based on context unappealling.  I'm undecided wherein the fault lies - with the english language or the language of development.  Quite probably both.

 

greg

gregor...@gmail.com

[Scott Bellware]

Colin,

I don't use "feature" as a unit of measure for software production. I
use it loosely. By convention, it can point to just about anything.
It's more a unit of conversation. It's still a useful term, but it's
more of an informal term in my use and not used specifically to point
to anything in the methodologies I use.

[Joe Ocampo]

@Scott

Just so I can understand better, can you give me some examples of what you would consider features?

[Scott Bellware]

Greg,

> "a construct that represents a verification of an
> expectation of a system's behavior or geometry"
> I find the opportunity for mis-understanding based on context unappealling.
> I'm undecided wherein the fault lies - with the english language or the
> language of development. Quite probably both.

For example, a test method is often called "a test". An execution of
an assertion API, eg: Assert.NotNull(myFoo), is called "a test".
Colloquially, they're necessarily both right. Technically, it's the
assertion that is the test, but following that definition rigorously
would seriously hamper our ability to use assumptions and context to
accelerate communication.

I'm referring to the test method (or test class for that matter) as
the construct, since we don't usually use "construct" to mean a single
line of code.

-Scott

[Scott Bellware]

Joe,

> Just so I can understand better, can you give me some examples of what you
> would consider features?

Funds transfer might be a feature, and on-line banking, and so might
logging. Architectural "ilities" are also frequently referred to as
features. Just about anything that can be observed of a system can be
thought of as a feature - if it makes sense to speak of it that way.

-Scott

[Ray Houston]

So can a scenario be an acceptance criterion?

[Laribee]

I would say so. So can a note or a snippet of IM conversation of a
description of an interaction of ux.

I think scenarios are a more set in stone kind of criterion though.

On Mar 13, 2008, at 11:09 PM, "Ray Houston" <ray.h...@gmail.com>
wrote:

[Ray Houston]

So a scenario can be a form of an acceptance criterion and can be
validated through acceptance tests. Is acceptance tests the correct
term? Is there something more spec like? Is it just a high level spec
or does it have a different format?

If they are end to end specs/tests/whatever, does that make a
difference to how you code them? We have .NET specs, Flex specs (for
the UI), then we have RBehave executable scenarios (I'm going to stop
calling these story runners) that validate end to end acceptance
criteria. So our scenarios become acceptance tests. Thoughts?

-Ray

[Joe Ocampo]

If I am dealing with the UI I tend to always refer to these type of test as acceptance test.  Which usually come from the modeling concerning the features of the application.  In each of these acceptance test there are scenarios that capture different aspects of the UI in order to have proper test coverage.  i.e boundary testing, range testing, negative testing etc.

But scenarios are not limited to just UI acceptance testing.  Because we are now moving into BDD at the lower level the context on which an artifact is governed steers(or is) a scenario.

my two cents.

--
God Bless,

Joe Ocampo
agilejoe.lostechies.com

"How do you benefit if you gain the whole world but lose your soul in the process?" Mark 8:36

[Ray Houston]

I've been under the impression that scenarios are similar to use cases
where you have a role that does something with the system for a
particular benefit. If scenarios are not that, then what's the term
for acceptance testing at the high level. As an X, I want to do Y, so
that I can Z. How do you create an automated test that proves that X
can do Y can happen? Generally there are a lot of collaborates to make
all that happen and you have specs for all of those and mine aren't
normally for end to end validation. When I say acceptance tests (maybe
this is the wrong term), I mean end to end tests.

[Derick Bailey]

> So can a scenario be an acceptance criterion?

Define: Scenario
"An account or synopsis of a projected course of action or events."
"A plausible description of how the future may develop, based on a
coherent and internally consistent set of assumptions about key
relationships and driving forces"

...

I think Scenario to Acceptance Criteria is the same relationship as
Rectangle to Square. Not all rectangles are squares, but all squares
are rectangles. Similarly, not all scenarios are acceptance criteria,
but by the loose definition of scenario, all acceptance criteria are
scenario.

To be perfectly clear, I don't think you should interchange the two
terms at will. A conversation could easily involve both terms, and you
really should keep them separate.

Dev: "So, what about this scenario: When ThisRole does
ThatThingOverThere, did You get ThatThing that ISentYou?"
Customer: "No. ThatThing was SentByWhatever, not You."
Dev: "Ok, so the acceptance criteria in this story should really
say ..."

[Derick Bailey]

> I've been under the impression that scenarios are similar to use cases
> where you have a role that does something with the system for a
> particular benefit. If scenarios are not that, then what's the term
> for acceptance testing at the high level. As an X, I want to do Y, so
> that I can Z.

That's a user story: "As [Role], I want to [Action], so that [Value]"

> How do you create an automated test that proves that X
> can do Y can happen?

This is where you need Acceptance Criteria. You can't write code
against a user story alone. You must have Acceptance Criteria to write
code against.

[Ray Houston]

I don't think I was clear on my question/point. How are people
validating their acceptance criteria that relates to the action that a
particular role wants to perform? I'm talking about end to end testing
and not unit testing. Is end to end validation also done in the form
of specs?

[Joe Ocampo]

Hmm...I think I understand your question but let me just tell you what I do.

The first thing we do is come up with the Persona's

As a Loan Administrator
As a Processor
As a Consumer

As a Consumer I want to add my information to a load application so that .....
As a Loan Administrator I want be the only one who can approve a loan so that...
As a Processor I want to be able to validat the pricing information on a loan  so that...

Each one of these epics covers several screens such as:
- Loan App Demographics
- Loan App Pricing
- Loan App Product selection
- Loan App Assets
etc....

Each one of the screens is going to governed by either a story or multiple stories and developed incrementally.

As each story is completed you excercise you acceptance testing.  But remember some screens could have multiple stories.  For instance demographics.

Demographics screen consists of the following modules:
- Address Information
- Employment Information
- Loan Property Information
- Spousal/Partner information

Each one of the modules is usually its own story and lends itself towards the completion of the demographic screen.

If you have automated acceptance testing employed it should be really easy to implement these modules incrementally.  Once the entire page is done then you start exercising feature level acceptance testing.

Now when all the screens are done to complete the work flow then we go into end to end testing.  For us (the corporate world) this is usually one of the final stages before we launch into production and the code is locked down.

Does this help?

[Ray Houston]

@Joe

>Now when all the screens are done to complete the work flow then we
go into end to end testing.

So if BDD is about the process of creating software, and end to end
testing is done after the software is written, then end to end testing
is not part of BDD?

Couldn't BDD work for end to end testing if you stub out your end to
end spec, then implement screens and collaborators (all with more
detailed specs) until your end to end specs are satisfied? (we're
trying to do this).

-Ray

[Joe Ocampo]

> Couldn't BDD work for end to end testing if you stub out your end to
>end spec, then implement screens and collaborators (all with more
>detailed specs) until your end to end specs are satisfied?

I suppose you could but for end to end testing we usually rely on more higher level abstract acceptance testing frameworks such as Selenium, WatiR, or WatiN  If you have a big budget then we use Mercury Quick Test Pro.

[Ray Houston]

Couldn't you use RBehave with WatiR? Does this still fall under the BDD label?

[Joe Ocampo]

>Couldn't you use RBehave with WatiR? Does this still fall under the BDD label?

In a way I guess it kind of fits in if you are authoring the End to End testing with a contextual approach and governing the context through the specifications.  But I am not sure if it fits in the BDD realm.  Great question!

[Jimmy Bogard]

Asking your question a different way, are stories/scenarios a viable grammar for acceptance tests?  That's what your WatiR tests are, correct?

[Ray Houston]

Correct. Specifically end to end acceptance tests.

[Joe Ocampo]

I say we bring in an English major to help us with the BDD grammer and usage.

[Colin Jack]

After reading a few more threads I'd be interested in someone giving
me a definition for the term "context".

I'm pretty sure I know what people mean but it is possible that my
understanding is different to other peoples.

[Scott Bellware]

Ray,

> So can a scenario be an acceptance criterion?

A scenario is a way to express acceptance criteria using an example-
driven approach. A scenario can express more than one criteria if
needed.

-Scott

[Scott Bellware]

Ray,

> I've been under the impression that scenarios are similar to use cases
> where you have a role that does something with the system for a
> particular benefit.

Scenarios are particular paths through a use case. Use cases have a
primary path, alternate or secondary paths, as well as exceptional
paths. You can express these paths as scenarios, or you can use
scenarios to express finer-grained paths that are too fine-grained for
description using use cases.

From an implementation perspective, data-driven testing is an example
of scenario testing, where each row or dataset is a scenario.
Scenarios can also be written in plain old code without the use of row
testing or data drivers.

Scenarios are just paths. They aren't synonymous with user stories or
use cases, but user stories and use cases can be decomposed into
scenarios.

> what's the term
> for acceptance testing at the high level

High-level acceptance testing :) Unfortunately, "high-level" isn't
objective and might not always offer anything of value in a testing
taxonomy, and testing taxonomies aren't objective either, so the
taxonomy itself isn't flawlessly applicable for communication with
people who don't live inside the direct confines of our own sculls.

> When I say acceptance tests (maybe
> this is the wrong term), I mean end to end tests

I say "end-to-end tests". It's the one term that folks seem to get
off the bat or this kind of thing that doesn't require further
qualification like, "and by acceptance tests, I mean 'end-to-end
tests'".

Acceptance tests don't have to be end-to-end, neither do customer-
facing tests for that matter.


-Scott

On Mar 14, 8:11 am, "Ray Houston" <ray.hous...@gmail.com> wrote:
> I've been under the impression that scenarios are similar to use cases
> where you have a role that does something with the system for a
> particular benefit. If scenarios are not that, then what's the term
> for acceptance testing at the high level. As an X, I want to do Y, so
> that I can Z. How do you create an automated test that proves that X
> can do Y can happen? Generally there are a lot of collaborates to make
> all that happen and you have specs for all of those and mine aren't
> normally for end to end validation. When I say acceptance tests (maybe
> this is the wrong term), I mean end to end tests.
>

> On Fri, Mar 14, 2008 at 7:59 AM, Joe Ocampo <agile...@gmail.com> wrote:
> > If I am dealing with the UI I tend to always refer to these type of test as
> > acceptance test. Which usually come from the modeling concerning the
> > features of the application. In each of these acceptance test there are
> > scenarios that capture different aspects of the UI in order to have proper
> > test coverage. i.e boundary testing, range testing, negative testing etc.
>
> > But scenarios are not limited to just UI acceptance testing. Because we are
> > now moving into BDD at the lower level the context on which an artifact is
> > governed steers(or is) a scenario.
>
> > my two cents.
>

> > On Fri, Mar 14, 2008 at 7:12 AM, Ray Houston <ray.hous...@gmail.com> wrote:
>
> > > So a scenario can be a form of an acceptance criterion and can be
> > > validated through acceptance tests. Is acceptance tests the correct
> > > term? Is there something more spec like? Is it just a high level spec
> > > or does it have a different format?
>
> > > If they are end to end specs/tests/whatever, does that make a
> > > difference to how you code them? We have .NET specs, Flex specs (for
> > > the UI), then we have RBehave executable scenarios (I'm going to stop
> > > calling these story runners) that validate end to end acceptance
> > > criteria. So our scenarios become acceptance tests. Thoughts?
>
> > > -Ray
>
> > > On Thu, Mar 13, 2008 at 11:13 PM, Laribee <da...@laribee.com> wrote:
>
> > > > I would say so. So can a note or a snippet of IM conversation of a
> > > > description of an interaction of ux.
>
> > > > I think scenarios are a more set in stone kind of criterion though.
>

> > > > On Mar 13, 2008, at 11:09 PM, "Ray Houston" <ray.hous...@gmail.com>

[Scott Bellware]

Joe,

> The first thing we do is come up with the Persona's
>
> As a Loan Administrator
> As a Processor
> As a Consumer

That's not exactly what I would colloquially think of a personas. I
got thrown when I saw this. Folks who work principally with personas
would probably be thrown by this as well. [comprehension and intra-
team communication velocity decelerating]

Loan Administrator, Processor, and Consumer seem more likely to be
roles rather than personas.

You could create personas for those roles: Linda the Loan
Administrator who is 50 years old, works principally at her desk and
communicates with blah, blah, blah, etc.

My 2 cents.

-Scott

[Scott Bellware]

Ray,

There's no meaningful way to map specific testing tools to BDD, or to
TDDD for that matter. Any formulaic, tool-based approach to
differentiation would be an arbitrary and limiting overlay on top of
BDD (or TDD).

You can run WatiX tests from any test framework or runner. No
permutation is more naturally appropriate.

For example, we ran Watir tests from RSpec rather than RBehave. I've
run Selenium tests from StoryTeller and from NUnit.

The kind of arbitrary separation that you're talking about here is
often the result of having discontinuous production flow where certain
kinds of tests are being produced at arbitrary points during the
flow. That kind of thing isn't necessarily endemic to testing, TDD,
or BDD. It's often more of a matter of being constrained by our own
assumptions.

-Scott

[Scott Bellware]

Colin,

> After reading a few more threads I'd be interested in someone giving
> me a definition for the term "context".

It's the temporal blob containing circumstances, conditions, pre-
dispositions, and a host of influences that describe the experiencer's
position in time, space, thought, and emotion relative to a goal.

Succinctly: Context is where it's at, man! :)

-Scott

[Colin Jack]

> Scenarios are particular paths through a use case. Use cases have a
> primary path, alternate or secondary paths, as well as exceptional
> paths. You can express these paths as scenarios, or you can use
> scenarios to express finer-grained paths that are too fine-grained for
> description using use cases.

I think I get the term but I think the choice of word could lead to
confusion. In use case terms aren't the primary and all alternate
paths "usage scenarios", but as you say the way we are using scenario
here isn't specific to use case driven approach (it works for user
stories too).

[Colin Jack]

> It's the temporal blob containing circumstances, conditions, pre-
> dispositions, and a host of influences that describe the experiencer's
> position in time, space, thought, and emotion relative to a goal.
>
> Succinctly: Context is where it's at, man! :)

Bah! :)

[Jimmy Bogard]

Yeah, that's how we use "personas" as well.  We don't always use them, but it helps when creating scenarios.  "OK, Sally SupportRep fields a support call to change Carl Customer's account information" etc.

[David Laribee]

A persona is just a role with some meat behind it. You might explain motivations, income, etc. A persona is good for marketing to get into a kind of method acting vibe and start to ideate what a real customer or aggregation of customers might care about. Very useful in the case where there isn't a distinct customer fitting a distinct role.

I also wouldn't start there up front. Stories can be refactored as roles emerge. Start with "user" and "administrator" but keep a critical eye on churning stories back to having a non-generic role. Also, keep in mind, developer is usually a role on a non-trivial application with an API (for example).

--


/ Dave

http://thebeelog.com

[David Laribee]

- or -

Context is where a behavior starts.

I like to think of it as Bruce Sterling's notion of "specs" or augmented reality. In the future, you see, we'll all wear these lightweight glasses that overlay information about the world we see. If, say, I'm looking at a coffee machine (context) the overlay will gradually fade in with "make coffee" or "order coffee" or "clean pot" or whatever...

To exercise these behaviors I need some context. In this example, a coffee pot.

But you don't have to take my word for it!

http://www.amazon.com/Good-Old-Fashioned-Future-Bruce-Sterling/dp/0553576429

[Colin Jack]

> Context is where a behavior starts.
>
> I like to think of it as Bruce Sterling's notion of "specs" or augmented
> reality. In the future, you see, we'll all wear these lightweight glasses
> that overlay information about the world we see. If, say, I'm looking at a
> coffee machine (context) the overlay will gradually fade in with "make
> coffee" or "order coffee" or "clean pot" or whatever...

Looks interesting but I was really looking for a definition that
relates directly to the way people are writing their tests/
specifications which I found on another thread.

[Scott Bellware]

Dave,

> I'm looking at a
> coffee machine (context) the overlay will gradually fade in with "make
> coffee" or "order coffee" or "clean pot" or whatever...

Man, I want to be a customer in your cafe!