SBE critique by a new user

[Bob Rundle]

Hi folks,

My first post in this group.  Just finished SBE book (the red book?).  Trying hard to apply SBE to green field project and struggling with a few aspects.  Would appreciate some advise and direction.  See below.

(Perhaps I should have put each of these issues in a separate post)

Issue	Comment
SBE seems to be for web projects only	The sweet spot for SBE applications seems to be a website or a web service.  Difficult to impossible to adapt for classic desktop applications, embedded software and in fact any software for which the web component is small.
SBE tools seems to be suitable for text only	Executable specifications are the core of SBE.  However all the examples of SBE I have found use tables of text as the example.  This is completely unsuitable for software that creates images or does image processing or any kind of 3D visualization.
SBE tools are unsuitable for non-programmers	The SBE tooling I have looked at requires programming tools to use.  For example SpecFlow requires Visual Studio to be installed.  SpecFlow+ supports Excel, but the Excel workbooks that must be created are fragmented and cannot be reasonably used by non-programmers.  What I have done is to organize my features and scenarios into a single Excel workbook that can be easily reviewed by all stakeholders.
Scenarios are organized around features	For some reason this took a long time for me to grasp.  Nowhere in the book is this explicitly stated.  I started out organizing features around scenarios.  This approach fails because the tools all explicitly  organize the scenarios around features.  So now things are going much better, but it seems that something is missing.  Even the simple scenarios called for in SBE often connect 2 features.  So what I have to do now is decide which feature the scenario will be part of.
Features have to be right-sized	I started out with a long list of features, many of which were very low level and others that were high level.  Neither the high level nor the low level features supported reasonable SBE-style user scenarios.  It seemed to me that a feature had to be at a certain middle level for user scenarios to be written.  So I eliminated all the high and low level features and things are going much better.  However the resulting specification will be incomplete.  All of the high and low level features that would be normally included in a waterfall-style specification are missing.  YAGNI you might say, but I am wondering.
No complete SBE example exists	All the examples I have seen of SBE are trivial.  It would be helpful to have some comprehensive yet small implementation of SBE on GitHub or elsewhere.  Perhaps one exists, but I cannot find it.
Living documentation not printable	The living documentation part of SBE is one of the aspects that makes it really compelling to many stakeholders.  However some the tooling I have looked at there is now way to render said documentation in an reasonable printed form.  Printed document is immediately obsolete you say…but not really.  It will obsolete weeks or months from the time it was printed, but it will be plenty relevant for the short period of time various stakeholders need to review it.  If you a really and truly want to communicate the SBE to all stakeholders, there has to be a way to render a decent document from it.

[jbenilov]

Hi Bob,

Here's a real example of some open-sourced specs that a UK government agency developed while delivering a new funding policy calculator:

https://github.com/SkillsFundingAgency/das-paymentsacceptancetesting/tree/master/features

Hope that helps,

Jake

[Bob Rundle]

Thanks, Jake.  This is really the 1st non-trivial example I have seen of SBE...very helpful.  Can I suggest a readme file?  It is hard to get an overview simply reading through the feature files.  What tool produced these?  I only have a shallow understanding of SpecFlow and SpecFlow+.  BR

[Nicole Rauch]

Hi Bob,

since nobody seems to have replied to your post so far, I'll try and have a go at the questions/issues you raised.

My first post in this group.  Just finished SBE book (the red book?). 

Actually, to get to know Specification by Example thoroughly, I recommend the other book called "Bridging the Specification Gap" ("the blue book"). (The book names keep confusing me too.)

Trying hard to apply SBE to green field project and struggling with a few aspects.  Would appreciate some advise and direction.  See below.

(Perhaps I should have put each of these issues in a separate post)

I'll try to address your points one by one:

"SBE seems to be for web projects only": No, not at all. SbE is not limited to testing through the UI. Instead, you can write fixtures (i.e. intermediate code between the spec examples and your production code) that directly invoke individual methods of your code. This way, the granularity of the code under test can vary from an integration test down to a unit test. The key idea underlying this is the idea of "acceptance tests" which are tests that cover a certain business aspect of the code. One such business aspect can e.g. be an isolated calculation or decision that is implemented in a single method. This way, you can cover all aspects individually which is very important when you need to understand errors.

"SBE tools seem to be suitable for text only": Unfortunately, I have no experience with the kinds of applications you mentioned. Sorry!

"SBE tools are unsuitable for non-programmers": I've successfully used FitNesse with business people who were not programmers. We discussed the table layouts and rough layout of examples in our spec workshops, then the devs went to implement the fixtures, while the business people fine-tuned the examples. They directly modified the FitNesse pages. So yes, you need devs to implement the fixtures, but non-devs can be tightly integrated as well.

"Scenarios are organized around features": Yes :-) That's very much in line with what I said about acceptance tests above. Maybe the blue book is more detailed in this respect? (I don't recall tbh)

"Features have to be right-sized": If you address individual methods or small components with your specs, I guess you will see that small features are no problem at all. Just the opposite imho: the smaller the better (but of course they still need to be features, i.e. relevant to the business!). I used this in the banking domain, and e.g. we tested each interest rate calculation method individually.

Or do you mean something other than "small" when you say "low-level"?

For large features, I agree with you, they are unsuitable because you'd simply get too many examples. So better trim down your specs to really small aspects, and then have lots of them. Then you can have more coarse-grained "integration" specs with very few "happy path" examples.

"Living documentation not printable": I think this also depends on the tool. I only know FitNesse, and there you can run all of your tests in one go, which produces a huge HTML page. I always dreamt of using this as printed documentation! Of course it looks a bit weird and ugly as-is, but that's nothing that couldn't be fixed by a decent HTML transformation step.

OK, that's my experience so far. HTH a little bit. And if others see things differently, I'm curious to learn from you :-)

Cheers,

Nicole

--
You received this message because you are subscribed to the Google Groups "Specification by Example" group.
To unsubscribe from this group and stop receiving emails from it, send an email to specificationbyex...@googlegroups.com.
To post to this group, send email to specificati...@googlegroups.com.
Visit this group at https://groups.google.com/group/specificationbyexample.
For more options, visit https://groups.google.com/d/optout.

[Bob Rundle]

Thanks very much Nicole.

I just started the blue book.  So hopefully I will soon see some daylight in the my current fog of SbE understanding.

Perhaps a significant part of my troubles in applying SbE is the toolset I am using.  I have to say I am very disappointed with SpecFlow.  I chose it because I have a Windows developer stack and SpecFlow integrates nicely into Visual Studio. But to require everyone working with the spec to have Visual Studio installed is a non-starter.  VS is a heavy weight toolset requiring GBs of disk space for installation.  It seems to me that an a key attribute of any tool whose primary objective is communication among all stakeholders is a small footprint so that peripheral stakeholders can participate just as easily as fully engaged stakeholders.  

I will give FitNesse a try.  I've avoided it so far because it seems to have nothing to offer the Windows developer.

BR

[Sergei Sergejev]

Hello Bob.

Don't put _too_ much effort into tools and remember that most important aspect of SbE and BDD is communication with client/stakeholder. If you start by putting up a test suite omitting discussing and agreeing the scenarios with someone on business side – you might introduce some unneeded complexity into a project. Unless you are also a business side of course :-)

Cheers!