When adding a feature, which comes first, the acceptance test or the feature?

[Dave Purrington]

What is the recommendation here? If I add the acceptance test first, the pipeline breaks. If I add the feature, I may not implement exactly what is required. Trying to do it all in one commit doesn't seem practical, because it may take me multiple commits to fully implement the feature. I could use a toggle to turn it on  when I commit the test. Or, I could also have the test toggle with the feature itself, only running when the feature is enabled.

What's the recommendation?

[Adam Rosien]

The last option seems the best to me, having both the acceptance test and feature both contingent upon the feature flag. 

[Jez Humble]

I've done both, and have a mild preference for doing the acceptance test after, although I am not religious about it (some of the ATDD crew are very religious and will say you must write the test first).

The important thing is that you can't say you're dev complete until you have a passing acceptance test.

If I add the acceptance test first, the pipeline breaks.

You should mark the test ignored so that it doesn't get run by the pipeline. Of course devs will still be able to run the test in their dev environment.

I wouldn't bother with putting in toggles just for this case unless you have the infrastructure all set up already - it's going to add unnecessary complexity.

Thanks,

Jez.

--
Jez Humble

Co-author, Continuous Delivery

http://continuousdelivery.com/

http://jezhumble.net/

[banoss]

Running the acceptance test before the system is ready is a little
cart before the horse. However to have the test early is a good
description of the intent. We have written the acceptance test
framework with the ability to run the test and report the result
without stopping the pipeline. Test metadata tags the test until the
behaviour is complete when the test becomes truly live.

The ability to run the test early is very useful, but the freedom to
see whats going on in the pipeline more so. So the test is not run in
the pipeline, but is run off-piste so to speak. Once everything is dev-
complete the test is promoted into the main pipeline. The risk here is
that you forget to promote your test. But this too can be automated.
And you'll need additional parallel resources to run the tests
against.

We have a custom AT framework to deal with the metadata to do this
kind of thing.