Economics of Continuous Delivery

[denisk]

Hi Guys,

I'm in the throes of battling for more frequent deployments. Actually,
maintaining the existing monthly deployments. And I'm looking for good
economic arguments.

I've developed a little model, which I thought was right, until it was
disputed by one the guys in my team, whose model I've also included.

So what I'd like to ask you guys is if you have any comment on my
model, or have a better model.

At the moment, I'm conflicted as to whether there is an economic
justification for CD.

Anyway, more details can be found here -> http://wp.me/p5uyz-2r

Hope it make sense, if not, happy to discuss further.

regards,
dk-

[Vasco Figueira]

Hi,

It seems to me from the post that:

1. The cost of your release is still high. Would the objections still hold if the release/deploy was one click away? If it hurts, tackle it first. Automate it away so it is not a reason not to release/deploy. Sure you will have the effort now, but the whole idea is to not have it recurring.

2. The model is simplistic. It's ok to make a point, but I wouldn't rely on it as "the proof" to show up to management.

3. The suggestion that you "need to work out the return on a daily basis and multiply the return by the number of days from its release" seems to miss that the key difference is the time each feature waits until production (think "grace period", inverted). So what you should be comparing is the benefits of anticipating the compounding, which I think is what your model does.

4. There are other aspects (perhaps not so easily modelled) that influence overall ROI such as:
  * an easier remembrance of the changes made by the development team when an issue pops up in production;
  * the reduced time an identified bug lives in production (negative and compounded);
  * the ability for development to more closely follow product priorities;
  * the ability to get to the market with good timing;
  * the ability to test on the real arena whether a given feature works or not as expected.

Depending on your business, you don't *need* to deploy/release often. But it's a good thing to be able to do it when you want, and for doing that with low risk you need to do it often.

Regards,

[Adam Rosien]

The book "The Principles of Product Development Flow" is all about the economic arguments of CD. It's a great read.

[Serhiy Yevtushenko]

Hi

The point that I would like to make is that often the justification for decreasing frequency of releases lays in:

- high overhead of testing before the release

- concerns about stability of the environment, as non automated deployment has high probability of failures after releases.

From other viewpoint, increased length of the development cycle has non-linear effects on ability to hit deadlines (i.e., the bigger the length of the cycle, the harder it's predict what would be delivered), and the worse quality is usually observed.

Third point is that there should be distinction between external & internal releases, and frequency of external releases depends upon market & peculiarities of the industries). I have seen cases, when release attempt could be performed only on weekends/or 2 hour period once in a week.

Fourth point that is not considered is length of window of oppurtunity within particular industry. In finance, for example, it happens that market window for some product exists only for 2-3 month, and missing it would make all development effort for the feature pure loss. Therefore, the distrubution of cost/benefit and pay-off function is non-symmetric and non-linear.

Fifth point - i.e., the duration between releases is the period of unclarity of status for program-based development - this means, that for controlling program, regular release train is one of the first things in order to be bring program under control.

To summarize:

- if trying to argue to for preserving cadence of releases/going to continuous deployment I would argue basing on:

     - increase of stability brought by automation of testing & automation of release process (better quality of service)

    - decrease of overhead per release due to automation  (better use of highly qualified people (with high labor cost))

    - increase in predictability/controllability

    - if there are market benefits related to more frequent releases, then they should be taken into account as well.

The model that is present, does not take into account as well difference of cost of release activities, related to different cycle length (i.e., the cycle of testing/deployment has different costs depending on the length of development)

Serhiy 

2012/5/8 Vasco Figueira <vasco.f...@knowledgeworks.pt>

[Jez Humble]

Vasco makes some excellent points.

I also strongly recommend you get a copy of The Principles of Product Development Flow by Donald Reinertsen, which makes the economic case for smaller batches very strongly. You might say that the point of continuous delivery is to reduce the transaction cost of a release so you can work in smaller batches.

There are many benefits of working in smaller batches - here are some (partly culled from PPDF):

* It reduces cycle time. This in turn improves feedback. That in turn allows us to make sure we're always working on valuable stuff - the biggest source of waste in software development is stuff that we build that is never or rarely used (>50% according to a report by the Standish Group). Faster cycle times also have many other benefits, as Vasco mentions in 4) in his email.

* Reducing batch size reduces variability in flow - indeed, queuing theory shows us that if you want to increase throughput without changing capacity or supply, the cheapest and simplest way to do it is to reduce batch size.

* Large batch sizes lower motivation, and cause exponential cost and schedule growth, via a mechanism called the "large batch death spiral" whereby the longer the gap between releases, the more product owners tend to want to cram their favourite features into this release rather than waiting for the next one.

Actually quantifying these kinds of things is hard, but achievable - I'd love to see somebody try it. One idea inspired by the book I'm currently reading - Hubbard's "How to Measure Anything" - would be to do a Monte Carlo simulation. He has some example spreadsheets here: http://www.hubbardresearch.com/htma-downloads/ - it's on my to-do list to come up with a quantitative model, but probably not in the next few weeks.

Thanks,

Jez.

--
Jez Humble

Co-author, Continuous Delivery

http://continuousdelivery.com/

http://jezhumble.net/

[Denis Krizanovic]

Sorry for delay in responding. Toddler Flu hit the family.

You guys make some good points. I am familiar with PPDF, but I feel that I only have a soundbite in which to play this snippet. The people who need to consume the snippet are the people making the investment decision. I don't have time to coach them into seeing the world through the ideas presented in PPDF. I need to communicate in a fashion that they are comfortable with, ie spreadsheets/ROI, less with academic queue theory. 

For information, I calculated that transaction cost of delivering monthly is 8%. And this "re-testing" is primarily driven by the need for the team to become familiar with new functionality before release. Our accuracy rate into their environment sits at 95%, most problems are to do with manual changes, that are difficult to automate, mainframe/ibm-junk/proprietary this-that/etc. 

During this week, we've created another model, this time using the FutureValue function, And I've graphed it in the modelC tab of the spreadsheet -> https://docs.google.com/spreadsheet/ccc?key=0AmyFzoLK0lN5dHBvVXlkdDNIYm1NMHVXeW10TWdXSHc

I think this model shows a more palatable relationship between investment and frequency of release. Recognising that there are many factors that may even make this relationship even more exponential, re PPDF.

Would anyone else table this graph to their management? Does it hold to reason?

regards,

dk- [