Continuous Delivery Success Stories?
Jez Humble
Jez Humble
Adam Rosien
Jez Humble
Eric L
Jez Humble
Banos
Eric L
For Testers: Quicker access to changes for testing = quicker test validation
Tim Coote
Eric L
Tim Coote
Sacha Labourey
This is half a plug, so anybody allergic to this please feel free to
ignore ny post, but I think it really fits the topic very well.
Not sure if you are aware of the CloudBees PaaS (http://
www.cloudbees.com/). This is a JVM-only PaaS that covers the entire
application lifecycle, from development to production. Essentially,
you just need i) a laptop and ii) to register (for free). Once done,
you can host your code on CloudBees' source repos (SVN, Git), perform
continuous integration with a "Jenkins as a Service" (resources are
dynamically allocated for your builds and tests, by the minute, but
all of your state from previous compiles/builds is preserved, this is
really neat), store your binary artifacts in a repo and then, you can
also push your application into production (where it will
automatically scale, failover, etc.). Those steps can either be manual
or fully automated (in which case, a git/SVN commit will initiate a
Jenkins job for build/tests, which will then initiate a transparent
push of the application to production, which will be transparent to
users and preserve ongoing sessions/transactions). You can also add
nicely integrated 3rd party services (Web UI testing, static code
analysis, etc.) to your continuous process.
Obviously, not all of our users and customers are using this "extreme
agility" and those who tend to get the closest to this model are
mobile phone application companies, interested in fast&small
iterations and very fast feedback loops. We will soon publish a quick
case study on such a customer, I'll post the link here if you are
interested.
For our own development at CloudBees (i.e. the PaaS itself), we have
setup a CI-based solution that can automatically update services in a
dev environment (we have another one for testing) in the cloud. This
dev/test environment is a complete replica of the services running in
production, with real data, all of it is simply sitting behind a
different domain name. Since we are currently using EC2, we can wipe
out this dev/test account entirely, and re-create it from scratch from
our various codebases and repositories (Chef, etc.) through a CI/CD
process. The cloud element obviously adds a precious layer of
flexibility in this picture since it is possible - even for relatively
small (and cheap!) entities - to create complete clones of their
production environment, wipe them out, restart, etc. with no capital
expense or hardware/infrastructure setup required. It is my bet that
once fully mastered and "industrialized", those models will generate
so much competitive gains, that the rest of the industry will be
forced to move towards similar practices. A bit like inclusion of Open
Source components: 10-15y ago, many companies did not allow to
incorporate any FOSS code in their home grown applications (FUD
mostly); today, I would be doubtful if *any* company still refuses to
do so, given the very high cost of reimplementing everything from
scratch without leveraging those 3rd party FOSS components.
Cheers,
Sacha
CloudBees, Inc.
On Aug 19, 4:08 pm, Jez Humble <j...@jezhumble.net> wrote:
> The *Continuous Delivery* book has been a roaring success, and it just won
> the Jolt Award <http://drdobbs.com/joltawards/231500080?pgno=7>. Clearly the
> is a strong interest in addressing them. However the most common question I
> when talking about continuous delivery is: who has actually done this stuff
> successfully?
>
> The ideal of continuous delivery is that your software is always production
> ready, and that you can release the latest build to users on-demand at the
> push of a button. Not many people have achieved that, but many people have
> managed to improve their delivery process substantially.
>
> So I was hoping that people on this list have some stories - successful or
> otherwise - that they can share in public. In particular:
>
> * What finally forced you to make a change to what you were doing?
> * How did you justify doing the work?
> months".
> * How long did it take you to achieve a measurable change?
> * What problems did you have to overcome to achieve your goal?
> * What effect did it have on your organization?
> * A little bit about your organization: how big, what domain (healthcare,
> financial etc); whether it was a website, product, embedded; any particular
> constraints like Sarbanes-Oxley, ITIL and so forth.
>
> If you're happy for me to publish these on my continuous delivery blog,
> please say so. Anything that gets published will of course get full
> attribution.
>
> Unfortunately I can't promise much in return apart from the satisfaction of
> helping the community, but I do have a copy of the book to give out, which
> I'll post to one of the people who replies.
>
> Thanks for your support and for sharing your experiences!
>
> Jez.
>
> PS - If you're interested in a list of talks and articles on continuous
> .
>
> --
> Jez Humble
> Co-author, *Continuous Delivery <http://continuousdelivery.com/>*http://continuousdelivery.com/http://jezhumble.net/
Tomas Riha
Jez Humble
Den fredagen den 19:e augusti 2011 kl. 16:08:05 UTC+2 skrev Jez Humble:
Tomas Riha
Well our change in architecture and delivery model actually came first. With these changes we felt that we needed. Continuous regression testing in order to ever be able to deliver on that model. That's what I ment with CReg beeing our business case, it was not the business case that lead to the rearchitecture. So it was our business case for doing CD.
Tim dB
My story covers the success we had at a statutory body which acted as the stock market for a deregulated electricity market, pushing billions of dollars a year through its clearing house. We ran a large bespoke java based market management application as well as a dozen or so satellite systems and another large customized settlement product.
What were the main problems in your delivery process before you decided to make a change?
Prior to my starting with the company they had just inherited the market management application’s codebase (from memory it was a million odd lines) from a tardy vendor and on receipt found there was no way to build it. Just a big mess of files!
Added to this there were around a half dozen satellite systems, most of which couldn’t be built from scratch and some of which they didn’t even have the code for.
In short the whole enterprise system was stuck and unable to change in all but the smallest of ways. Testing was like pulling teeth and often couldn’t really be done until the system linked up with everything in production.
This had been bearable for the past year or so but major changes to legislation were coming and this status quo wouldn’t last.
What finally forced you to make a change to what you were doing?
The system was responsible for pushing billions of dollars of energy money through a clearing house, was in terrible shape and change was on the horizon. For me continuous delivery was the only safe and workable option other than handing in my resignation.
How did you justify doing the work?
My main justification was risk. The body had relatively easy access to justifiable money through fees so I didn’t have to push the cost savings barrow. What I think did prick the CEO’s ears was that this was the best way to engineer change without risk. He knew change was coming and didn’t want to be worrying about a system that would send a billion dollar bill to the wrong energy market participant.
What measurable changes did you achieve? For example, actual cost savings, or "We moved from releasing once every 6 months to once every 2 months".
Before CD the system took a month to release the smallest of changes to production. There had also been several failed attempts at creating test environments. Our poor sys admin chap threw his hands in the air on a couple of occasions and declared it impossible.
By the time I left we were releasing what I called portable environments on demand. One for each developer if they so wished. Releases to the heavyweight environments were happening on demand. We had offshore developers in Singapore contributing to the codebase, using webex and skype and the humble phone to talk to onshore users and then dropping their work into the pipeline.
How long did it take you to achieve a measurable change?
2 years to get to a point where I would have considered us 70% of the way there. We still had some issues in the heavyweight environments to resolve and the testing process was still struggling a little (though this was due more to non CD related issues). The settlement system didn’t play CD with the rest of us so change in this area was difficult. Around the java product however the ability to see change meant I could get my best guys onto fixing it and they refactored the entire product, dropping the line count by more than half and delivered a vastly superior product.
What problems did you have to overcome to achieve your goal?
Huge political opposition. It’s highly risky stuff and until you start delivering can take you through some pretty rocky moments! My first advice to people doing this stuff is start small. Get it working well in one area and prove the benefits and then move from there. Since trying to push this from a consultancy I would also say a huge barrier to CD is cost and the fact that the benefits easily dry up upon the smallest obstacle to “continuous”. When you’re competing with other consultancies that cost can be the difference between getting the work and not. When you’re working with financially constrained organizations you can often find yourself spending lots of cash to “almost” get CD before they pull the plug and you watch any semblance of CD fade into the jungle like some lost city.
What effect did it have on your organization?
Nothing short of transformational. Agile actually worked once we put these things in place. I’m a big believer in agile but it’s fragile and trust based and hence an easy target for politics. I’ve found that shortening the point between inception and delivery the best defence against politicking.
One of the classic comments from my somewhat challenging CEO was “I don’t know if I like this agile stuff but I sure do like the energy it injects into the group”. We were seeing quick results and the users, BA’s, devs and testers’ energy levels rose accordingly. I think in the end the CEO liked that agile stuff.