Automating a deployment pipeline on a timeline

[Thomas Bell]

I have been given the opportunity to lead a team of two engineers to automate our build pipeline and get us to a place of continuous delivery. While my direct supervisor is a great evangelist for continuous delivery my sense is that we are going to have about 2 months to start showing real value if we want to get continued investment and buy-in from the organization at large. I had a thought that in order to have a chance of showing real progress in 2 months we should condense our sprints to 1 week rather than our standard two weeks. My thought is that this will shorten our feedback loop and give us more opportunity for improvement more quickly. Is this a good idea? Any other suggestions on how to get value into the system as quickly as possible?

To explain our situation more thoroughly I am a software engineer and have been for about 5 years. Our current development structure is better than most we have a continuous integration process whereby we run unit tests on every push and only merge feature branches into mainline branches (currently staging, develop, and master) when a build is green and the code has been reviewed by at least one developer. Builds are run automatically in Jenkins CI and we have one click deployments from Jenkins to each of our environments which are currently: staging, develop, and production. Our biggest hurdle is that while some of our infrastructure is under version control it is far from most. We have a "devops" person who is supposed to support configuration management but for a long time he has been pulled in too many different directions to adequately maintain the configuration management especially if we wanted to provision and configure servers automatically. Myself and the other two engineers have limited experience with configuration management, but are sufficiently intelligent to adapt pretty quickly. 

I feel like getting the configuration management in a better place should be our number one priority so that we can automate our acceptance testing in the cloud. Is this right? Is there something else we should focus on instead?

To reiterate my main questions:

* Is it a good idea to condense our sprints to 1 week?

* Is there something other than configuration management that should come first?

Thanks in advance.

[Jon Arild Tørresdal]

In my experience the better you get at Continuous Delivery and the more frequent you deploy/release, the more frequent you want your iterations. Again, in my experience, the higher the frequency the more problematic a time boxed sprint type system becomes. Taking your example where you want more frequent feedback loops, so you shorten your iterations to 1 week. Lets say you do that successfully and then after some time you decide you want to shorten the feedback loop even more. Iterate every day? At this point in time you start to see that time boxed sprints becomes a harness holding you back and you start to look for options.

The more experienced you and your team become as Agile practitioners, the better you get at shaping your process to fit your needs. Don't let a process like Scrum or similar dictate/limit your deploy/release frequency/feedback loop.

So to answer your questions based on my experience:

Is it a good idea to condense our sprints to 1 week?

It is a good idea to shorten your feedback loop (as you said), so yes it probably is better for you to do 1 week instead of two. Saying that, look out for symptoms related to having a time boxed sprint and look toward general Lean principles for guidance. Also, the phrase "if it's painful, do it more often" might be applicable here, but that also means whatever problems/manual work you have every 2nd week today, you will get every week now - so automate.

To put it in perspective, the team I recently worked for ditched time boxed iterations all together in favour of a delivery pipeline based on features. They release the feature as soon as its ready, and don't wait around for a time boxed iteration to end. They now deploy to production multiple times a day, and release every feature when it's ready. 

As for you other question, you are the best to answer what's most painful for you. If your team spend lots of manpower on infrastructure changes that can be automated with configuration management, it should probably be prioritised. If you have lots of manual acceptance testing that now will change from every 2nd week to every week, you might be able to automate some of that testing. This is a typical area where more frequent feedback loops might add additional strains on your organisation, and you need to look for ways to change the way you work and interact. However, you might be in a situation where you can increase your feedback loop with more frequent deploys, without additional strains. Note that the process your in now will bring pain points to the surface more often, and you need to be prepared to solve them.

[Thomas Bell]

Jon, 

Thanks for the comprehensive response. We have begun our work and will be working on1 week sprints for the next 8 weeks. I will post our progress here as we go.

--
You received this message because you are subscribed to a topic in the Google Groups "Continuous Delivery" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/continuousdelivery/ZLyCBlTJshU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to continuousdeliv...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.