Continuous Delivery
Published 01 February 2010The book that Dave Farley and I have been working on for nigh on four years, Continuous Delivery, is finally up as a rough cut on Safari. I'm also very proud to announce that it has recently been accepted into Martin Fowler's Signature Series. The book covers build and deployment automation, continuous integration, test automation, managing infrastructure and environments, configuration management, version control practices, data migration automation, and even governance. That's a lot of material, and read like this it seems like a bit of a grab-bag of activities that normally gets second billing when you're delivering software. However these turn out to be absolutely essential activities if you want to get high quality software into the hands of users as fast as possible, and then keep delivering them valuable new features1.
The book has two themes: automation and collaboration. Delivering software of any complexity involves people with a bunch of skills: testers, developers, and sysadmin / operations personnel. The last group of people often gets left out of the process of delivering software until the end, and even testers are often not as heavily involved in the development process as they should be. It's a pattern we see over and over again when helping people deliver software, and it inevitably leads to unacceptable numbers of defects, poor architectural decisions, and lengthy and unpredictable delays to getting releases out of the door. So one of the main aims of the book is getting everybody involved in the delivery process working together right from the beginning2.
Automation is key to enabling collaboration. If deploying software to testing environments, managing infrastructure, testing, building and releasing your software are manual activities, they are of course terribly error-prone. Worse, delivery teams then typically spend a large proportion of their time fire-fighting to get branches merged and new builds created so that they can be deployed into a production-like environment and tested. This inevitably means that the feedback loop from testing is incredibly slow, and doing things like load testing on staging environments gets left till late in the process, rather than being done from the beginning when problems can be fixed cheaply. Automating as much as possible of the delivery process ensures a much tighter feedback loop, and frees people to focus on high-value activities like evolving an appropriate architecture, exploratory testing, and making deployments and releases low-risk, push-button processes.
There is also an organizing principle for all of these activities: the deployment pipeline3. The description and elaboration of this pattern forms the core of the book. The idea behind the deployment pipeline is to model the part of your project's value stream that goes from check-in to release, and then to automate it. Every check-in triggers a new build, which then passes through the deployment pipeline as shown below (inspired by Chris Read).
The deployment pipeline allows everybody involved in delivering software to get fast feedback on the status of every change that is introduced to an application. It makes it simple to trace which builds have been through which environments, what the results were, and what's currently in each environment. Finally, it allows testers and operations people to self-service the build of their choice into the environment they control. The upshot is faster feedback, better collaboration within delivery teams, and - because each process from check-in to release is automated - fewer errors, more reliable releases, and shorter cycle times.
The book is still in Rough Cuts, which means we're actively seeking out feedback to help us improve it. Please let us know what you think: the book has a feedback section, and we've created a group to discuss the issues we cover in the book. We'll be posting outtakes, code examples, and further material from the book here, as well as providing regular updates on the state of the art in this space. We look forward to seeing you here!
1 We got the idea for the book's name from the first of the Twelve Principles behind the Agile Manifesto: "Our highest priority is to satisfy the customer through early and continuous delivery of valuable software".
2 There's a whole movement devoted to increasing collaboration between developers and operations types.
3 Dave, along with Mark Rickmeier, came up with the idea of deployment pipelines back in 2004. The idea then circulated within ThoughtWorks for some time, and was first documented publicly in 2005 (in a blog entry by our colleague Sam Newman). Dave and I have both written papers on the subject. More recently, there has been a new variation on the theme in the shape of continuous deployment.
