Continuous Integration Maturity Model

submit to reddit StumbleUpon Delicious Save to Delicious
Update: Based on the outstanding feedback we received on this post, we wrote a white paper on the subject. You can get that here: Enterprise Continuous Integration Maturity Model. We also have a wall chart of the further developed model here: ECI Wall Chart / Flyer.

One of the great sessions of CITCON (Continuous Integration and Testing Conference) NA 2009, was a group effort to design a CI maturity model or road map. The group brainstormed up a number of things that could be included in a CI effort and the categorized them. Beginning CI steps were placed on the left, and advanced CI wizardry was placed on the right. Other items fell in the middle.

I'd also note that we definitely were discussing maturity with respect to CI tooling, rather than the maturity of a development team and how often they actually integrate their code.
Unfortunately, we then tried to prioritize items within their difficulty categories which led to the loss of some elegance. Photos of what we came up with are available up on Flicker. So I've recreated the basic picture, keeping the group's prioritization in mind below.



For the most part, I think the maturity model is pretty good. We could quibble about the difference between the Novice level deploying to "dev" test and Intermediate level deploying to testing environments, but the general trend that deploying to more environments is considered more mature is pretty good.

Personally, I would recommend looking at this effort as a very rough first crack at the question by a fairly diverse group of people. For some teams, items on the list could be completely irrelevant or even undesirable. That said, looking at this as a range of possibilities and as a rough gauge of progress should be helpful. If a team is doing most of the novice and intermediate stuff and a little of the advanced, they can probably safely place themselves at the intermediate mark.

Does this model look reasonable to you? Do you see anything in particular that's objectionable?

The original CITCON session notes are up on its wiki.



Re: Continuous Integration Maturity Model

You've laid out some interesting ideas here. I develop my own custom CI system and so I have used the table to evaluate where my system stands (still very novice, as you might expect). Here's my write-up:

http://multimedia.cx/eggs/continuous-integration-maturity-model/

There are a number of items I'm not clear on, including: Automated deploy to dev, Automated deployments to testing environments, On demand deployments to controlled environments, Data roll-up, Continuous deployment to prod, and basically anything else that mentioned "deploy". :-) I suspect these refer more to things like web services?

Re: Continuous Integration Maturity Model

Yes, I think that for the most part "deploy" items refer to web services or applications. I've also seen systems that "deploy" embedded software to a simulator and starts some automated tests against that simulator.

Most teams have some test equipment. I'd look at the deploy things as "gets my software to the test environments". Continuous deployment to production makes more sense for web applications, but could also mean things like keeping a "latest tested build" link on your website and automatically making the build of your software a customer would download be that latest build that passed tests.

Thinking BIG, what would it take to automatically update what ISO image you are stamping onto CDs?

Data-rollup is a nod towards moving away from just having information about "this build". How have the various pieces of information you collected changed over time for this project? How does one project compare to another project?

Re: Continuous Integration Maturity Model

At Urbancode, we're expanding on the idea of a CI maturity model with a CMCrossroads webcast featuring CITCON participants Jeffery Fredrick and Eric Minick.

Re: Continuous Integration Maturity Model

And we followed up the webcast with awhite paper.

Indentify?

Indentify sounds like a cool way to identify problem source code by adding a //TODO: comment and indenting the suspected problem code. Probably, it's just a typo though.

Enterprise CI Cultural Maturity

Today we offer a guest blog entry from Paul Julius. Paul extends the Enterprise Continuous Integration Maturity Model that Eric and I developed by bringing in an important missing element: Culture. As I started to use this model more and more, I realized ...

© 2006-2010 Urbancode, Inc.
Anthill, AnthillPro, and AnthillOS are trademarks of Urbancode, Inc.
All other trademarks are owned by their respective owners.
tel: (216) 858-9000 fax: i (216) 393-0006 email:info@urbancode.com