The Two Most Important Metrics for DevOps

In my earlier blogpost I discussed the purpose of DevOps, namely reducing the time and effort it takes to put software into production and allowing for more (continuous) production deployments. Why is this important? The typical answer is that in the digital age, SPEED is the name of the game and it is important to be able to increase the speed to market of IT solutions. Doesn’t everyone expect cool new features for their favorite software to be available on an on-going basis? Doesn’t “the business” require enhancements and new capabilities in business applications to be in production as quickly as possible? So great, DevOps (and Agile) should give us this speed to production. But how is success measured? Increased frequency of releases? Reduction in the amount of time it takes to perform a deployment? Reduction in the lead time from the start of a “project” to the production release of this project? Levels of automation? All these are certainly valid ways to measure progress in DevOps, but they are not the most critical “macro” metrics that tell us if we have improved SPEED in IT. These are  WIP and Throughput.

Let’s start with WIP. WIP stands for Work in Process and represents the sum of all the cost tied up in the entire IT department forall ongoing projects and feature development efforts that have been started but not yet been put in production. So, this is really all the capital tied up in the system at any given time trying to produce software. If a decision was made to halt all development activities, this number would represent all the investment dollars that would have to be written off. Any time a new project starts or any time effort is expended on existing projects WIP is increasing. Any time a project or feature set is put in production WIP is reduced by the amount of the investment made on this project or feature set. If a project is cancelled mid-stream, WIP is similarly reduced by a write-off of the corresponding investment.

Since WIP represents tied up capital, it isgood to have low WIP, and bad to have a lot of WIP. Increasing the release frequencies and thus allowing to have (parts of) projects put in production earlier, reduces WIP and thus frees up working capital. There is a long way to go, because most IT organizations have a lot of WIP, sometimes up to a full year of productive capacity of IT. Because of internal resource dependencies between projects, having a lot of WIP also causes poor throughput.

Throughput is the amount of WIP that is put in production in a given period, usually a month. If you have high throughput, you are not only putting a lot of projects or features in production in a given period, you can also start many new initiatives or projects without increasing WIP. So high throughput is good and low throughput is bad. Obviously, increasing release frequencies will increase throughput. Again, there is a long way to go, since most IT organizations have very low throughput (and high WIP) and if you have high WIP and low throughput, it will take a long time to produce anything.

Which now brings me back to the purpose of DevOps and Agile, which is to increase SPEED. Speed is gaged by looking at throughput and WIP. Measure these and you will know if your DevOps initiative is actually helping accomplish your goal or just another excuse to add a couple of cool tools to your inventory.