Make Sure Measuring Agile Metrics Is Really Leading to Improvement
Measurement is easy. But measuring the things that help you improve can be hard.
Organizations seek metrics to help understand how well they are doing relative to goals, and sometimes to evaluate teams and individuals for bonuses. Agile teams—or at least teams new to agile—want to see numbers that describe how well their process is doing. While it seems reasonable to ask what you should measure to see how well the team is doing agile, it's more important to understand the rationale behind the measuring.
"How are we doing?" is not an easy question for an agile team to answer, as there are many common misperceptions about what agile is. New agilists, in their effort to process all the details of an agile implementation, can lose track of the fact that the one core element of agile is adapting to change by inspecting and adapting your process.
The risk is that by picking the wrong metric and measuring teams and people against it for the purposes of evaluation could make it harder for the team to experiment. For example, if a metric involves how much of the backlog is estimated, you're assuming a deep backlog (as in many stories, not as in DEEP). So it's important to be cautious of metrics that lead to a specific way or working, lest you encourage people to optimize for things that hurt the team in a big-picture way.
To even begin to use a metric effectively, you need to understand why you care and what you plan to do with the results. One common reason to have metrics is to incorporate them into performance reviews and to calculate bonuses. Measuring performance is hard, as I learned from Rob Austin's book Measuring and Managing Performance in Organizations. It can be a challenge to devise a metric that leads to good results rather than specific behavior.
This is the key reason incentive pay can be tricky. Having a goal to measure against is good; that is the principle at the heart of agile's "inspect and adapt" mechanism. But the easy-to-quantify metrics may be counterproductive. In your quest to figure out how your team is doing with its new agile process, gathering data can be useful—as long as the data-gathering does not add significant overhead to your project or get in the way of delivering customer value.
Be sure to correlate the easy-to-quantify data with the information you get from the usual agile feedback points, such as retrospectives and reviews. Also remember that agile methods are about feedback and improvement, and agile feedback cycles are more about risk management and improvement than performance measurement.
At a high level, the thing that matters most for an agile project can be the most difficult to quantify. If you have happy stakeholders because you are delivering enough value in a timely and efficient manner, you're probably doing OK. If you are not, you need to do better. In either case, you eventually will want to quantify "enough," "timely," "efficient."
But don't let the desire to quantify your improvement get in the way of improving.
What metrics do you capture for your team, and how do you use them?
