A Lean, Flexible Measurement Dashboard for Agile and DevOps
We ask many questions before, during, or near the end of our sprints and releases. When will it be started, and when will it be finished? What are the key milestones and criteria? Have they been met? At what rate are we progressing? Are we on track or behind? Is our software good enough? How do we know?
As our software development methods continue to evolve, we’re moving to a “continuous everything” world. A constant flow of ideas, requirements, and change requests drive continuous development, testing, deployment, and release. This demands that our monitoring and feedback systems also be near real-time so that at any given hour we can assess our progress and make timely adjustments as necessary.
This means our measurement and metrics dashboards need to be agile and responsive to our business needs.
In working with agile and DevOps teams, I’ve found five key measurement and metrics categories—what I call instruments on the dashboard—to be beneficial in providing the team with rapid feedback while keeping with the lean and flexible measurement requirement:
Value: This dimension provides information on our effectiveness in implementing and delivering stories. Measurement and metrics examples are running tested features (RTF) and team velocity and burndown.
Productivity: This helps us understand our efficiency based on capabilities (both human and technological) to deliver completed work. Examples are the ratio of time addressing technical debt or defects to delivering new features, and automation velocity—the time associated with implementing and maintaining automation.
Quality: Both functional quality, such as product, feature, function, or interface, and nonfunctional quality, such as compatibility, interoperability, performance, and usability, are important to take into account. Measurement and metrics examples are business value delivered (BVD) and defects.
Predictability: This gives information about our track record—our ability to deliver completed work. Examples are the percent of time our builds are red, yellow, or green, and the percent of time a given resource, such as the test environment, is available.
Issues: Rather than quantitative measurement, this category is usually expressed in qualitative statements of key concerns or risks. Examples are a critical skill or resource not being available, excessive technical debt, and lack of a specific set of information.
Placing a few metrics that are clearly tied to defined goals in each category helps link our measurements and metrics to our desired outcomes. Each measurement and metric must be operationally defined, linked to defined goals, and supported by specific questions to be answered. It’s likely that some measurements and metrics cut across categories and serve multiple purposes, which keeps the dashboard lightweight and responsive.
If you’re moving from a more traditional software development approach to agile and DevOps, or if you’ve already implemented agile and DevOps and are struggling with implementing metrics, consider reviewing, revising, and refining your measurements. Leave those that add no value behind and look at a monitoring and feedback system that includes the essential categories above.
