The Importance of Just Enough Software Measurement and Metrics
Is there a relationship between how good your measurements are and how good your software will be? Do better metrics mean better software? And does having no metrics automatically mean you’ll have bad software?
Let's imagine a world without measurements. I drive my car without a speedometer or oil pressure, temperature, and fuel gauges; when I go to the doctor, she pulls medications off the shelf without regard to the expiration date and administers them randomly; we build our homes using no blueprints or measuring tools. I could go on, but the point is, our world relies heavily on measurement and metrics, and without them, there would be chaos.
Yet, when it comes to the development, testing, and deployment of software, some argue that metrics have little value. Others take measurement to the extreme and have books of metrics without meaning or action, recording metrics just so they can check some box or meet stakeholder requests. In my experience, metrics are vital to the creation, evaluation, and delivery of software—but it’s most beneficial to have just enough metrics.
Testing is a measurement activity. We conduct both postive and negative tests to evaluate software and determine whether it meets the user's needs. In creating, analyzing, and delivering software, we want to understand the quality of the software being produced, the effectiveness of the evaluation methods (testing), the progress we are making against plans (status), and the resources (time, money, capacity, etc.) being consumed.
Measurements and metrics, at their best, are simple, easy to collect, easy to communicate, and focused on processes and products, not people. Further, these indicators serve as an early warning system that provides information to make us pause and ask why and whether any corrective action is necessary.
At their worst, measures and metrics do not exist, have poor operational definitions, measure the irrelevant, are difficult to collect and communicate, and are not linked to our key business goals. Worse, sometimes metrics are collected but not used to drive improvement. I once worked with a company in Germany that had a four-person team dedicated to the production of an eighty-page book of software engineering, testing, and quality metrics once every quarter. When I asked what positive actions and improvements had resulted from these extensive graphs, tables, and charts, no one could link the book to changes or results.
Having just enough measurements and metrics is important to the effective and efficient delivery of software. If you haven’t taken a look lately at the metrics you’re collecting, maybe it's time for an assessment of your measurement dashboard. Here are two great questions to start with: Can you easily link your metrics to business goals? And are beneficial actions being implemented from your measurements? If the answer to either is no, you should think about the amount of measurement you’re doing and whether it’s in the right areas.
Mike Sowers is presenting the tutorials Plan, Architect, and Implement Test Automation within the Lifecycle and Measurement and Metrics for Test Managers at Better Software West 2016.
