How to Use Statistics to Become a Software Super Tester
Several blogs and other forums often talk about traits and qualifications of a good software tester. One such qualification that hasn’t been discussed much is the need for a tester to be a statistician. This does not mean a tester needs a degree in statistics. However, a core understanding of statistical principles and how they need to be leveraged in software testing goes a long way in not just giving a tester an edge but also positively helping the product's or application’s quality.
At a simplistic level, statistics is the science of learning from data and of measuring, controlling, and communicating uncertainty. Statistics touches various disciplines in many different ways. Software testing is one such major discipline it connects with, whether in analyzing the data from software tests or at a fundamental level in generating the software tests and optimizing the test effort.
Testers handle data on a day-to-day basis, and obviously data alone doesn’t mean anything. It is important to have base data, but the tester wearing the statistician’s hat has to draw meaningful inferences from such data in communicating his interpretations to the rest of the product team. In his post “Do Statistics Lie?” Shrini Kulkarni talks about how numbers by themselves are largely meaningless; but when context is added, numbers all of a sudden become very valuable.
The role of statistician in a tester’s life becomes more inevitable as he goes up the career ladder. This is especially true for test managers, as they increasingly have to draw inferences about product quality, predict outcomes based on the current status of defects and test results, and make important product decisions around the release.
Metrics management is becoming an important element in bringing objectivity into the testing process, which is essential in statistics. For instance, analyzing and interpreting defects data based on modules, priority, severity, experience from past releases, regression patterns, etc., go a long way in making decisions about whether to fix a bug at a given time and how that translates to product quality and the overall release cycle. Recently, specific trends—especially big data computing—call for an additional focus on statistical and analytical abilities to work through data across all disciplines in a product team.
While statistics seems to be more of a science, it has an artistic side too. Testing also is both a science and an art, and if you think of it as that combination, you should be able to more smoothly leverage statistical methods to make testing more meaningful in a software development lifecycle.
This can mean using statistics to project what past correlations will mean for future test cases. Lloyd Roden calls this being a weather person—able to predict outcomes using your analytical abilities. This should help you not just survive in your career—but also thrive as a tester.
