Test Teams Evaluation to Improve Effectiveness
Evaluating a test team member’s abilities is crucial in recognizing deficiencies within the team, and it provides an opportunity to outline a strategy to remedy the weaknesses.
It’s been said that there is no “i” in team; however, there is an “i” in failure. A failure at an individual level will most likely have negative repercussions at the team level.
Selecting the proper method for evaluating the test team’s abilities is vital. Two beneficial methods include utilizing the appropriate metrics along with appraising various skill sets.
One commonly used test team metric is measuring the number of defects found per release per tester. In reality, this metric is ineffective. It’s too broad in scope to identify the strengths or weaknesses of the test team or its individual members. Rather, this metric represents an aggregate performance of development, deployment, and test teams together with the software’s architecture complexity and software processes applied. A highly-skilled development team working with well-designed software architecture, performing flawless deployments, and employing effective processes should all contribute to a lower defect count. So this is in no way a reflection of an ineffective test team.
Metrics that more accurately reflect the test team’s abilities include:
Number of defects found by customers in combination with severity and likeliness to occur—A high percentage of critical defects discovered by customers indicate defects are escaping from the test team.
Incubation period of a reported defect—How long did the defect go undetected? A defect originally introduced in an early deployment into test but not found until a later deployment into test has a high incubation period. Ideally, defects should be detected as soon as they are introduced.
How often a defect’s severity is changed—A high percentage shows a poor understanding of the issue versus its relevance to the product usage.
Defects rejected due to working as designed—A high percentage indicates a lack of product/requirement understanding. This could also mean there are poorly documented requirements.
Defects that cannot be recreated by the developers and defects that need additional information from the reporter—A high percentage shows a poor ability to articulate the issue being observed and/or the steps necessary to recreate the problem.
Metrics aren’t the only measuring means; there are attributes that exemplify the abilities of a tester. Deficiency in any of the following can limit the effectiveness of your test team:
Professional Expertise—how well versed they are in testing tools, techniques and methodologies
Product and Industry Familiarity—how knowledgeable they are of the product under test and the industry in which the product resides
Skills Stacking—how proficient they are in the use of automation tools, development, and deployment
Planning—the ability to accurately estimate and design the test effort
Mentoring—a desire and ability to share, advise, and guide; wanting to raise the bar of all teammates
Constantly Improving—an interest in continuous process, product, and self-improvement
Autonomy—the ability to work on tasks by themselves with little input and help from others
Interpersonal and Communication Skills—how well they exhibit people skills; being approachable and diplomatic; working well with others; being able to admit fault when rightly deserved; communicating both written and orally for understanding and not out of obligation
Current Methodologies—how familiar they are with technologies that support modern software development
Profession Passion and Contribution—whether their passion resides in testing or are they using testing as a stepping stone position to becoming a developer; whether they contribute to and participate in test-related information, such as blogs, informational websites, online articles, and forums
If you don’t measure it, you can’t improve it. By using the proper techniques for evaluating your test team’s abilities you’ll be better situated to identify ways of improving the overall performance of the test team. If you identify weaknesses, you can bring them to the team member’s attention and work alongside the individual to devise a plan and timetable to remedy the shortcomings. A rising tide will raise all boats.
