7 Biases That Impact Testing
“The eye sees only what the mind is prepared to comprehend.” This quotation from French philosopher Henri Bergson is very relevant to the business of testing.
This idea could be applicable to all our roles—for example, as a tester validating an application, a quality assurance professional benchmarking testing processes, a strategist assessing organizations, or even the head of a testing organization. We tend to judge based on biases that are a result of our environment, background, culture, and experiences.
Here are seven biases that could be altering the impartiality in your testing and quality assurance practices.
Anchoring bias: In the world of benchmarking and testing assessments, QA assessors often get “anchored” on a few sets of traditional metrics and use them as a baseline. They ignore changes in business indicators and the maturing test organization landscape, which drastically alters those metrics.
Curse of knowledge: This causes testers to focus on the areas they are most knowledgeable in rather than evaluating areas that the user of the software would find most critical. For example, while testing financial applications, I viewed real-time rates as the most critical factor—until a trader on the floor informed me that daily rates are the ones most used in trading platforms.
Automation bias: With the proliferation of tools, there is a tendency to automate everything, which may not necessarily bring down testing time. The need for velocity should drive automation, not vice versa.
Risk aversion bias: Often, testers tend to build inflated regression testing suites instead of taking calculated risks on coverage. This results in unnecessarily large execution times.
Ethnicity bias: Thisrefers to the testing of software standards inherent in one's own ethnicity. For example, English readers assume that it is natural to scan a visual field from left to right and top to bottom, which is not true in all languages. And Japanese people do not place an X in a checkbox to indicate acceptance; to them, this indicates refusal.
Cognitive inertia: This is prevalent in business development. There is a tendency to sell the same testing service, particularly if sales targets have been historically met by selling that service. Market dynamics are fluid, and a compelling testing story a year ago might not make sense any more.
Hierarchy bias: In western society, it’s common for test professionals to approach senior management with different ideas, techniques, and viewpoints and, at times, even confront management. But in other cultures, there is a tendency to bow down to hierarchy. Senior leaders who nurture that hierarchy bias tend to create an environment that is detrimental to creativity and innovation.
The most difficult part of recognizing biases is honest introspection. Examine, identify, and, most importantly, accept your biases. Once you are able to do that, correcting them and making the right decisions will naturally follow.
