The Debate over Testing versus Checking
Michael Bolton began talking about something he called “testing and checking” around 2009. Testing and checking is a riff on the work sociologist Harry Collins described in two books, Tacit and Explicit Knowledge and The Shape of Actions.
In short, testing is what people do. Testing is social and hard to describe because we constantly change our decisions and judgments based on what we learn and our value system. Checking, on the other hand, is algorithmic—it answers the same questions in the same way every single time.
This distinction has helped many people understand why automation falls short of managers’ expectations. It has also been used as a blunt tool to signal affiliation with some imagined elite group of skilled testers—and a way to bludgeon people who use the terminology in a way some don’t agree with.
I’ve seen the latter on Twitter many times. The typical 140-character argument usually goes like this:
“Fantastic day! First time pairing and building tests with a developer.”
“Building tests? Do you mean unit tests running in CI?”
“Yes, we have a full automated test suite for this new feature.”
“That isn’t testing!”
One person is excited that they are able to exert influence in the development process as a tester by guiding test design and pairing on code. The other person jumps into the conversation purely to point out that someone is using language in a way they don’t like.
There is a strange plague in software testing of people who feel the need to put themselves in the role of a teacher and force other people into the role of the student. Teacher-student relationships are hard to define, and they can flow back and forth at any time. But there are contracts and rules of engagement built into them. The teacher doesn’t randomly jump into conversations to point out that a stranger is wrong while simultaneously removing that person's identity as a tester. Some people interpret the fact that these conversations are happening online and in person as a signal that anyone should jump in and debate.
Testing and checking, like a lot of language that develops in software testing, is there in order to more precisely describe something we are already doing. However, the fact that someone coined a new term doesn’t magically make people want to change language that has been used for decades.
If you want to understand what someone is telling you, it might be better to start by asking what they mean. Spend some time listening to what the other person is saying. After that, if you still care, maybe mention that you use different words to describe what they were telling you.
You can buy into the work of Bolton and Collins, and that is fine. Or you can buy into the work of sociologist and anthropologist Bruno Latour, who thinks tools are a natural extension of people and exert their will. Regardless of which ideas you think are right, using them to police other people's language is foolish. Our time is better spent discussing testing than arguing over terminology.
