Listen Up, Testers: The Power of an Example for Context
My friend recently tweeted this:
Q: How many Context-Driven Testers does it take to change a light bulb?
A: It depends on context.
His simple question got more than two dozen replies, mostly asking questions. To make it a little harder, allow me to offer my spin on it: How long does it take a tester to change a light bulb?
Here are a few questions I came up with in a minute:
- Which light fixture?
- How far is it from here?
- With what light bulb?
- Do we have that light bulb in stock?
- How high up is the outlet?
- How tall is the tester?
- If the outlet is too high, are there other things the tester can use to reach it?
- Are those things readily available? Where?
- How quickly do you need an answer?
- What happens if I get the answer wrong?
- How accurate does the answer need to be?
- If I had more time to make a more accurate answer, should I take it?
- What additional questions should I be asking?
And, of course, the infinite regress: Can you tell me what you mean by “long,” “tester,” “change,” and “light bulb”?
All these questions help the tester explore the context—the things that make this situation different from others. Context really does matter; many a tester has told the boss testing would “take an hour,” but he meant “When the environment is available and after I spend a few hours on setup, then doing the testing will take an hour, plus perhaps an hour for documenting, and, of course, then we’ll have to do it over again if we find issues that need fixing.”
“About an hour” is an answer, but the second, longer answer explains what that hour means.
Imagine we are doing things a little differently. Instead of your asking questions and struggling to a solution, I start off this way:
Matt: “How long should it take for a tester to change a light bulb?”
Tester: “That’s a no-context question, Matt. Can you give me an example?”
Matt: “You are the tester. This is the bulb [hands it over] and this is the fixture [a lamp that is four feet high]. The timeframe is immediate.”
Suddenly it’s an easy question to answer, isn’t it?
The exact display after the conversion on the screen, the specific error message that should appear, the minimal length of time to return from a search, a sample output of results from a search within an existing database—all of these can cut the time we need to spend on context and reduce the chance we'll get it wrong.
The agile folks call it acceptance test-driven development—or sometimes specification by example, or behavior-driven development, or something else.
No matter what you call it, concrete examples can be very powerful. My biggest concern is when people get overly into the shiny automated tools and try to write the examples by themselves without collaborating to gain context.
The collaboration is the thing. The tools are the bonus.
Aleksis Tulonen, who posted the original light bulb tweet, is Finnish. As for collaboration in the tech community, though, we are only just getting started.
