Where Do Software Bugs Come From?
I was discussing the new program to guarantee bug-free software with a friend, an agile coach I respect. He pointed out that there are basically two places bugs come from: either a programmer screwed up, or there are missing, misunderstood, or misinterpreted requirements. According to my friend, the first group can be driven to near zero with modern techniques, while the second is “always ten years away.”
Keep in mind, this is some who takes software development seriously, as craft.
It makes me wonder if he’s ever opened up a bug tracker, looked at the last hundred bugs, and tried to categorize them. I have, and I’ve found a few more categories than just those two.
Platform problems: The programmer could implement the code correctly in Google Chrome, yet a different browser could interpret the code differently. For that matter, the new browser might not even exist yet; the code that works today could fail under the next version of the browser. There can always be problems when new devices are created. Applications that worked just fine on the desktop failed to translate to the first version of the iPhone, iPad, and other mobile devices.
In some cases, the very behavior we expect becomes obsolete between devices—you can’t right-click on an iPad, for example.
Unintended coupling: The most common form of this is code that is cut and pasted, then changed in only one place. You could argue a programmer screwed up, but the maintenance programmer involved did not know the second (pasted) set of code existed!
Security bugs: These are not missing, misunderstood, or misinterpreted requirements. Instead, security bugs are the presence of a feature that is not defined anywhere and should not exist.
Unintended consequences: Requirement A describes something to happen under some conditions, and requirement B describes something else. Together, A and B combine in interesting ways that no one thought about. Each requirement by itself was understood and interpreted correctly; it’s just that no one thought about what happens when you have (A and B) or perhaps (A and not B) or (not A and B).
Defect in a dependency: The code works fine, even for the values you tested on, but there are hidden combinations that do not work for a code library or web service you’re calling.
Black swans: You got all the performance requirements as right as possible, but then suddenly your website made the front page of The Wall Street Journal. Sometimes, unpredictable things happen.
If, as Jerry Weinberg put it so well, a tester is someone who knows things can be different, then a good tester can at least warn that these areas are unknown. A great tester might test for them, even when told not to worry about it.
Open up your bug trackers, look at your last hundred bugs, and see if there is anything to learn.
Then, let’s get to work.
