When I Really Do Break Software
I break software. For years, this has been the dinner-party version of what I do as a software tester. Of course, it’s not strictly true—the software is or isn’t broken before the tester ever lays eyes on it—but it’s a useful description that people understand, and unless they are testers themselves, they won’t call me on it.
Recently, however, I have been exploring the concept of actually breaking software: installing or operating respectable software in a way that renders it unusable. I think of it as the “super-sad path,” and it’s often relevant to enterprise and business-to-business software.
For many purposes, simple sad-path testing is enough. We think about invalid inputs and unexpected or interrupted workflows. We think about where the users might take a wrong turn, or where they might be using the product more zealously than we anticipate (such as longer hours or with more resources). This testing tends to speak to what the users are doing, and it finds bugs that could affect anyone.
The problems that arise from super-sad path bugs aren’t that cut and dried. They often stem from environmental issues such as complicated networks, security restrictions, or conflicts with third-party products. In these situations, we don’t know exactly what’s going on in any one user’s environment, but fortunately, the bug database can tell us about the kinds of problems that users experience, and we can work from there.
These are just some of the scenarios I have seen from our customers:
- Installing the software and housing resources under different user accounts, or with root privileges
- Providing invalid network configurations during software set-up or loss of connectivity
- Integrating problems from third-party products
- Running on unsupported operating systems
When we test against these kinds of issues, we are not necessarily looking for things we expect to be able to fix—and they’re not necessarily even defects—but they do present obstacles for our users. The end goal is usually to detect and communicate these problems to set the user back on the right track.
Breaking is an activity I would always reserve for software that is already stable and relatively healthy. (If it ain’t fixed, don’t break it!) But being open to testing beyond what your product is strictly responsible for can save your company many support cases—and, perhaps more importantly, the goodwill of your users.
Pamela Gillaspie will be presenting the session Improve Testing with a Zone Defense at STARWEST 2015, from September 27–October 2 in Anaheim, California.
