Improving Requirements with Preemptive Testing
Requirements defects may account for most of the defect costs on a project because they are plentiful and expensive when detected late. IBM and Bell Labs studies show that 80 percent of all product defects are created during requirements definition. Many software problems begin with defective requirements that then breed defective designs, data, code, tests, and documentation.
The cost of a requirements defect discovered late, such as in production, is the total cost of:
- Implementation failures, ranging from annoying to life-critical
- Implementation defect isolation
- Patches or corrections to designs, data, code, tests, documentation, and requirements
- Change control and configuration management
- Reverification
- Redistribution
- Contributing cause analysis
- Hazard mitigation
But the cost of detecting the same defect early, during requirements development, is only the total cost of:
- Early detection
- Contributing cause analysis
- Hazard mitigation
To baseline your reality, perform causal analysis on post-development failures from several recent projects. If defective requirements are a major contributor, consider the following testing strategy.
Assign the responsibility for significantly reducing requirements problems to software testers. Their task is not to develop requirements, but to effectively identify requirements defects as they are being developed, and to identify mitigations for their contributing causes. In many organizations, no one “owns” requirements defect prevention and detection, but identified ownership is much more effective than project team ownership.
This added responsibility will require a focus on requirements quality as well as code quality, a broader understanding of requirements development, and a variety of new collaborative skills. It will also significantly increase the organizational value of testing, leverage the inherent strengths of skilled testers, and reduce requirements volatility. We call this strategy preemptive testing.
Even defect-free requirements can be a testing challenge. What if a product manager wants her product cycle time reduced? When you ask what “reduced” means, she answers a question with a question: What are the alternatives and price tags? Many interesting problems have this characteristic. Design alternatives and costs must be understood before a requirement can be made precise. Premature precision—essentially, guessing—often leads to unhappiness.
Negative and tacit requirements also present testing challenges. Struggling to develop effective test suites may reveal both requirements defects and challenges. Designating responsibilities for requirements quality through preemptive testing can help reduce the defects.
David Gelperin is presenting the session Testing Imprecise Requirements at STARWEST 2018, September 30–October 5 in Anaheim, California.
