The Advantages of Utilizing Formal Test Design Techniques
When it comes to test design, there are those who firmly believe in the use of formal test design techniques and those who believe that those same techniques cause rigid thinking and limit creativity. I believe formal techniques have value as a basis for formal analysis and design as well as for creative thinking.
Formal test design techniques include the following:
Black box methods
- Equivalence partitioning
- Boundary value analysis
- Decision tables
- Cause and effects graphs
- State diagrams and tables
- Pair-based methods (orthogonal and combinatorial arrays)
- Classification trees
White box methods
- Statement coverage
- Decision/branch coverage
- Decision/condition coverage
- MCDC (Modified condition/decision coverage), multiple condition coverage
- Path coverage (cyclomatic complexity)
- Integration complexity (design predicates)
Two of the main advantages of both formal black and white box techniques are their consistency and repeatability. No matter who is doing the testing, there are set rules as to how to apply the techniques. This ensures that there is a minimal level of consistency in the test design. Some will say this limits a person’s creativity, but to some degree, this is not necessarily a bad thing.
I find it is important to ensure the tester comprehends the basics of test design before he goes venturing off into the world of creativity. I am all for creative testing, provided the tester has some idea of what he is doing, why he is doing it, and how to convey that to others who may come after him and have to reuse and maintain those tests. Over the years, I have found that people need a base from which to work, and formal test design techniques provide that base.
At certain stages of testing, such as component testing, formal methods help ensure a minimum level of testing is performed. This can help significantly reduce the amount of wasted efforts due to rework caused by defects later in the process. In testing logic structures, there is little room for overly creative test design. The techniques also help in the establishment of automated tests.
Formal techniques can also be used as an analytical tool. The very process of applying the techniques to an element can find problems with that element. For example, when applying equivalence and boundary techniques to variables, we often find problems in the definition of those variables. This analytical process can be done both formally on paper and in the head of the tester (in that case, using formal methods informally).
These techniques have the added advantage that when a tester picks up another tester’s test cases, she can at least understand the general intent of the tests without having to seek out the person who designed the tests. Of course, this also requires that there be a minimum level of documentation.
A general knowledge of formal techniques can serve as a basis for ascertaining an individual’s general knowledge and skill level when it comes to testing, both formal and creative.
