Reality-Driven Testing in Agile Projects
Make it real. That is the plain, simple message of reality-driven testing.
Some teams expend great amounts of energy constructing unit tests as part of test-driven development. Code chunks have dozens of excellent unit tests, which drive design decisions and give programmers prompt feedback about unexpected regression bug injections. Robust development also depends on solid unit tests. Unfortunately, to get a product to a potentially shippable state, these unit tests may be the only tests implemented by the team.
Agile teams can choose to implement example-driven, acceptance test-driven, or behavior-driven approaches. Business stakeholders identify and express acceptance tests for each requirement. A lot of effort is spent mapping acceptance criteria to a customer’s business needs. Teams collaborate with stakeholders to refine, improve, and disambiguate requirements. This represents a lot of work for very short sprints. Will the team have time to address other risks?
Often forgotten are agile tests based on reality. Three realities I use to identify meaningful test ideas are the technical work being implemented, the job that the user is performing, and the environment the product will run on.
Reality-driven testing tasks are identified early in the sprint during agile planning. This testing looks at real changes made to the code, the data, the schema, the architecture, the API, and any other element.
Technical work is related to testable objects. If module Fizzbin is modified to implement Tribbles, then the team would seek out other testable objects using module Fizzbin. How will changing Fizbin impact the Tribbles? How will changing Fizzbin impact other modules? Reviewing actual changes helps expose failure modes the team may wish to simulate in the sprint.
Reality-driven testers identify testing tasks by considering test ideas from the user’s perspective. Testers model the user and what users do to come up with relevant test ideas, often inviting members of the user community to agile planning sessions.
Consider everything a user can do. Do not limit yourself to what users are expected to do or what users are required to do. Users have been known to operate software in some strange, unexpected, and dramatic fashions, and this may lead to great testing ideas.
Lastly, reality-driven testers consider the target environment. This includes the platform, memory organization, operating systems, locales, network connection, and bandwidth. The environment also includes other applications operating at the same time as the system under test. Learn as much as you can about the target environment, and make it available to the agile team.
Reality-driven testing is not a complicated process. It involves asking questions that help agile teams make it real. The tests you run and the concerns you identify will have significance to your business, ultimately helping you focus on doing the right things well and in a timely fashion.
Reality-driven testing complements test-driven, example-driven, acceptance test-driven, and behavior-driven approaches, but it does not replace them.
Have fun making it real.
