Automation: Testing or Checking?
Interactive exploratory testing and organized automated testing seem to be on opposing ends of a spectrum, but much of that depends on how you apply them.
Exploratory testing is an approach that emphasizes testing as a learning experience by an individual who has freedom and responsibility throughout the project. You can learn more about this in the writings of people like Cem Kaner, Michael Bolton, and James and Jon Bach, something I highly recommend. Even though exploratory testing describes a style with freedom and learning, it does not have to be unstructured. It can be well organized with charters and planned “sessions.” Exploratory testing just tries to avoid boxing the testers in.
On the other hand, automated tests are often viewed as shallow and boring. Some speakers differentiate between “testing” and “checking,” where checking is nothing more than verification of predefined checks, with low business value.
In my writing and speaking on automated testing, I not only recommend automating as many of the tests as possible but also organizing these tests into “test modules,” each with its own well-defined scope. One could wonder how much room this leaves for testers to do their magic, though.
The approach with test modules, however, does not box testers in either. It does not prescribe what or how to test. Test modules with their well-defined scopes can stimulate deeper and more creative testing. Testers are encouraged to create new modules if tests don’t fit existing ones.
It is true that automated testing can degrade into shallow “checking”—but only if you let it. If you work from a point of view of curiosity and are ready to learn from written sources and interaction with stakeholders, you can achieve an interesting and exploratory process for the domain that is defined by the scope of the test module you’re working on.
This domain might be the interaction with the UI of an application, or it might be the business process the application is supporting. This can be done without necessarily having to have the application under test in front of you. Michael Bolton was showing in one of his great classes how you can come up with interesting tests by just using a whiteboard. The exploratory approach was focusing on the domain rather than the system details. Maybe a term like domain exploratory test design could cover this.
Represent your profession as tester in your explorations. Apply testing techniques, and think of events that could make an application fail. Don’t make testing boring just because of automation. Explore, learn, and create good tests. And at the same time, for the sake of efficient automation and maintenance, try to keep it all organized.
Hans Buwalda is presenting the tutorials The Challenges of BIG Testing: Automation, Virtualization, Outsourcing, and More and Introducing Keyword-Driven Test Automation at STARWEST, from October 12–17, 2014.
