Top Down or Bottom Up? Designing Effective Test Automation
Even though many improvements have been made over time, maintenance of automated tests is still a challenge. Rapid cycles for continuous integration and continuous delivery mean demands on automated tests are higher than ever before. They have to be executed frequently, often in a variety of configurations, and they should be stable, because if the test breaks, the pipeline breaks.
Test automation, however, is not necessarily a technical challenge. The real focus is on the structure and design of the tests and their automation, in particular for tests that need to run through the UI.
Functional tests are usually sequences of what I call actions: operations on the application under test, and checks on the resulting behaviors and output. Such actions can be low-level, like “click button,” or higher-level, closer to the business of the application, like “verify payment.”
Just as with software, tests can be designed from the top down or from the bottom up. Which is better when it comes to test automation?
A typical bottom-up approach that I believe is not scalable for larger and more complex tests is to search for common sequences of low-level actions in existing tests, and then bundle them in higher-level actions. For example, the steps to log in to an application can be bundled into a “log-in” action that takes the user name and password as arguments.
Designing tests top-down may be better, but this is also challenging because of the diversity of items tests typically need to address. Many tests are created by nontechnical testers or domain experts, who do not necessarily have much experience in structured programming.
A more effective approach is to combine a top-down approach for test design with a bottom-up approach for the automation. This is the basis of a method aptly called action-based testing, which organizes tests in test modules and develops them using the earlier mentioned actions, typically in a spreadsheet format. The test modules form a prestructure that can be created in cooperation with testers, engineers, and domain experts. Testers can then populate these test modules later on with their tests, in the actions format. It is a top-down approach that can be understood and applied without being overly technical.
While for test design the approach is top-down with test modules, automation can follow a largely bottom-up process. Experienced automation engineers can identify and implement common actions that are likely to be needed, like for logging in or entering an address in a form. For webpages, page objects are a good starting point. These actions constitute a domain language for quickly implementing tests.
Together, the top-down test modules that support the test design and the bottom-up test language that supports the automation allow technical and nontechnical team members and stakeholders alike to create and understand tests that are both fully automated and flexible enough to deal with application changes and differing configurations.
