Designing Data-Driven Tests with Keywords for Automation Success
When automated tests are well-organized and written with the necessary detail, they can be very efficient and maintainable. But designing automated tests that deal with data can be particularly challenging. Tests need certain base data to be available and in a predictable state when they run.
Here, I share some common approaches I have had success with. These examples use the action-based test method, which I am partial to, but they are equally applicable for any method that uses keywords. These tests are organized in "test modules" that contain a sequence of test cases written with keyword-driven actions.
Brute Force or Arranged
The most thorough solution for base data is to follow a "brute force" physical approach—for example, using an "import" feature if the application has one or a database load command. Virtual machines and cloud options are increasingly providing solutions to plan and manage environments that can include base data.
If a brute force approach is not available, a more precise arrangement is needed for the creation, modification, and use of data. This requires rules like:
- Base data cannot be modified by a test.
- Base data has to be restored by that test before finishing. This can be a problem if a test doesn't finish properly, and it can help to have a "cleanup" module or script that removes or rolls back any leftover modifications.
- Base data has to be created by the test before use.
Real World Versus Constructed
Another question is whether to use data specifically constructed for a test or real-world production data. Constructed data can be more targeted and is, in fact, part of the test design. Real-world data is not necessarily under control of a tester, but it can expose bugs in an application that the tester wasn't looking for.
If real-world data is not predictable, a simple query strategy can help. For example, consider an action like:
You could substitute that with variables. Having an initialization action "get customer" will select a suitable customer (the argument could also be an SQL query) and assign the values for the retrieved customer to variables. In these examples, the variables are denoted with hash marks.
Data-Driven Testing
Data-driven testing is a popular use of data in action-based tests. A test will contain variables that are populated using rows in a data set, as pictured below. An action line with "create account" is executed multiple times, with the values for the arguments coming from a data set.
The data set would contain rows with values. The column headers are the names of the variables to be populated during the test execution.
Data sets like this can also contain filters that work only on selected rows, and they can be populated dynamically with run-time data, for example, coming from a database query.
How to successfully work with data, in particular base data, is an important part of a test design strategy. Combinations of these techniques and other methods can help make tests effective and easy to manage and maintain.
