Improving Test Automation—What About Existing Tests?
Whenever I talk or write about scalable test automation, I stress the importance of test design to make automation successful. There are two reasons why a good test design is important: it improves the quality of the tests, helping to add breadth and depth and it facilitates efficiency, in particular for automation.
Obviously this is all well and good when starting a project from scratch, but what do you do when tackling a project with existing tests?
Working with Test Modules
I suggest that you take the same approach as you would to begin a new project, and start with a “high level test design.” This will help you create an orderly flow of test cases that can be grouped into “test modules.” In our Action Based Testing (ABT) method, design and test modules are the central concepts, but you can apply the principal to whatever method you use. By defining an organization of tests into groups early, the individual tests can be more focused, which improves test manageability and maintainability.
An analogy I like to use to help everyone understand test modules is to think of cabinets in your home or garage. When you have a new cabinet or you’re moving, it is a good idea to label the shelves in the cabinet first, so you know where tools, paint, shoes, etc. need to go. Over time, you fill the shelves with the actual items.
It’s the same for test modules. Once they have been defined, you fill them with the test cases that fit the scope of the module.
Existing Tests
However, this is all easier said than done. For existing tests, you will probably want to make some managerial decisions. Most companies and teams are not in the position to upset and redesign all or a large part of their tests, but you can perform triage by asking questions: Are the tests worth saving? Are they in bad shape design-wise? What is their level of automation?
This will let you quickly recognize what needs to be done—whether that means doing nothing or discarding tests altogether.
Assuming it is worthwhile to salvage some or all of the existing tests, emulate a "clean slate" approach. Rather than trying to restructure existing tests, make a high-level design as if you’re starting from scratch. In other words, label the shelves of the closets. Then review the existing test cases one by one, and fit them into the new structure.
In the process of fitting existing test cases into a new structure, don't be afraid to make tough decisions. Some test cases may need to be split because they have an unclear scope. For example, if a test case has both low- and high-level checks, it is not advisable to keep those together in the same test module, as this will decrease the clarity and maintainability of the tests.
Add Business Tests
When you have an existing set of tests, a good question to ask yourself is “to what extent are there business-level tests here?” In many projects I've seen, most test cases are detailed interaction tests. However, those are the hardest to maintain and often do not represent many real-life business scenarios. Just like spring cleaning around the house, retiring tests that don't fit a new approach is not a bad thing.
Deciding when to improve a test set can be a difficult decision. If a project is ongoing there is often a lot of pressure and little interest in the restructuring of tests. However, in periods without pressure, improving tests may also not be a priority. Understanding the business value of good automated tests and being able to convince decision makers of their value can be key drivers for getting this improvement project done.
