The Top 5 Test Automation Mistakes
I’ve heard from hundreds of people about the biggest mistakes they’ve made in their pursuit of test automation success. Here are five of the most popular answers—and recommendations on how we can stop repeating these mistakes.
Automating Everything
Not all tests can or should be automated, so I’m surprised when I hear that teams are still being measured on the number of tests automated versus the overall value the automation is bringing to the team. A better way to approach this is to have a selection process to quickly determine when something should be considered for automation. When you are held to a meaningless metric, your creative license is essentially removed, and you may not be performing the most valuable tests.
Having an Unclear Strategy
If you don't have a clearly defined automation strategy, it will be harder to measure success across teams. Our strategy focuses on automating our build verification tests first, smoke tests second, then a "dealer's choice" of what makes the most sense for the specific solution. The key is that the first two are standard across teams, and more measurable and meaningful.
Not Knowing What to Automate
The goal should be to automate the right tests. In our selection process, we determine the build verification tests and smoke tests first and automatically make them candidates to automate. These are by far the most executed tests throughout a cycle for us, so this aligns well with our strategy. However, we also have clear criteria to ensure there is some thought process around determining whether the ROI makes automation worth it.
Thinking Automation Takes Too Much Time
Don’t get caught in the trap of thinking it takes less time to run a test manually than to automate it. Sometimes that’s the case, but be realistic with the number of times you run the test. Even though it takes only a few minutes to run it manually once, if you run it many times within a sprint, performing the test manually may exceed the time it takes to automate it. The key is to make sure the tests are meaningful to the goals of the project team and their results are provided quickly.
Treating Automation as a Separate Role
I see a lot of organizations use their automation engineers in a matrixed role where they only automate the tests a manual tester instructs them to. On the other hand, I've also seen automation engineers look down on traditional testing activities and refuse to do anything other than automate tests. Somehow, we decided that automation is so specialized that it’s more valuable than other kinds of testing. These engineers should still be intimately involved in the product they are testing.
A good tester will utilize all tools and technologies available to ensure their solutions are comprehensive and efficient. Test automation is a valuable process, and if you avoid these common mistakes, you’ll be able to take full advantage of its effectiveness.
Melissa Tondi is presenting the session Innovations in Mobile Testing: Expanding Your Test Plan at the 2016 Mobile Dev + Test conference. Learn more at https://mobiledevtest.techwell.com.
