Can You Over-Automate Your Testing?
For the longest time, test automation has been the Holy Grail of software practices. After so many years of regression testing, smoke testing, and other processes being done manually, the concept of setting up a tool or some program to do all the tedious work almost automatically sounded too good to be true.
And to a certain extent, it is. That is, it’s a pipe dream if you think you can just flip a switch and automate entire swaths of testing without lifting a finger. Automation is integral to today’s agile and DevOps world, but it’s easy to fall in love with it and lose track of what should and shouldn’t be automated.
Randy Rice, an author, speaker, and consultant in software testing and software quality, recently spoke to StickyMinds about test strategy creation, pointing toward test automation as something you need to keep an eye on before it gets out of hand.
“It's almost like we've reached a tipping point to where enough people are being successful with it that it's almost, I would say, maybe even to a dangerous point to where people are thinking it's the only way to do testing,” Rice explained. “What can be automated? What should be automated? What shouldn't be automated? And how do we carry that forth? What do we see as things that can be automated and how do we make sure that we don't lose in the process?”
Even if it can’t be used for everything, Rice expressed concern around the idea that automation would be relegated to simple regression tests. Although that is a part of what you should automate, load testing, performance testing, data-driven testing, and a medley of other procedures can also be streamlined through smart test automation.
But some functions need to be validated by actual humans. Some complex features and functions need that extra layer of scrutiny that a tool or machine just isn’t able to provide. Simple, repetitive processes are easier to automate, but that doesn’t mean we suddenly don’t have a need for human testers when things start escalating in complexity.
Testing has changed, and even if we over-automate in spots, we’re learning as we go. According to Rice, we’re on the right path, and we’ll only get better with more reps.
“It's been something we've been struggling with for many, many years. So I'm really glad to see that a lot of companies are getting a good grip on it. It's changing up how we think about testing.”
