Automation Face-Off: A Conversation with Michael Bolton, Isabel Evans, and Chris Loder
Automation is a pressing topic, constantly discussed and debated among software professionals working to keep up with the changing pace of software delivery.
In a recent STARCANADA keynote session, industry experts Michael Bolton, Isabel Evans, and Chris Loder took to the conference stage with moderator Melissa Benua to discuss some of the burning questions about automation from test professionals in the audience. Here’s a summary of their conversations.
_______
What does automation do well, and what do you like about automated testing?
Loder: The best things I do are the little things that make other people’s lives better. To me, that’s where automation’s best success is.
Bolton: Don’t automate something that isn’t worth doing in the first place. Tools can help amplify our superpowers, but tools don’t do things.
_______
How do you best prioritize what to automate first: either the quick, repetitive, and boring, or the longest, most time-consuming regression?
Loder: What hurts the most, and also, what’s the easiest to do? Try to find the happy medium.
Evans: It might not be test execution you need to tackle first. It might be reporting or the build process. Consider all the pain points throughout the development process.
Bolton: Most people don’t want to drive somewhere; they want to be somewhere. Before we ask ourselves what do we want to automate first, first ask, what do we want to get done first? Then answer, how can we get this done? Tools might play a role in that.
_______
How do you get started building an automation framework?
Loder: You don’t start by writing an automation framework. You have to start by finding out your pain points. Find out what hurts first, then fix that. If you need an automated framework, there are a lot of great frameworks out there.
Evans: Developers also have issues with their toolsets. We shouldn’t rush to code things, because it’s not healthy. It’s not just a testing problem.
_______
How do we test artificial intelligence?
Evans: Angie Jones said, “Just use plain old common sense.” That’s what makes us human. We know, as human beings, if we use empathy, we’ll know if it’s okay or not okay. I don’t think we’ll be able to use an AI to test another AI, because AI doesn’t have human empathy.
Bolton: Testing is all about looking for trouble. We need to remember that the machine was fed data by people. There’s always a human evaluation involved in it. AI has to be rooted in the effects it’s going to have on society.
_______
Where is automated validation—not verification, not testing—heading in the next five years?
Bolton: It’s going to be the same place it’s been for the last twenty years, unless we start talking about it differently. AI is a metaphor. These machines are cool, but they’re not really intelligent.
Evans: I think we’re a little way off from something that could validate in a human-like way. I think we need to be very careful about what we’re describing as intelligence.
_______
There were a lot of questions addressed, but we have to keep asking questions—or “looking for trouble,” as Bolton said. As testers, we have to always be asking questions, so we’ll continue to have these conversations.
