Using Systems Thinking to Extend Your Test Automation Power
My first experiences with test automation were completely frustrating. I hammered automated GUI tests into perfect working order only to watch them break with the next build. Monday afternoon, a test flashed a happy green light. Tuesday morning, it turned an angry shade of red because a button had moved, a dialog box displayed slightly different text, or—worst of all—a performance hiccup caused a timeout.
It was 1997 and I was using SQA Robot to automate a native Windows 95 application, but my story will sound familiar to anyone who has struggled with GUI tests. Some challenges show no respect for advancing years or technologies.
Around the same time, one developer asked another, "Can we do this?" His companion replied, "We can do anything. We have the source code." That optimism made me stop and think. If only I had the source code, maybe I could do anything, too. Maybe I could design a stable mechanism to communicate with the application under test.
Then it hit me. SQA Robot spoke a dialect of Visual Basic that limited itself to features considered useful for automating GUIs, but it retained the ability to load arbitrary Windows libraries. That was my hook. I fired up Visual C++, built a simple DLL, and was able to execute its functions inside an ordinary test script. Just like that, I had the source code, and I could do anything. Stabilizing those frustrating tests was just the beginning.
This was not a unique discovery based on some accidental quirk of one automation tool. Many of the tools we use today allow us to extend their reach with a little custom code—and they make it much easier than SQA Robot did.
By their very nature, tools like Cucumber and Selenium WebDriver depend on a general-purpose programming language, so they make the most obvious examples. Nothing prevents a Cucumber setup routine written in Ruby from managing its test environment with Chef, or an ordinary test written in Python from intercepting a WebDriver exception and sending a screenshot to cloud storage.
Other tools invite us to extend and integrate them by providing APIs. For example, some project management systems, like JIRA, expose functions for reading and writing defect reports. A test script could leverage this to save execution time by running only if a particular report has been closed. If the test fails, it could reopen the report (with a helpful robotic comment, of course).
The key insight here is not that we can send a file to cloud storage or touch a defect report. It's much bigger. Our power extends beyond the features that our tools provide out of the box. We are a software shop. We have the source code. With some systems thinking and a little imagination, we can do anything.
Mike Duskis is presenting the session Integrate Your Test Automation Tools for More Power at STARWEST 2017, October 1–6 in Anaheim, California.
