Continuous Testing, Shifting Left, and Test Automation: Getting It Straight
Today’s executives face relentless pressure to deliver innovative software faster than competitors. Digital transformation has elevated software delivery to a C-level conversation, and software testing must be included in that conversation.
On one hand, testing is all too often the roadblock that stands between highly accelerated dev processes and highly automated ops-driven delivery processes. But on the other hand, testing is essential for ensuring that the release doesn’t place the business at risk—undermining the very customer experience that digital transformation is dedicating to enhancing.
How can you achieve the optimal balance between speed and risk to deliver engaging customer experiences faster than competitors?
This is where continuous testing comes in.
Continuous testing is the process of executing automated tests as part of the software delivery pipeline in order to obtain feedback on the business risks associated with a software release as rapidly as possible. It really boils down to providing the right feedback to the right stakeholder at the right time.
For decades, testing was traditionally deferred until the end of the cycle. At that point, testers would provide all sorts of important feedback … but nobody really wanted to hear it then. It was too late, and there was little the team could feasibly do except delay the release. With continuous testing, the focus is on providing actionable feedback to people who really care about it, at a time when they are truly prepared to act on it.
At some point the concept of continuous testing was inappropriately conflated with the “shift left” trend. But to deliver the right feedback to the right stakeholder at the right time, continuous testing needs to occur throughout the software delivery lifecycle—and even beyond that to production, like monitoring information from production and feeding that back into the quality perspective.
Just as the name indicates, continuous testing involves testing continuously. Simply starting and finishing testing earlier is not the goal.
So where does test automation fit in? It’s essential for continuous testing, but it’s not sufficient.
Test automation is designed to produce a set of pass/fail data points correlated to user stories or application requirements. Continuous testing, on the other hand, focuses on business risk and providing insight into whether the software can be released. Beyond test automation, continuous testing also involves practices such as aligning testing with your business risk, applying service virtualization and stateful test data management to stabilize testing for continuous integration, and performing exploratory testing to expose “big-block” issues early in each iteration.
When it comes to digital transformation, it’s not simply a matter of more tools or different tools. It requires a deeper change across people and processes as well as technologies.
