Data-Driven Testing Skills in an Agile and DevOps World
Data plays a critical role in a continuous testing, continuous integration, and continuous deployment environment, so it’s becoming increasingly useful in an agile and DevOps world. There are at least a couple of dimensions of data-driven testing to think about in our faster, better, cheaper software development and delivery models.
First, we must test data in the context of our applications and systems under test. An element of our test strategy must include approaches to ensure that the data used within our applications and systems yields accurate results in support of the business. Because our data sets are getting larger and more complex, this is a growing challenge.
Second, we need to employ data in planning, designing, and executing testing. The short cycle times in bringing new features to market demand that we leverage as much fact-based information as we can to focus our testing and manage risks.
In this digital age, testing requires those in testing roles—whether traditional testers or anyone else working on the testing team—to be excellent forensic detectives and employ analytics in their critical thinking.
So, if the use of data and information is increasingly critical in managing risks, what are some ways to actualize this requirement?
Testers need to understand if the application or system being tested processes big data. This requires us to think about what functional and nonfunctional tests are necessary and what information is needed to verify the data accuracy, conformity, data duplication, consistency, validity, and completeness for batch, real-time, and interactive data.
The data test strategy also must include performance testing so that we can determine things like how fast the data can be consumed and whether the queries and data transformation functions meet the speed requirements. Architectural testing is also necessary. Is the application and the data architected in a manner that accelerates performance or degrades it?
To achieve all this, we need data and analytical skills, which leads me to the second point. Some organizations are hiring or creating roles for data engineers—software engineers who design, build, and integrate data from various resources and manage big data—and data scientists—analysts who apply statistics, machine learning, and analytic approaches to solve critical business problems—within their testing teams. Testers are extending or retooling their skills toward those of data analysts and business intelligence developers as a means of increasing test effectiveness.
For agile and DevOps, an understanding of data analytics, machine learning, and AI combined with the necessary automation are helping teams accelerate development, testing, and deployments. As we continue to enhance our testing effectiveness, data analytics skills are an important dimension in managing risks in a “continuous everything” world.
