Continuous Testing: New Improvements on an Old Idea
There are lots of definitions on the web about what continuous testing is, and most relate to test automation. But from my experience, it’s much more than that.
I would say continuous testing is the practice of running static and dynamic tests across every activity in the software development lifecycle in order to uncover and fix unexpected behaviors as soon as they are injected. Testing starts as early as in requirements definition and occurs all the way into production, deployment, and beyond.
The whole point of testing an application continuously throughout the lifecycle is to remove defects as they happen. Ideally, we want to prevent defects instead of reacting to them, but when that is not possible, this is the second best option.
This way, defects aren’t able to progress farther down the development chain, when it is much harder to detect, reproduce, and remove them. For instance, it is easier to find a defect and fix it during unit testing than in system integration testing.
The concept behind continuous testing is far from new, but what’s different now is that software development practices have evolved to a point where developers are embracing testing as part of their activities, as in test-driven development and acceptance test-driven development. In addition, technology had to improve to support these new practices, which means testing is slowly moving from being an “event” or “cycle” to becoming an activity that all stakeholders across the development lifecycle build into their tasks.
Some believe that one day continuous testing will be properly adopted by everyone in the software development process, and with the appropriate practices and technologies, after all the sprint code is integrated, only integration smoke tests will have to be performed. The thinking goes that by using service virtualization, test data management, and model-based testing, team members will have found and removed all defects by the time the code is fully integrated at the end of the sprint.
That line of thinking won’t be acceptable to many teams at first. But just like continuous integration and continuous deployment slowly became the norm in software development, those more progressive views on continuous testing could also get traction organically as organizations start seeing positive results and benefits at a small scale. Then the risk tolerance will likely increase, or the perception around risk will change.
By then, we will have learned even more about continuous testing and will have even better success stories.
