DevOps and Test Automation: You Missed a Spot
DevOps has taken center stage in our industry. In essence, DevOps is the term we use that means the entire process of developing, building, testing, and deploying/releasing becomes a target of development, orchestrated by automation. It is a logical step after agile, in which multidisciplinary product-oriented teams have taken over from the traditional functional teams, such as development and testing.
A complication can occur with test automation in sprints when the automation is not finished at the end of a sprint. Approaches like test-driven development and exploratory testing partially prevent that, but many of the functional tests and their automation might not be ready. However, in DevOps, it is very handy if automated tests are available for re-use after a sprint has finished. This makes them products of the sprint themselves and not just artifacts within the sprint. Tests also need to be available to run for different configurations and sometimes different versions of a system under test.
When tests are not "done" by the end of a sprint, they may need to be further developed and automated after the sprint. To show why this is not desirable, consider a painter painting your house. If you take a look while the work is being done, you may notice an uncovered area and say "did you miss a spot?" The painter will gladly fix that before the paint is dry. If you wait a few weeks, that same painter may not be that happy to fulfill the request anymore. Similarly, if developers are still working on the source code of a sprint, they will usually be more than happy to immediately fix issues. They will also be willing to provide testability support, such as hooks for timing, identifying properties for interface elements, and white box access to data.
There are several ways to get automation more ready in a sprint. Good design of both the system under test and the tests themselves helps a lot. In addition, test-driven development and acceptance test-driven development help keep test development and system development in sync. Testability support in an application can be another major contributor.
Certain test and automation related tasks should be kept out of application sprints, which should focus on the application in development and not on solving basic matters like failing automation technology. Such matters should be resolved prior to a sprint encountering them. Tooling and infrastructure should be available, as well.
The key success factor however, is the commitment from teams, managers, and other stakeholders. There should be agreement that tests and their automation can be important re-usable products, which need attention and cooperation to be able to support approaches like DevOps effectively.
