Testing in Agile and DevOps: Where Are We Going?
It is interesting to see how much of the software industry is fashion-driven. This is often with good reason. New ideas and concepts vie for attention, and the good ones stick.
When looking at what the software market is currently talking about, the top item is DevOps and Continuous Integration/Deployment (CI/CD), which seems to be taking over some of the spotlight from agile and is now a widely accepted new normal.
If DevOps is a major goal, agile approaches are the way to get there. Multiple ways are possible; it is a context driven decision. Scrum is an important option, but the open source world is also teaching companies how processes can be organized. In particular, lifecycles based on tools like Git and Jenkins, with pull requests and continuous build strategies, are changing the way we think.
The service-oriented thinking is also making its mark. It lets us rethink what work should be done where, with the front end focusing on user experience and the back end on processing transactions and other data. Arguably the most important new trend is containers as a better option for server virtualization, but also exciting is the relatively new "serverless" approach offered by cloud vendors. In this model, you no longer rent a server, but you define and use services directly, where the cloud vendor will allocate sufficient resources to handle them.
For testing and automation this has consequences—generally good consequences. We follow a method called Action Based Testing (ABT) that defines tests as products that can be developed and managed like other artifacts. Tests are written with keyword-driven actions, which makes them easy to produce and share with non-technical project participants. There is also a two-way converter between BDD scenarios and the actions. ABT has a product-oriented approach and allows tests to be produced and executed within lifecycles like classical waterfall, sprints-oriented Scrum, and continuous approaches, like the distributed development that we know so well from open source projects.
Having a good design for tests and a good process to create and maintain them within an agile project also make it easier to combine UI and non-UI testing. There is a lot of testing that does not go through a UI (desktop or web), but we see the most movement in testing services—particular REST services—increasingly driven by interface description standards like WSDL or Swagger (now known as "OpenAPI").
REST is also taking hold in the Internet of Things world. Testing individual gadgets is a very local activity, like embedded software testing has been over time. The interesting part is when all devices, services, back end processing, containers, apps, Web UI, big data, AI, start working together as total and flexible products and solutions. For testers, the focus will shift even more to organizing tests well and making them deep and intelligent, allowing them to find unexpected issues in complex environments.
