Two Reasons Why Test Automation Projects Fail
Very few things catch the fancy of management as much as the compelling prospect of automating tests. Test automation, after being projected as a magic wand, often fails to live up to the expectations. The core question is—Why do test automation projects fail?
One of the likely reasons is the failure to recognize the difference between the aspects of testing that machines can do well and the aspects of testing only a skilled human can do. The permutations of tests that a human mind can come up with cannot all be programmed because the human brain is situation aware, whereas the utility of an automated script is as limited as the part that’s coded.
James Bach and Michael Bolton, in an insightful article, use the word checking to refer to what tools can do and testing to refer to what only skilled people can do. Unless the distinction between a test and a check is clear, the test automation efforts will tend to promise more and deliver less.
Another reason for failure can be attributed to more focus on how to do automation rather than first thoroughly addressing why automation is needed. The need to first address why for any business endeavour can be further understood from Simon Sinek's Golden Circle concept. Simply put, great business initiatives first address and communicate Why, How, and What—in that order.
In practical terms, test automation is done to serve a business need. The need can vary based on the context, but mostly it is to improve test efficiency, find bugs, and reduce complexity, among other potential things. These could arguably form the why of an automation activity. Once the why is clear and measurable, the how constitutes the overall planning and resources to achieve the why, and what is the tools, technology, and overall approach that complies with the how. That’s an inside-out approach.
A common mistake in automation efforts is placing too much upfront focus on what, such as the tools and technologies, without an adequate focus on getting a consensus on the core why. If the test automation is done without keeping a definitive eye on overall returns from the effort, it is more akin to addressing what first and then struggling to prove the benefits when the efforts are completed. That’s an outside-in approach, and many automation efforts fall into this trap.
Authors of the book Lessons Learned in Software Testing say, “Use automation when it advances the mission of testing. Evaluate your success at automation in terms of the extent to which it has helped you achieve your mission." It could thus be derived that comprehension of the testing mission should drive the core why of test automation. It can either help the test team be more productive or waste its precious resources.
Automation should not be treated as an end—rather it’s the means toward achieving the bigger test objectives.
Do you agree? Share your thoughts in the Comments below.
