Test Estimation in the Age of Agile and DevOps
You’re well on your way to transforming your testing team, embedding testing roles into agile teams and rapidly embracing DevOps. However, there remain several nagging challenges. One of them is how to rethink test planning and test estimation.
Gone are the days of using project planning software and work breakdown structures to define and estimate each category of work and the associated tasks. You’ve said goodbye to your tried-and-true rules that served you well in waterfall and learned that estimating in isolation is a recipe for failure.
Your new philosophy and culture requires timely and “good enough” estimates to encourage failing fast and adapting continuously. The focus is on delivering valuable increments of working software, so making and meeting estimates is obviously less important than delivered value. Yet, in some command-and-control environments, even though they are embracing agile, demands are still placed on teams for well-defined and accurate estimates.
Stating the obvious, we’ve never been very good at estimating, regardless of the methodology!
Here are some new rules and prerequisites for estimation:
- Empower the team to establish its own estimation approach. Many use story points, but this is simply a means for the team to secure a common understanding of what’s required (analysis, design, development, testing, and so forth) to implement the stated value. The definition of a story point from one agile team to another will vary, so don’t compare story points—or velocities, for that matter!
- From an estimation point of view, investing in well-defined stories that communicate the value for a given role or user is most helpful. What is less helpful are stories with dozens of tasks that lack the value definition for the intended user.
- Employ lightweight test specification techniques such as test-driven development (TDD) and behavior-driven development (BDD), as these approaches nicely complement the story value definition and encapsulate the validation of the delivered value.
- Include automation as part of the work to be done for the story, whether that’s automating testing, the movement of the code, or quality gates.
So, what are some strategies and approaches for estimating testing, in light of our new realities?
- Remember that our focus in waterfall was "How long will the overall test project take?" In agile and DevOps, this context changes to "How long will these stories take to test, and how can we quickly demonstrate that this change meets the users’ needs and move it forward through the quality gates?”
- In estimating the story, we still need to consider the effort required for test design, test implementation, test execution, and the effort to automate.
- Our estimation approach must be dynamic and responsive to changing requirements.
- The best test estimates are crowd-based, drawing on the wisdom of the entire team and leveraging all key stakeholders. Estimates should not be done in isolation.
Estimating testing in the contemporary world of agile and DevOps demands some new rules. We must embrace collaboration and focus on validating the delivered value and getting it to our end-users quickly.
