How Are You Managing Your Test Debt?
Debt does not always need to have a negative connotation. Just as debt can be good and bad in everyday life (such as a home mortgage), debt in the engineering world can also be good and bad. This applies to quality engineering as well—with good and bad test debt. As testers, how do we create a balance and stay at the right test-debt quotient?
With short sprints sometimes spanning just a few days, the test teams may have to pick up the slack. They are often unable to bring in the required test coverage. Yet given the business criticality of getting the product released, once the core acceptance criteria have been met, collectively discussed compromises are often made to go ahead with the release. While this may be good and acceptable in short doses, over a period of time this can soon become a fall back for inefficiencies in operations and lack of test coverage, leading to an adverse impact on product quality.
The debt could be due to a lack of test coverage or from deferring product bugs to subsequent milestones. If test coverage is lacking, some approaches that help minimize the debt include an increase in test automation (especially regression scenarios), ongoing bug bashes to bring in more exploratory focus, moving from N-1 to In-Sprint Automation in a continuous delivery mode, and allowing room for testing by providing more bandwidth whenever product milestones have flexibility—especially in the case of critical sprints. Also, the automation strategy should be very robust to scale and handle product changes with minimal maintenance. Given the tight timelines, a miss in the automation strategy can be a very expensive debt to take on.
On the defects side, testers are increasingly understanding the need to pick the right defects to focus on. While the ideal scenario is that all valid bugs need to be fixed, given the priorities at hand and the severity of open issues, the right choices need to be made. Others that don’t meet the cut will have to go into the "postponed" or "won’t fix" bucket. These are choices that the triage team has to carefully and collectively make, because postponed, valid defects can easily accrue in the debt bucket.
If the product team sees an accumulation of bugs that would really help make a difference once fixed, a conscious decision needs to be made to focus on a quality sprint that gets the backlog of defects taken care of instead of implementing new features. Addressing test debt not only helps the product and its quality once released, but it also motivates the testers that their work is important and gets attention, rather than just accumulating more debt. Process debt is another debt that testers may have to handle, but it has significantly been reduced over the years due to increased collaboration within engineering teams.
