Figuring Out Your Regression Testing Strategy
I have been working on a combo change for the last six months: a large refactor to help us integrate with a third-party sibling application, and a complete overhaul of how client-side authentication works. There are a handful of small change requests coming in, but for the most part we are done. We are scheduled to go to production in a couple of weeks. Today, the development team was asked what our regression testing strategy is.
This is a perfectly reasonable question, but a lot of people have a hard time answering it.
My regression testing strategy on this particular project is the aggregate of all testing we have done during the development cycle. There is no hardening sprint; there is no regression cycle. For us, testing is built into the development process.
We started this long project with the authentication changes. When we were developing these changes, we had a list of existing scenarios that needed to continue working when we were done, as well as a handful of new scenarios.
The development flow worked something like this: Change some code, see tests fail and figure out what happened, fix things, try it in the UI and see what no longer works, fix things, and so on. It was a “stumble, walk, stumble, walk” cadence that most people working in software development will be familiar with.
When we thought we were at a good point, this was more of a gut feeling than anything codified by acceptance criteria. We would run our Cypress test suite and a select section of our API tests. After this, we walked through the product as a customer would and used the software to try to discover problems our other tests hadn’t yet discovered.
Regression testing was built into our development process, so answering questions about our regression testing strategy was easy. Your project may not work this way.
Still, you will probably be asked about it at some point. Your regression testing strategy, just like mine, is the aggregate of everything that happens during a development cycle.
To answer the question about defining your strategy, clearly take a look at what your developers are doing. Are they writing unit tests? Are they writing API tests? Are they writing tests that run in the browser? When do all of these run? How much time do developers spend in the product?
If you have testers working in tandem with your development team, also think about how they are augmenting testing already done by developers, what they are covering, what they are not covering, and when their work is done in relation to a code change.
Distill all of this down to a couple of sentences that are understandable by someone not in a technical role.
Your regression strategy probably isn’t something you need to come up with on your own. This is a description of what you are currently doing to get ready for production. Analyze your development process, see what other additional testing happens, and figure out how everything meshes together. That is your strategy.
