Is It Impossible to Perfectly Simulate Real-Life Testing Situations?
Allegedly, practice makes perfect. If you do something enough times outside of the real scenario—whether it be shooting a basketball, running a marathon, or effectively releasing a mobile application to millions of people—you should be well prepared for the real thing.
However, we’ve seen athletes who are workhorses in the practice gym fail to make the big shot in a championship game even after knocking it down thousands of times in a test environment. And to reference everyone’s favorite mobile application illustration, Pokémon Go, all the simulations and testing the team did before the phenomenon actually went live didn’t properly prepare them for the number of people accessing their content in the real world.
That brings us to the critical question at hand: Even with test automation, service virtualization, artificial intelligence, and the medley of other software innovations we’ve seen in the last few years, is it possible to accurately simulate real-life situations for your application within the myriad testing phases prelaunch?
We’re getting closer and closer, and we shouldn’t overlook that fact. But Randy Rice, an author, speaker, and consultant in software testing and software quality, recently spoke to StickyMinds about how we’re still on a quest to overcome the challenges of test strategy creation.
“It's the big challenge of where the real world meets the paper world. It's that classic thing of, wow, that really did look good on paper. Even it sounded good as an idea, but then executing on the idea is the real challenge,” Rice explained. “We thought we had a great strategy going into something, then once we started to design the test and even start to perform some of the test we realized, whoa. Wait a minute. It's not scalable, it's not workable. We've got to go back and re-think the strategy.”
We need to accept that we don’t know what we don’t know, and based on that fact, we must be prepared for failure once something goes from testing into the real world. So, what’s the solution? We fail, and we turn that failure into a better test strategy and a better product.
“If we allow ourselves the ability to fail and not see it as a totally destructive thing and allow ourselves in our planning the option of coming back and doing a restart, or two, or three perhaps...then it really makes it a planned part of your strategy,” Rice continued. “I know that sounds weird but to say, but part of our strategy is to fail. At least two or three times.”
We always shoot to release something that’s been properly tested and designed to thrive within our connected world. But perfectly simulating real-life scenarios is exceptionally difficult. Prepare to the best of your abilities, anticipate failure, and learn from these mistakes.
