Fine-Tuning and Expanding Your Mobile Test Plan
When mobile debuted, it became one of the bigger disruptors in the technology world. Many companies struggled to quickly go to market with mobile product offerings—and ended up sacrificing some engineering practices throughout the journey. As most of us have experienced, when new technology is introduced, test teams generally have a period of scrambling to figure out how to support those engineering efforts, without necessarily having the luxury of an increased budget for tools and additional staff.
A few years ago I was hired to build a mobile testing lab where we focused on three main components: developing people, processes, and tools to help companies decrease (and in some cases, eliminate) the disorganization period. It’s worth noting that in this case, the “lab” was the culmination of people, processes, and tools.
Once the people were hired, we quickly immersed ourselves in research and development to identify challenges companies would face when introducing mobile. Here are two:
- Transitioning traditional test team members to support mobile
- Continuing to stay efficient while supporting more testing permutations on devices
Once we identified the challenges, we got started on creating innovative ways to address them and help make test teams’ transitions smoother.
For the first, we spent many weeks decomposing devices and creating a list of testing types that we believed mobile introduced. We defined those types and then added them to our overall test plans. We created six categories of tests—peripheral, connection, gestures, interruption, syncing, and integration—and dozens of new tests to add to them, and we logically placed the more detailed tests into each of those categories.
We addressed the second challenge by talking to clients about how they selected which devices to acquire for their physical device lab. Most were simply purchasing the latest devices by manufacturer as they came to market, but that caused device maintenance and staffing challenges.
We determined which of the test types we created and defined were acceptable to test against simulators and emulators, and which test types should be tested against physical devices. We then fine-tuned our physical device lab by creating the device matrix technique: We made a list of categories for devices that allowed for a greater cross-section across multiple devices. We then matrixed those categories to a list of characteristics that were common across all devices. This allowed us to select key devices to test against, and remove some devices that were redundant. We were able to reduce the number of physical devices but still have a very high percentage of coverage.
By incorporating these three techniques, we were able to efficiently increase our device coverage while introducing minimal impact to the existing test team. This made for a smoother transition for everyone, without having to give up speed or quality.
Melissa Tondi is presenting the session Innovations in Mobile Testing: Expanding Your Test Plan at the 2016 Mobile Dev + Test Conference. Learn more at https://mobiledevtest.techwell.com.
