Assemble an Efficient Mobile Device Farm to Maximize Your Testing
Sooner or later, mobile application testers encounter the challenge of how to assemble the most appropriate and efficient collection of mobile devices. You need to know which devices and operating systems are in demand, but you don’t want to have to maintain (and test) every mobile device on the market.
Here’s how you can set up the most relevant farm of mobile devices so that you can feel secure that your application will work correctly for the majority of your users.
First, you need general information about the popularity of operating systems. For iOS, I use statistics from Apple itself, as well as data from the research agency Apteligent and iOS developer David Smith. Most iOS users update quickly to the latest version of the operating system. In my company, we usually support only the current and most recent version of iOS. This allows the team to save time and reduces the number of necessary devices, considering Apple does not allow you to freely change the version of the OS on your devices.
To collect information about Android systems, I go to the developers dashboards and check out statistics from Apteligent and the crowdsourced data from OpenSignal. My team previously supported all versions of the Android OSes since Android 4.2, but due to the mass transition of manufacturers to Android 5.1, we decided recently to abandon devices that do not support Android 5.0. At the last Google I/O conference it was announced that they would make an effort to simplify and speed up the updating of old smartphones, which would mean a reduction in support for older versions.
An equally important consideration for testers, especially in the case of Android, is the device screen size. For iOS devices the maximum that you might need is four types of screens, but in the case of Android, the list can go on forever. The previously mentioned sources have information on the most common screen sizes so that you can prioritize devices accordingly. For a minimal Android farm, I think you need one device with an mdpi xlarge screen, a pair of hdpi or xhdpi normal screens, and one large screen.
Another concern is the device manufacturer. Companies can add their own custom firmware to Android devices, which can affect the functionality and stability of your application in different ways. In this regard, you should always pay attention to the brand of smartphone you buy. The previously mentioned sites and AppBrain have good statistics about device popularity. For maximum audience coverage, I recommend getting several devices from Samsung, one from LG or Sony, and a couple from Chinese manufacturers like Huawei, Lenovo, and Xiaomi.
Finally, I advise you to find out details about the audience of your particular application, because it may not correspond to the average picture. What are the age, gender, country, and lifestyle of your users? Collecting all these statistics and answering these necessary questions will let you adjust your list of devices in order to maximize your mobile app testing time—resulting in happier testers and happier users.
