Improving Application Testability
Automating functional testing is almost never easy. As testers, how we organize and design tests has a big impact on outcomes, but developers can—and should—have a role in making automation easier. This ease or lack of ease is part of what is known as "testability." A lot of testability can come from the design and architecture of an application. If a system has a clear structure with tiers and components with well-designed relations, tests can be focused and easier to automate and effectively include non-UI automation.
In addition to good application design and architecture, there are dedicated measures that can be taken to specifically help testing and automation. Such measures usually require very little effort but have a great deal to do with enhancing testability. Testability is—in my view—a policy matter. Everyone involved in the project should understand and agree on testability as a priority. When a plan is made for a system or a feature, one of the first questions should be "how do we test this?" Incorporating a testing and automation strategy early in the life cycle pays off down the line.
When testing via user interfaces, the test has to interact with elements that are meant for human interaction. A constant problem is that UIs are inherently volatile, as they usually undergo frequent changes and improvements. While UI testing tools provide mapping features that can identify screen elements using the properties—or attributes in HMTL—exposed by them, creating and maintaining mappings can be a pain and you need a working version of the application. However, the process can be made much easier when developers assign values for hidden identifying properties.
While the identifiers cannot be seen by human users, test tools can access them easily. Examples are the "name" of a UI control and the "id" in an HTML element. Other properties can also be used, such as "accessibility name" available in accessibility APIs or even custom made properties. Developers and testers can work together to define and share such values long before the UI is even built.
Another item to help testing is white-box access. User-facing systems often visualize data and events that are processed in an internal model; for example, a set of numbers is shown as a graph. Many tests do not need the user-facing representation. An application should therefore provide non-UI ways for a test to access—and even modify—data that otherwise would be hard to access and verify or trigger events that the application needs to be able to handle well. We have done game testing projects where we only could access the graphical representation. The automation would have been much simpler had there been an API call to answer questions, such as "where are the monsters”?
Another example of useful white-box access is timing. A test tool often needs to wait for an application’s operation to finish in order to complete the test. A common example is a table that needs to be populated with values before a test. The automation script can pause a preset amount of time, but this can lead to tests breaking if wait times are too short or become slow if they are too long. In particular on virtual machines, waiting times can be hard to predict. It’s better to actively wait for a condition to become true. If a condition is not readily available, developers can help adding one to the application. In the example of the table, they could set a "ready" property for the table that becomes true if the table is not loading.
Last but not least are emulation features. Operating systems like Android and iOS allow for location emulation to facilitate testing of location based scenarios. But an application can also have these features built in. An example is "time traveling." Let's say, in an ecommerce system, a discount might expire after 14 days. It’s helpful when a test can use a function to set a clock 14 or 15 days into the future without having to change the OS clock.
One thing to be careful about when providing information and functionality for testing purposes is security. It should not be possible for an outsider to call the functions or view the data that is meant for testing purposes. Consider using measures like encryption or a special "testing" mode.
Agile teams are particularly well suited for achieving testability, since there can be a natural cooperation between development, testing, and automation. This is even more true when a DevOps approach is followed. To achieve a smooth and automatic integration and deployment situation, efficient and stable automated testing can be a key contributor throughout the cycle. An example is the ability to redirect part of traffic through a new version of a component ("A/B testing")—something that is doable if it is addressed as part of development.
Close collaboration between testers and developers can improve the testability of an application, and in turn, provide better project outcomes.
