Virtualization—A Tester's Dream?
Virtualization has been around for a long time. As early as the 1960s, IBM was supporting virtualization on mainframes to ease the cost of migration among multiple generations of their systems. Languages like Pascal, Java, and C# translate into virtual machine languages that are then either interpreted or further compiled ("just in time compilation") into actual machine code.
Shielding applications from the details of underlying configurations is still a driver for virtualization—in particular the use of virtual machines. This has the added advantage of providing more efficient physical hardware use. For testers, virtual machines can be a game changer. To what degree the game really changes depends largely on how an organization decides to work with virtual machines and how active the testers themselves are in recognizing and leveraging virtual machines’ possibilities.
Virtual machines often need to be requested from IT departments, essentially making a test team be at the mercy of IT’s request. A tester won’t be able to write a script to dynamically generate virtual machines from images, run tests on them, and bring them back down as effectively without access to the virtual machine. And an IT department might dimension virtual machines on physical machines based on manual use, but an automated test is typically more demanding than a human user.
Another issue is that it is often hard to predict timing behaviors of virtual machines. Automated tests often rely significantly on timing. Virtual machines might be swapped out during an operation, which then can take longer than the automation expects. We see tests break on virtual machines that previously were always running stably on physical machines. To address this, make sure to use "active timing," wait for something measurable, don't use hard-coded sleeps and allow a high maximum wait threshold for your tests.
Managing virtual machines and their images can become a job in itself—in particular when virtual machines are used at a large scale, such as to mimic the many configurations of the application under test. Work with a plan and have a designated person own it. Some questions to be answered include what configurations do you need that you will define and run virtual machines for, when to run them, and on what hardware.
Even though generous storage size is common on physical machines, the virtual machine images are large, and the amount of them should be controlled.
An exciting new wave of virtualization is "containers". These are processes that have their own pre-defined environment—file system, libraries, and settings. The containers have been made popular by the open source tool Docker, which allows them to be moved around and quickly instantiated and started on any machine. Organized this way, the containers form mini-virtual machines, with sizes measuring in megabytes rather than the gigabytes that full-blown virtual machines need.
The natural use of containers is running servers in a service-oriented architecture (SOA). The containers give operations a great tool to organize configurations and run instances of the system and its components. For testers, containers make it easier to build and maintain environments that are functionally equivalent with production. These benefits work even better if an "ambassador pattern" is used for the containers. It allows for easy replacement of major service providers, such as databases or external REST services, by mock-ups. When you use Action Based Testing, you can even consider controlling the behavior of the mock-up with actions. You could use them to let the mock-up capture requests coming from the process under test and to generate predefined replies. For more on DevOps and testing, read "Testing In the New World of DevOps" in LogiGear Magazine.
Virtual machines and containers can offer a great deal of opportunities to testers, in particular for the operations side of their work. A well thought through approach, with the support of all involved, can be very helpful in achieving these benefits.
