The Cloud Is Metered
As we all know, the cloud has taken off in a big way. In particular, the main contenders like Amazon, Microsoft, and Google offer many services, and their businesses are growing.
For companies, running systems in the cloud is an interesting option. It is an easy way to obtain reliability and scalability, and individual departments can make their own arrangements as they see fit. Also, resources can be scaled up or down based on a project’s needs.
For testers, the cloud is a form of virtualization, allowing them to define images and to spin up machine instances with them. That way, various versions and configurations of a system can be tested—and even elaborate test cycles can be shortened because many machine instances can share the work.
Cloud use at this scale is relatively fresh, and we're still learning. One game-changer I see is that the cloud is metered; you pay by the hour, by the gigabyte, or by some other metric. The numbers might not necessarily be high, but they draw attention from managers. As testers we should look at these numbers as well.
The type of tests and their timing largely depend on business considerations, such as how often you want to deploy, what risks you are willing to take, and how much you want to invest. You can build and unit test every time a source file has been updated, but do you also want to configure and provision dozens or hundreds of machines to run higher level tests on multiple configurations that frequently?
The Action Based Testing (ABT) method, which is a modular and keyword driven approach, often results in well-organized and efficient tests. The modular approach facilitates decisions about which tests to run at each point in a DevOps pipeline.
Keep in mind that instances are often priced by the hour, so try to dispatch at least close to an hour's worth of testing work when you start an instance, not just a few minutes. And when provisioning is done with automation, verify scripts or recipes carefully, since bugs in them can needlessly cost money. The involvement of experienced cloud managers can help you use the cloud efficiently.
An interesting possibility is the "spot instance" in AWS (Amazon), and other vendors have similar models. To use spot instances, you specify a maximum price you are willing to pay, and if the spot price is lower, the capacity is allocated to you at that price. This could be used to execute tests in a round-robin fashion, to detect unforeseen issues. It will need tests to be organized as small individual modules, with well-configured images to start the instances.
Cloud-induced metering can be a game-changer. The cloud offers many opportunities, but metering requires attention. If you haven't done so yet—and you feel it is worth the trouble—it is smart to start taking a look at the costs of putting things in the cloud.
