Lower Risk of Downtime by Testing with Production Traffic
The accelerated development timeline of the DevOps model can bring unplanned outages, resulting in lost revenue and decreased productivity. The problem is how organizations typically do software testing.
Developers must continually release new application features; today’s sped-up development cycles require testing in days or even hours. This means bugs and problems are sometimes pushed through without the testing required. And that potentially leads to network downtime.
Developers also need to maintain many different third-party components to balance two opposing forces: Changes to a software component may introduce unexplained changes in the behavior of a network service, but failing to update components regularly can expose the software to flaws that could impact security or availability.
Bugs can lead to downtime and rollbacks. It typically costs four to five times as much to fix a software bug after release as it does to fix it during the design process. There are also the recovery costs of determining what caused the outage and then fixing it.
Software flaws are the gift that keeps on giving. They create immediate problems but can also lead to security issues further down the road. These flaws can be exploited later, particularly if they weren’t detected early on.
The automated tests triggered by code changes are looking for specific issues; they can’t know everything that could possibly go wrong. So then, things go wrong in production.
To overcome all this risk, software teams need a means of identifying potential bugs and security concerns prior to release—with speed and precision, and without the need to roll back or stage. Companies need a new approach that streamlines testing but protects against certain bugs and errors, increases productivity, and reduces downtime.
By simultaneously running live user traffic against the current software version and the proposed upgrade, users would see only the results generated by the current production software. Meanwhile, administrators would be able to see how the old and new configurations respond to actual usage.
This would allow teams to keep costs down while ensuring both quality and security, as well as the ability to meet delivery deadlines—which ultimately helps boost return on investment. And for developers, building and migrating application stacks to container and virtual environments would become more transparent during development, and more secure and available in production when testing and phasing in new software.
Developers who are able to test software updates with production traffic can verify upgrades and patches using real data. They can quickly report on differences in software versions, including content, metadata, and application behavior and performance. Software teams are able to investigate and debug issues faster using packet capture and logging, as well as encourage upgrades of commercial software by reducing risk and measuring performance benefits.
Fixing issues before releasing saves money, time, and reputation—all advocating for the practice of testing in production.
About the Author
Robert Ross is the founder and CTO of Curtail, Inc. Curtail is changing how IT is implemented for service providers, government agencies and enterprise organization that are developing and launching new software and services, particularly in DevOps environments. He also developed deception-based systems and high-speed network intrusion detection systems at Recourse Technologies, which was acquired by Symantec. He holds more than 15 patents in computer security, database and distributed systems technologies.