The Evolution of the DevOps Quality Management Office
The DevOps quality management office is a function to drive testing efficiencies in DevOps initiatives. It is composed of a set of processes, tools, and competencies to reduce test execution time between development and operations.
The prime goal is to optimize QA efficiencies upstream, downstream, and centrally, so it is basically playing the role of an accelerated bridge.
DevOps itself is still evolving as a movement, and as teams adopt it, they will see the need for increased continuous quality along with continuous delivery. Here are some critical guidelines to build this function.
A shift-left and shift-right assessment framework: This is a DevOps-centric automation readiness assessment. The primary objectives are extending the QA scope to the developer and operations world and shortening test cycle time. The output of the assessment is a transformation roadmap and business case. Below is a sample of a feasibility assessment:
A tool competency framework for continuous delivery: This is an extension of the industry standard continuous integration methodology and is focused on automation of the development to operations lifecycle. Below are tools and test team competencies required for every component:
Component | Test team competency | Indicative Tools |
Test-driven development | Proficient | JUnit, xUnit |
Behavior-driven development | Proficient | Cucumber, JBehave |
Configuration management | Aware | PVCS, VSS |
Continuous integration | Aware | Jenkins, TFS |
Build automation | Aware | Make, GNU Make, Apache |
Test lifecycle and defect management | Excellent | QC, Rally, Jira |
Service automation | Excellent | Parasoft, HP Service Test |
Functional automation | Excellent | QTP, Selenium |
Nonfunctional automation | Excellent | Performance, security |
Continuous monitoring | Aware | Application, server, database monitors |
Infrastructure as a code provisioning | Aware | Puppet, Chef |
Model-based testing workbenches: Model-based testing is a methodology to accelerate the creation of test scripts. The tools let users describe the system under test using process models, which then automate the generation of test cases. The amount of testing effort can be reduced by as much as 30 percent.
Optimization: For test cases that are executed manually, scientific techniques such as orthogonal array testing and risk-based testing are effective to minimize the amount of test execution, with minimal sacrifice on coverage.
Estimation: The estimation function focuses on having a common widget of measurement across the QA function, which will be continuously improved.
Lean technique and agile adoption: Using lean and agile techniques such as kanban, value stream mapping, and a dependency structure matrix can minimize the development-to-deployment cycle.
Close alignment of key performance indicators: DevOps aims to reduce development to deployment time, and the DevOps quality management office (QMO) should have the same goal. Below is an example of how you would track key performance indicators:
DevOps KPI | QMO SLA | QMO KPI |
Increase in number of releases | Reduction in test cycle time | Reduced cycle time due to:
|
In a nutshell, speed is of the essence, and quality cannot be compromised. When you start delivering your DevOps quality management office initiative, the measurement starts instantly. Celebrate success and learn from mistakes.