The Evolution of the DevOps Quality Management Office

The DevOps quality management office is a function to drive testing efficiencies in DevOps initiatives. It is composed of a set of processes, tools, and competencies to reduce test execution time between development and operations.

The prime goal is to optimize QA efficiencies upstream, downstream, and centrally, so it is basically playing the role of an accelerated bridge.

DevOps itself is still evolving as a movement, and as teams adopt it, they will see the need for increased continuous quality along with continuous delivery. Here are some critical guidelines to build this function.

A shift-left and shift-right assessment framework: This is a DevOps-centric automation readiness assessment. The primary objectives are extending the QA scope to the developer and operations world and shortening test cycle time. The output of the assessment is a transformation roadmap and business case. Below is a sample of a feasibility assessment:

A DevOps-centric automation readiness assessment

A tool competency framework for continuous delivery: This is an extension of the industry standard continuous integration methodology and is focused on automation of the development to operations lifecycle. Below are tools and test team competencies required for every component:

Component

Test team competency

Indicative Tools

Test-driven development

Proficient

JUnit, xUnit

Behavior-driven development

Proficient

Cucumber, JBehave

Configuration management

Aware

PVCS, VSS

Continuous integration

Aware

Jenkins, TFS

Build automation

Aware

Make, GNU Make, Apache

Test lifecycle and defect management

Excellent

QC, Rally, Jira

Service automation

Excellent

Parasoft, HP Service Test

Functional automation

Excellent

QTP, Selenium

Nonfunctional automation

Excellent

Performance, security

Continuous monitoring

Aware

Application, server, database monitors

Infrastructure as a code provisioning

Aware

Puppet, Chef

Model-based testing workbenches: Model-based testing is a methodology to accelerate the creation of test scripts. The tools let users describe the system under test using process models, which then automate the generation of test cases. The amount of testing effort can be reduced by as much as 30 percent.

Optimization: For test cases that are executed manually, scientific techniques such as orthogonal array testing and risk-based testing are effective to minimize the amount of test execution, with minimal sacrifice on coverage.

Estimation: The estimation function focuses on having a common widget of measurement across the QA function, which will be continuously improved.

Lean technique and agile adoption: Using lean and agile techniques such as kanban, value stream mapping, and a dependency structure matrix can minimize the development-to-deployment cycle.

Close alignment of key performance indicators: DevOps aims to reduce development to deployment time, and the DevOps quality management office (QMO) should have the same goal. Below is an example of how you would track key performance indicators:

DevOps KPI

QMO SLA

QMO KPI

Increase in number of releases

Reduction in test cycle time

Reduced cycle time due to:

  • Automation
  • Test case creation productivity
  • Lean adoption
  • Model-based testing
  • Environment provisioning

In a nutshell, speed is of the essence, and quality cannot be compromised. When you start delivering your DevOps quality management office initiative, the measurement starts instantly. Celebrate success and learn from mistakes.

Up Next

About the Author

TechWell Insights To Go

(* Required fields)

Get the latest stories delivered to your inbox every month.