Introduction
Congratulations! You have set up a DevOps practice. Now, with the hard work done, you can sit back, relax, and witness the collaboration between your Dev and Ops teams as they deliver better quality software faster.
If only it were that easy.
Capabilities to Develop
As we look at today’s applications, microservices, and DevOps teams, we see leaders are tasked with supporting complex distributed applications using new technologies spread across systems in multiple locations. Because of this, the way we measure and understand critical services and applications has also changed. The emerging concepts of working with DevOps metrics and DevOps KPIs have really come a long way.
DevOps metrics to help you meet your DevOps goals
While DevOps is often referred to as “agile operations,” the widely quoted definition from Jez Humble, co-author of The DevOps Handbook, calls it “a cross-disciplinary community of practice dedicated to the study of building, evolving, and operating rapidly-changing resilient systems at scale.”
With this in mind, we begin to explore the next iteration of DevOps success. Your next challenge is ensuring your DevOps processes, pipelines, and tooling meet the intended goal. Like any IT or business project, you’ll need to track critical key metrics.
Here are nine key DevOps metrics and DevOps KPIs that will help you be successful.
The big four: DORA’s Four Keys
Let’s start with the four most common metrics Google’s DevOps Research and Assessment (DORA) team established known as “The Four Keys.” Through six years of research, the DORA team identified these four key metrics as those that indicate the performance of a DevOps team, ranking them from “low” to “elite,” where elite teams are twice as likely to meet or exceed their organizational performance goals. Let’s dive into how these metrics and DevOps KPIs can help your team perform better and deliver better code.
1. Deployment frequency
Deployment frequency measures how often a team successfully releases to production.
As more organizations adopt continuous integration/ continuous delivery (CI/CD), teams can release more frequently, often multiple times per day. A high deployment frequency helps organizations deliver bug fixes, improvements, and new features more quickly. It also means developers can receive valuable real-world feedback more quickly, which enables them to prioritize fixes and new features that will have the most impact. Deployment frequency measures both long-term and short-term efficiency. For example, by measuring deployment frequency daily or weekly, you can determine how efficiently your team is responding to process changes. Tracking deployment frequency over a longer period can indicate whether your deployment velocity is improving over time. It can also indicate any bottlenecks or service delays that need to be addressed.
2. Lead time for changes
Lead time for changes measures the amount of time it takes for committed code to get into production.
This metric is important for understanding how quickly your team responds to specific applicationrelated issues. Shorter lead times are generally better, but a longer lead time doesn’t always indicate an issue. It could just indicate a complex project that naturally takes more time. Lead time for changes helps teams understand how effective their processes are. To measure lead time for changes, you need to capture when the commit happened and when deployment happened. Two important ways to improve this metric are to implement quality assurance testing throughout multiple development environments and to automate testing and DevOps processes.
3. Change failure rate
Change failure rate measures the percentage of deployments that result in a failure in production that requires a bug fix or roll-back.
Change failure rate looks at how many deployments were attempted and how many of those deployments resulted in failures when released into production. This metric gauges the stability and efficiency of your DevOps processes. To calculate the change failure rate, you need the total count of deployments, and the ability to link them to incident reports resulting from bugs, labels on GitHub incidents, issue management systems, and so on. A change failure rate above 40% can indicate poor testing procedures, which means teams will need to make more changes than necessary, eroding efficiency.
4. Mean time to restore service
Mean time to restore (MTTR) service measures how long it takes an organization to recover from a failure in production.
In a world where 99.999% availability is the standard, measuring MTTR is a crucial practice to ensure resiliency and stability. In the case of unplanned outages or service degradations, MTTR helps teams understand what response processes need improvement. To measure MTTR, you need to know when an incident occurred and when it was effectively resolved. For a clearer picture, it’s also helpful to know what deployment resolved the incident and to analyze user experience data to understand whether service has been restored effectively.
For most systems, an optimum MTTR could be less than one hour while others have an MTTR of less than one day. Anything that takes more than a day could indicate poor alerting or poor monitoring and can result in a larger number of affected systems. To achieve quick MTTR metrics, deploy software in small increments to reduce risk and deploy automated monitoring solutions to preempt failure.
It takes more than four
DORA’s Four Keys make a good foundation to improve the performance of your development practices, but they are only a start. Here are five more to help your DevOps team perform more optimally.
5. Defect escape rate
Defect escape velocity measures the number of bugs that “escape” testing and are released into production.
6. Mean time to detect
Mean time to detect (MTTD) measures the average time between when an incident starts and when it’s discovered.
7. Percentage of code covered by automated tests
Percentage of code covered by automated tests measures the proportion of code subject to automated testing.
8. App availability
Application availability measures the proportion of time an application is fully functioning and accessible to meet end-user needs.
9. Application usage and traffic
Application usage and traffic monitors the number of users accessing your system and informs many other metrics, including system uptime.
Monitoring DevOps metrics for cloud resources and distributed systems
A successful DevOps practice requires teams to monitor a consistent and meaningful set of DevOps KPIs to ensure that processes, pipelines, and tooling meet the intended goal of delivering better software faster.
