DORA Metrics: Measuring software delivery performance | LeanIX (2023)

The ultimate guide to

Find out what DORA stats are and why do we need them? Learn how to measure and improve DevOps performance for value stream management.

Entry

Digital transformation has turned any company into a software company, regardless of the industry it is part of. Companies are expected to react faster to changing customer needs, but on the other hand, to provide stable services to their customers. To meet these demands, DevOps teams and Lean practitioners must continuously improve.

DORA stats andFlow statisticsaddress this need by providing objective data to measure the performance of software delivery teams and drive product improvements.

Read on to find out what these stats are and how they can help youvalue stream managementattempt.

What are DORA stats?

DORA metrics are used by DevOps teams to measure their performance and determine whether they are "poor" or "elite". The four metrics used are deployment frequency (DF), change lead time (LT), mean time to recovery (MTTR), and change failure rate (CFR).

Delivery frequency:It refers to the frequency of successful production software releases.

Change handling time:Captures the time between committing a code change to a deployable state.

Average time to recovery:It measures the time between failure due to deployment or system failure and full recovery.

Change error indicator:Indicates how often team changes or fixes lead to bugs after code is deployed.

The acronym DORA stands for DevOps Research and Assessment Team. As part of the seven-year program, this Google research group analyzed DevOps practices and capabilities and was able to identify four key metrics to measure software development and delivery performance.

It has revolutionized the way DevOps teams work as these metrics provide visibility and real-time data that can be used as a basis for improvement and decision making.

To create this practical guide to DevOps, Google's research group analyzed data from over 32,000 DevOps professionals around the world - in addition to the official DevOps report, it also published the ROI whitepaper on DevOps transformation and the book "Accelerate: The Science of Lean Software and DevOps: Building and Scaling High-Performance Technology Organizations, co-authored by Nicole Forsgren, DORA team leader.

In the following sections, you'll learn more about the four DORA metrics and why they're so useful for value stream management.

Fulfillment frequency

As the name suggests,Fulfillment frequencyrefers to the frequency of successful software releases to production. In other words, it measures how often a company deploys code for a particular application.

The metric, which relates to the total number of deployments per day, is developed from production concepts that measure and control the batch size of inventory supplied by the company.

Of course, the more successful companies tend to make smaller and much more frequent deliveries – or in the DevOps world, more frequent but smaller deployments.

In general, the standard is one deployment per week, while a high-performing company publishes up to seven deployments per day. The standard number of deployments varies from product to product, of course.

For example, mobile apps that require customers to download the latest versionUpdate, usually a maximum of one or two releases per quarter, while a SaaS solution can be deployed several times a day.

Bron: 2019 Accelerate State of DevOps, Google

Once DevOps teams realize they fall into the underperforming category, they can install more automated processes when it comes to testing and validating new code, reducing the lag between bug fixes and delivery.

Change handling time

This metric measures the time it takes committed code to reach production. While deployment frequency measures the cadence of new code being released,Change handling timemeasure the speed of software delivery. Used to better understand the DevOps team's cycle time and how to handle request spikes. The shorter the turnaround time, the more efficiently the DevOps team deploys the code.

Two types of data (time stamps) are required to measure the turnaround time of changes: the exact time of approval and the exact time of implementation - in other words, the time from start to finish of the product; it is then used as an indicator of overall performance.

Bron: 2019 Accelerate State of DevOps, Google

When long lead times are detected, DevOps teams can implement more automated deployment and review processes and break down products and features into much more compact and manageable units.

Average recovery time

ZAverage recovery timeThe metric measures the time it takes to restore a service after a failure. No matter how well a DevOps team performs, unplanned downtime or incidents happen. And since failure is inevitable, it's really the time it takes to repair or restore a system or program that makes all the difference.

When a company has a short recovery time, management tends to feel more comfortable with sensible experimentation and innovation. This in turn creates a competitive advantage and improves the company's turnover.

The metric is important because it encourages engineers to build more robust systems. It is usually calculated by tracking the average time between a bug being reported and a fix being deployed.

Bron: 2019 Accelerate State of DevOps, Google

To improve their performance in terms of MTTR, DevOps teams must constantly monitor and prioritize corrective actions in the event of a failure. It is also useful to create an action plan for immediate response to an outage.

Change the error indicator

Bron: 2019 Accelerate State of DevOps, Google

To achieve a high average, teams need to reduce the number of failed deployments and time lost due to delays.

Benefits of tracking DORA stats

So why should all DevOps teams use DORA metrics? The answer is quite simple: if there is no data to measure performance, it is difficult or almost impossible to make improvements.

DORA metrics break down the abstract processes in software development and delivery and make them more tangible and visible so technical managers can take concrete steps to streamline processes and add value to software.

Below is an overview of the most convincing benefits of DORA indicators.

Decision

Companies that streamline the development and delivery process increase the value delivered by software and are more successful in the long run. By tracking performance with DORA metrics, DevOps teams can identify trends that drive informed decisions that lead to positive change.

Deliver value

in recent yearsvalue stream managementhas become an important part of software development. In this context, DORA metrics play a major role as they show what kind of value is being delivered to the customer and what level of performance is required to achieve the desired business results. So when DevOps teams use DORA metrics, they usually see value increase over time.

Didyscyclus

When performance is measured, there's a good chance it's a game. This means that people who feel responsible for a certain stat will adjust their behavior to improve the stats on their side. While this can be disruptive in various contexts, it has the desired effect in DevOps: it helps to eliminate inefficient processes and reduce waste.

The main challenges of DORA statistics

While DORA metrics are a great way for DevOps teams to measure and improve performance, the practice itself has its own challenges. For most companies, these four metrics are just a starting point and should be tailored to the context of each application, not a team or organization.

Below are four DORA challenges to keep in mind.

• Decentralized environment and data: Data is scattered across different sources in the IT environment.
• Data extraction: Data is only available in raw format.
• Data transformation: Data must be converted and combined into computable units.
• Speed ​​versus stability: Individual stats need to be put in context. A high change error rate may indicate poor quality control, while a high deployment frequency says nothing about code or product quality.

DORA measurements and value stream management

Earlier, we mentioned DORA metrics and their importance in value stream management. Today, more and more organizations not only use DORA metrics to streamline and optimize software development and delivery, but alsoValue stream managementto create end-to-end visibility of the entire production process.

Application

By far, DORA is the best way to visualize and measure the performance of engineering and DevOps teams. However, organizations should not stop there. To unlock the full value that software can bring to the customer, DORA metrics must be part of everyonevalue stream managementattempt.

Combining service catalogs, flexible scheduling and delivery platforms in oneplatform with LeanIX VSM, your software organization gets the holistic insight it needs to reduce waste and streamline software development and delivery.

Free poster | 17 metrics to help build better software