How do you improve DevOps and strengthen your operations? It all starts with measurement. You need to understand where you’re starting from and where you want to go.
When it comes to software delivery, there are different metrics development teams can use to measure and track performance. Teams need visibility into data to understand their strengths and weaknesses and how they can improve their DevOps capabilities.
That’s exactly why DORA created the four DORA metrics in DevOps. In this guide, we’re highlighting who DORA is, what the four DORA metrics are, and the pros and cons of using them.
What are DORA Metrics?
Before we outline the four key DORA metrics in DevOps, let’s cover a brief history lesson to understand where these metrics came from.
DevOps Research and Assessment (DORA) is a DevOps research team that Google acquired in 2018. DORA uses data-driven insights to deliver best practices in DevOps, with an emphasis on helping organizations develop and deliver software faster and better. Today, DORA continues to partner with the Google Cloud team to release DevOps research and reports to improve software delivery within organizations.
In their 2018 research report, Accelerate: The State of DevOps, DORA compiled insights from six years of research to identify four key metrics known as DORA metrics. You can use these four key metrics to measure the performance of a software development team and improve the efficiency and effectiveness of your DevOps operations overall.
We’ll take a closer look at the four metrics soon, but for now, here’s a high-level overview:
- Deployment frequency
- Lead time for changes
- Change failure rate
- Time to restore service
Since publishing the metrics, many organizations have started adopting and using them as the gold standard for software and DevOps teams.
What are the four key DORA metrics?
Let’s look at each of the four key DORA metrics in detail to understand how they can help you measure your team’s performance.
1. Deployment frequency
You might already be familiar with deployment frequency since it’s an essential metric in software production. Deployment frequency is about how frequently your organization or team deploys code changes to production. This ultimately reveals your team’s speed because it indicates how quickly your team delivers software. And while speed may be viewed in a positive light, it’s crucial to keep quality top of mind. Frequency matters, but you also want to deliver value to your users.
So, is there a right or wrong answer when it comes to deployment frequency? Not necessarily, but DORA qualified different deployment schedules in their 2018 report. In Accelerate: State of DevOps 2018, DORA suggested that elite performers are available on demand and commit to multiple deploys per day. High performers deploy between once per day and once per hour, while medium and low performers are between once per week and once per month. Don’t panic if you’re currently sitting in the low or medium-performer groups. There’s always room to improve and shift your way toward becoming an elite performer who delivers smaller code changes more frequently.
2. Lead time for changes
This is another metric that can be used to measure the speed of your team. Lead time for changes is defined as the amount of time it takes one commit to get into production. In other words, how long does it take to move from code commit to code running successfully in production?
You can calculate the lead time for changes by averaging the lead time for changes over a period of time for various commits. Calculating the mean is important because no two changes are the same and lead time will vary across different scopes and types of changes.
Why does this metric matter? It measures how quickly your team can respond to needs and fixes, which is crucial in the development world. Your team can better plan how much to commit to with an understanding of how long it takes to get your changes in production. And perhaps most importantly, this metric is essential for helping your customers. If your customer has an urgent bug that requires fixing, they likely won’t want to work with a team that will take weeks to deliver a fix versus a team that can get them back up and running within hours. A team that’s able to produce changes quickly will keep customers satisfied.
According to DORA’s research, elite performers have a lead time for changes that’s less than an hour. Talk about a quick turnaround! High performers turn around changes somewhere between one day and one week. Medium performers fall between one week and one month, while low performers take between one and six months.
If we go back to the customer who needs an urgent fix on their application, do you think they’re more likely to work with a high or low-performing team? While the answer might be based on many factors, it seems most likely that a customer would choose the quicker turnaround time and stick with the high-performing team.
3. Change failure rate
Next up is the change failure rate, or, simply stated, a measurement of the percentage of deployments that cause failures in production.
Deployment frequency was all about the speed of deploying code changes in production, and change failure rate emphasizes the quality of the changes being pushed to production. It’s important to note that a failure in production can be different depending on the software or application. A failure might be a rollback, patch, service outage, or degraded service. When using this metric, it’s essential to define what a failure is in your work for your team.
It goes without saying that you want to keep your change failure rates low. While it’s inevitable to avoid failures completely most of the time, you don’t want to lead to team or customer frustration. As you measure your losses, make it a team goal to learn from them so you can perform better the next time around.
DORA classifies elite, high, and medium performers at a 0-15% change failure rate and low performers at a 46-60% change failure rate. Diving into change failure rate even further, DORA reported that elite performers have seven times lower change failure rates than low performers.
4. Time to restore service
And finally, we have the time to restore service, also known as the time to recovery.
Let’s face it – service interruptions and outages aren’t ideal, but they do happen. While they might not always be avoidable, what’s important is how you respond to them. The time to recover or restore service measures how long it generally takes to restore service when an incident such as an unplanned outage occurs. It’s critical to recover and restore service as quickly as possible. The goal of optimizing time to recovery is to minimize downtime and prepare to diagnose and correct issues when they occur.
According to DORA, elite performers can recover in less than an hour. High and medium-performing groups take less than a day to restore service, while low performers can take anywhere between one week and one month to get back on track. Improving your time to recovery is a great way to impress your customers.
Why use DORA metrics?
So, why should you use DORA DevOps metrics? Sure, metrics and performance measurement are valuable, but what is it about DORA metrics that makes them uniquely reliable?
DORA has been researching and studying DevOps for years. They consistently and regularly publish their findings and insights to evolve and drive DevOps teams. DORA is, without a doubt, a well-known leader in the industry and its expertise is trustworthy and valuable. The four key metrics didn’t fall from thin air – they’re rooted in data-driven research.
Additionally, the DORA metrics will give you a broad understanding of your team’s delivery levels and capability. The metrics can be used to identify how you compare to competitors in your industry, and most importantly, they can help you better grow and take care of your team.
When you measure and track DORA metrics over time, you will be able to make well-informed decisions about process changes, team overheads, gaps to be filled, and your team’s strengths. These metrics should never be used as tools for criticism of your team but rather as data points that help you build an elite DevOps organization.
What are the pitfalls of DORA metrics?
DORA metrics are great tools to use, but as with any form of measurement, there are some considerations to keep in mind.
It’s challenging to use one set of metrics for different products and teams because no two products or teams are the same. Your product might be more complex than someone else’s. Your team might be three times smaller than another development team. Every team operates within its own context and circumstances, so it may be more challenging for certain teams to become an elite performing group.
Another consideration worth noting is that there’s more to the picture than the DORA metrics alone. Teams who perform in the elite or high category across the four DORA metrics may appear to be successful, but they could be having other issues that aren’t accounted for outside of these metrics. It’s important to remember that there’s a bigger picture beyond these measurements. They aren’t the be-all and end-all, so be sure to keep that in mind.
Recording your DORA metrics in Wrike
If you want to support your developers and product teams further, consider using a project management solution like Wrike to track your DORA metrics, assign tasks to the team, and manage the software development process in one centralized location.
Whether your teams follow Agile or another methodology, Wrike can help structure your software project management and make it more seamless. Get started with a free trial today.
