Understanding an organisations’ DevOps KPIs is key to assessing progress. By providing access to DORA metrics on a per-team or service level, helps fuel the decision-making process, effectively optimizing from what you can successfully measure, rather than pure guesswork.
The book Accelerate by Nicole Forsgren, Jez Humble & Gene Kim provides a de-facto standard on how to measure development efficiency, supported by clear scientific findings. It identifies four key metrics that support software delivery performance, namely:
- Lead Time - how long does it take to go from code committed to code successfully running in production
- Deployment Frequency - how often your software team deploy changes to production
- Change Fail Percentage - what percentage of changes to production (software releases and configuration changes) fail
- Mean Time to Recovery (MTTR) - how long it takes to resolve an error or rollback a faulty change in production
By balancing tempo (Lead Time, Deployment Frequency) with stability metrics, it is ensured that increased speed does not result in suffering quality.
Don't let top-down KPIs ruin your engineering culture
KPIs are often related to control. Yet, you can only optimize what you can measure.
As a member of a product team: Get best-practice engineering KPIs to drive your quest towards maturity.
As a product leader: Objectify your engineering throughput & stability.
As an engineering leader: Identify, prioritize, track and communicate efficiency issues based on real-time metrics sourced from diverse software development tools.
Work in Progress!
Please reach out for any questions!
LeanIX Value Stream Management provides access to DORA metrics on a per-team level. The metrics are aggregated across all Software Artifacts owned by a team, but they can also be explored on a per Software Artifact level for a more detailed analysis.
Lead Time can be typically obtained from a Code Repository system such as GitHub. The Lead Time is here the time between the initial code commit into a branch to the merge of the resulting PR and the deployment into production.
The Deployment Frequency is measured by counting the number of deployments of a Service into product.
The simplest way to measure Change Fail Percentage is to look into your CI/CD builds, reporting on how often a build succeeds or fails. More sophisticated, if you capture information on regression in your support or ticketing system, it can be measured how often you are introducing a bug into production in relation to all deployments into production.
The ideal end-to-end way to obtain MTTR KPIs is from a support system like Zendesk, or an automated monitoring system such as PagerDuty. With its help, major bugs observed by customers or identified automatically can be tracked from initial reporting to complete resolution. Alternatively, recovery data can be fetched from systems like Jira as well.
Updated 4 months ago