The book Accelerate by Nicole Forsgren, Jez Humble & Gene Kim provides a de-facto standard on how to measure development efficiency, supported by clear scientific findings. It identifies four key metrics that support software delivery performance, namely:
- Lead Time - how long does it take to go from code committed to code successfully running in production
- Deployment Frequency - how often your software team deploy changes to production
- Change Fail Percentage - what percentage of changes to production (software releases and configuration changes) fail
- Mean Time to Recovery (MTTR) - how long it takes to resolve an error or rollback a faulty change in production
By balancing tempo (Lead Time, Deployment Frequency) with stability metrics, it is ensured that increased speed does not result in suffering quality.
Don't let top-down KPIs ruin your engineering culture
KPIs are often related to control. Yet, you can only optimize what you can measure.
How VSM can help
As a member of a product team: Get best-practice engineering KPIs to drive your quest towards maturity.
As a product leader: Objectify your engineering throughput & stability.
As an engineering leader: Identify, prioritize, track and communicate efficiency issues based on real-time metrics sourced from diverse software development tools.
From combined EAM/VSM customers, we get from time to time the question whether the DORA metrics can be made available in the EAM workspace for Enterprise Architects as well. Believing in empowered teams, and understanding the importance of trust, the answer to that ask is no. Why? We think context matters for efficiency metrics such as DORA, and we provide these primarily as a mean for the Engineering teams to optimize their efficiency - and not to other to measure team's efficiency. Please get in touch to discuss details and your use cases.
LeanIX Value Stream Management provides access to DORA metrics on a per-team level. The metrics are aggregated across all Software Artifacts owned by a team, but they can also be explored on a per Software Artifact level for a more detailed analysis.
Lead Time can be typically obtained from a Code Repository system such as GitHub. The Lead Time is here the time between the initial code commit into a branch to the merge of the resulting PR and the deployment into production.
The Deployment Frequency is measured by counting the number of deployments of a Software Artifact into product.
The simplest way to measure Change Fail Percentage is to look into your CI/CD builds, reporting on how often a build succeeds or fails. More sophisticated, if you capture information on regression in your support or ticketing system, it can be measured how often you are introducing a bug into production in relation to all deployments into production.
The ideal end-to-end way to obtain MTTR KPIs is from a support system like Zendesk, or an automated monitoring system such as PagerDuty. With its help, major bugs observed by customers or identified automatically can be tracked from initial reporting to complete resolution. Alternatively, recovery data can be fetched from systems like Jira as well.
Updated 4 months ago