The Problem with Metrics
In modern knowledge work, no two tasks are exactly the same; no two teams work exactly the same together; and no two project contexts are quite like each other. This means that when we abstract these complex situations into numbers for metrics, they inherit all of that same complexity and imprecision. Now what looks like clear and simple numbers is actually a set of ranges and guesses.
To understand why people get frustrated with metrics, I want you to imagine you’re driving down the road and suddenly you see flashing lights behind you. You stop and the police officer comes up to the car and says “The speed limit is 60 kPH with a minimum speed of 45 kPH, but you are doing 30 kPH.” The officer asks why you are below the minimum? “It’s icy, I want to get home (deliver the value) safely (with quality).”
Would that feel fair (to receive a ticket for driving under the minimum speed)? When things are exactly the same (like speed in a car) we can make some judgements based on hard numbers, but in all other cases, those numbers show a very limited picture and they can feel completely unfair. In this case, due to the conditions we’re working in, doing the job with quality meant we had lower speed.
How Can We Use Metrics Effectively?
There are a few simple changes we can make to how we measure and what we measure that can change it from an unfair judgement into a tool for positive improvement.
1. Ask honest and curious questions about the metrics.
We can think of metrics a lot like the check engine light in our car. It doesn’t mean much by itself, but it indicates what we should look into it more. If a team’s burndown chart is flat, we want to ask the team if they’re struggling with something that is stopping them from completing work.
2. Look at trends over the raw numbers.
Even when the numbers are imprecise, the trends may tell us something meaningful. Is the team’s velocity erratic? Is the number of defects rising? These trends have relevance even if the exact numbers are hard to pin down.
3. Encourage the team to tell stories, supported by metrics, about what has happened.
Think of the difference between a team saying “We feel like our quality is better” and another team saying “See here where our number of defects starts going down? This is where we made the change to how we test our code.” If you’re starting this habit with a team, you might want to start with good stories. If the customer satisfaction score takes a dip for a week, focus the conversation on how they brought it back up – the problems will usually come out in the conversation without you asking.
Here is a quick summary of the Do’s and Don’ts to use your metrics more effectively:
As the saying goes: old habits die hard. If the team or team members are used to defending themselves from metrics, they’re going to be wary of any measurement. Take it in stride. Keep creating those good experiences and over time, those will win out over the bad ones.
Work with the team to discover what measures they think are valuable, and, particularly, improve their delivery. Measure that the team recommends are more likely to be accepted by the team and be effective.
If you want to learn more about using metrics to reflect trends back at the team that helps them achieve better performance, join us on May 6th in Vancouver at the ORGANIC agility conference. Register for the Team Coaching Workshop track >> Link