A Metric’s Risk Is Not a Property of the Metric

Two companies adopt the same metric. Call it revenue per employee, on-time delivery, or net promoter score. It does not matter which. Three years on, the first company is sharper for having used it. The second is worse off, though you would not learn that from its dashboard, where the number still reads a healthy green. The difference is in what people did to keep it green. In the first company, the metric focused good work. In the second, people learned to serve the number in place of the thing it was meant to represent: they cut the corners the metric could not see, they protected their own score when it worked against the customer, and they timed their effort to the measurement rather than to the result. A dashboard cannot warn you about any of that, because a gamed number and an earned number look identical on the screen. Same measure. Opposite outcome. If the metric itself were what mattered, one measure could not produce both.

By now, the warning is familiar. What gets measured gets managed, and what gets managed gets gamed. Goodhart first made the point in economics, and Marilyn Strathern compressed it into the line people quote: when a measure becomes a target, it stops being a good measure. Decades ago, Steven Kerr wrote the most honest paper on the subject and called it the folly of rewarding A while hoping for B. Plenty of leaders can recite some version of this, which has not stopped them from building the very structures it warns against. Microsoft ran a forced stack ranking of its people for years, and retired it only in 2013, long after the damage that model does was well documented. It was hardly alone. Knowing the warning turns out to be a different thing from being protected by it.

The warning does not help because it aims the fix at the wrong place. It puts the danger inside the number, so the natural response is to go looking for a better number: swap the metric, add a balancing metric, tune the target. That work rarely lands, for the reason this piece is about. The risk was never a property of the metric in isolation. A pulse oximeter is a precision instrument, and on most patients it reads true. Clip it to someone who has breathed carbon monoxide and it shows a calm, healthy number while the person suffocates, because a standard oximeter cannot distinguish oxygen-carrying blood from the carbon-monoxide kind. The device is not broken. It is faithfully measuring the one thing that has stopped telling the truth, trusted in a condition it was never built to read.

The same measure, two systems

A metric is not dangerous or safe on its own. It is inert or corrosive depending on what is arranged around it. The number is a passenger. The system it rides in does the driving.

I worked with a professional services firm that measured its people on billable hours, the way most such firms do. Hours are visible, easy to track, and tied directly to what the firm invoices, so the metric looks not just reasonable but responsible. The number itself was fine. What made it corrosive was everything built on top of it: bonuses rode on billable hours, an internal ranking rode on them, and nothing in that structure connected a person’s hours to whether the firm actually grew. People did the rational thing and protected their hours. A method that would have solved a client’s problem in two hours instead of six went unused, because six hours billed better.

To a Director whose own bonus rides on the same company-wide billable hours, this may not look like a problem at all. It may look like exactly what you want: more hours, more revenue. That reading holds only as far as the invoice. The client who slowly senses that a job was padded rarely confronts anyone. That client simply does not call again, and does not put the firm’s name forward to anyone else. Referrals and repeat business are where a professional services firm grows, and both of them run on a client’s trust that the firm is working the client’s interest ahead of its own. The metric that looked identical to revenue was quietly taxing the two things that generate most of the firm’s future revenue. The staff were not the problem. The metric was, and only because of the system wrapped around it. Move that same hours number into a firm that uses it to spot burnout and balance staffing, with nothing riding on it, and it does no harm at all. Same measure. Different system. Opposite behavior.

This is what the accounting researchers Willie Choi, Gary Hecht, and Bill Tayler named surrogation: the slow substitution of the measure for the goal it was only ever meant to represent. People stop managing the outcome and start managing the number, and they do it hardest when a single measure carries a real consequence. Donald Campbell put the same idea in sharper terms years earlier. The more a quantitative indicator is used to make high-stakes decisions, the more it will be gamed, and the more it will distort the very thing it was meant to track. Notice what both are saying. The corruption is not in the indicator. It is in the stakes around it.

The three dials

If the metric is the passenger, something else does the steering. In my work it comes down to three things arranged around any number, and you have to read all three before you can say a word about the risk.

The first is consequence: what rewards or punishments ride on this number. A metric with nothing attached is a thermometer, and a thermometer is a useful, honest instrument. The moment a bonus, a ranking, a promotion, or a public standing rides on it, you have changed what the number is for, and you have changed the behavior of everyone measured by it. This is where the most counterintuitive damage happens. When you bolt an external consequence onto work that people were already doing for internal reasons, you do not add motivation. You can replace it. Uri Gneezy and Aldo Rustichini watched a daycare begin fining parents for late pickups and saw lateness rise, because a fine reframed a broken promise as a service a parent could simply buy. Alfie Kohn spent a career on the larger pattern, that rewards attached to intrinsically motivated work tend to crowd out the motivation they were meant to strengthen. The metric did not do that. The consequence bolted to it did.

A consequence does not have to be formal to be real. Picture a company where no bonus touches sales at all, yet the leader reads the top earners aloud at every monthly meeting and lets the ranking hang in the room. Nothing material is attached to that number, and everyone measured by it still behaves as though something is, because being named in front of your peers is its own reward and its own punishment. Status is one of the oldest stakes there is, and publicity hands it to whatever number a leader chooses to tout. This is what qualifies the thermometer. A thermometer read aloud and ranked at the all-hands is no longer a neutral instrument, because the reading itself has become the attachment. Measurement drives attention, and attention is a consequence. A number does not need a bonus to change behavior. It only needs an audience.

The second is authority: the gap between who is held to a number and who can actually move it. Hold a person accountable for a result they do not substantially control and you do not get performance, you get anxiety and workarounds. Robert Austin, in his work on measuring and managing performance, showed how measuring an incomplete slice of someone’s job all but guarantees they will optimize the slice and starve the rest. W. Edwards Deming was blunter. Most of the variation you see in a person’s numbers is not the person at all, it is the system they work inside, common cause and not special cause, and punishing individuals for it is both unfair and useless. When accountability and control come apart, the number stops measuring performance and starts measuring exposure.

The third is workload and timing: the cadence of the measure against the clock of the real outcome. When a number has to show movement faster than the thing it stands for can truly move, you have built a machine for short-term behavior. Lisa Ordonez and her colleagues catalogued what aggressive targets and hard thresholds do: the narrowed focus, the distorted risk-taking near the line, the quiet erosion of everything not on the scorecard. I saw the whole set once on a sales floor measured by a monthly, publicly stack-ranked quota, in a business where the deals that mattered took a quarter or more to close. The word quota was not the problem. The rank, the thirty-day cadence, and the bar that rose every period were the problem. Reps learned to pull deals forward before customers were ready, to discount in order to manufacture a close before month end, and to stop helping each other, because on a ranking a colleague’s win is your loss. Take away the rank, lengthen the cadence, and stop ratcheting the target, and the same quota does a fraction of the harm.

Look back at those three dials. In every case, the metric never changed. The system around it did, and the behavior followed the system.

The misdiagnosis that costs money

This is not an academic distinction. It is the difference between spending money on the right problem and the wrong one, and it is where I watch organizations lose the most.

Because the risk lives in the system, the expensive error is misdiagnosis, and it runs in two directions. Treat a structural problem as a metric problem and you will swap the KPI, feel productive, and watch the same rot reappear in a new form six months later, because you changed the passenger and left the driver alone. Treat a metric problem as a people problem, which is the more common and more damaging version, and you will coach, retrain, or replace people while the structure keeps manufacturing the exact behavior you keep punishing them for.

The longer I do this work, the more certain I am that the visible problem is almost never the real one. The number that looks wrong, or the team that looks like it is gaming, is usually just the place where a structural problem finally ran out of room to hide. It is the symptom that reached the surface, not the condition underneath it. Deming warned that the figures that matter most are often unknown and unknowable, a hard thing to accept on a dashboard built to make everything look known and settled. The read you need is not on the screen. It lives in an arrangement the screen was never built to show.

The question worth asking

The discipline I would offer, in place of the reflex to hunt for better numbers, is to hunt instead for better outcomes. Stop asking whether a metric is good or bad. The question has no answer, because the same metric is both, depending on where it sits. Ask instead what system the number lives inside, and what behavior that arrangement invites. That shift, from judging the metric to reading the system, is the entire distance between complaining about numbers and diagnosing them.

I will be honest about where that leaves the easy part and the hard part, because the line between them is the whole game. Naming the risk a given metric invites is cheap now. You can do a fair amount of it from the number and its purpose alone. A risk invited is not a risk fired, though. Whether the risk a metric invites actually materializes, and whether the number is a real problem or only standing in for a structural one, is settled by your specific arrangement of consequence, authority, and workload, none of which can be read from the number, because it lives in the system the number cannot see. That harder read is the one organizations get wrong before they spend.

If you want to see the first half of that in action, I built a small tool called the Risk Read that names the behavioral risk a metric invites, in plain language, for free. It stops exactly where the number stops. The second half, telling a metric problem apart from the structure it may only be standing in for, and deciding what to do about it, is the conversation. That is not a withholding for its own sake. It is the honest boundary between what a number can tell you and what only your system can.

If you are about to swap a KPI, bolt on a balancing metric, or coach a team over a number that looks wrong, pause before you spend. You may be treating the passenger and leaving the driver alone. Reading which one you face, the metric or the structure it stands in for, is the work this piece has described, and it is worth a 45-minute conversation before the budget goes out the door.

The number is never the problem, and it is never the solution. The system it sits in is both.

---

References

1. Steven Kerr, “On the Folly of Rewarding A, While Hoping for B,” Academy of Management Journal, 1975.
2. Charles Goodhart and Marilyn Strathern, on measures that cease to be good measures once they become targets.
3. Donald T. Campbell, on the corruption of quantitative social indicators under stakes.
4. Willie Choi, Gary Hecht, and Bill Tayler, “Lost in Translation: The Effects of Incentive Compensation on Strategy Surrogation,” The Accounting Review, 2012.
5. Robert D. Austin, Measuring and Managing Performance in Organizations.
6. W. Edwards Deming, on common cause versus special cause variation and the limits of visible figures.
7. Lisa D. Ordonez, Maurice E. Schweitzer, Adam D. Galinsky, and Max H. Bazerman, “Goals Gone Wild: The Systematic Side Effects of Overprescribing Goal Setting,” Academy of Management Perspectives, 2009.
8. Uri Gneezy and Aldo Rustichini, “A Fine is a Price,” Journal of Legal Studies, 2000.
9. Alfie Kohn, Punished by Rewards.