August 2024

|

Performance

Metrics That Obscure Rather Than Reveal

Dashboards can create the illusion of management without its substance.

The proliferation of business intelligence tools has made it easy to measure almost anything. The harder question, which often goes unasked, is whether measurement is producing insight or merely producing numbers.

A common failure mode is measurement without consequence. The dashboard displays dozens of metrics, updated in real time, but no one can articulate what action would follow from any of them. The numbers are observed, perhaps discussed, but they do not connect to decisions. Measurement has become ritual.

Another failure mode is measurement of the measurable rather than the important. Revenue and cost are easy to track; the quality of customer relationships or the health of the product pipeline are harder. The result is overemphasis on what can be quantified and underemphasis on what actually matters.

Metrics also tend to calcify once established. A measure that made sense in one context continues to be tracked long after the context has changed. The organization inherits a measurement infrastructure built for yesterday's problems, and no one has the authority or inclination to question whether it still applies.

The most insidious failure is when metrics become targets and thereby cease to be good metrics. This is Goodhart's Law in practice: once people are evaluated on a number, they optimize for the number rather than for the underlying performance it was meant to reflect. The metric improves; actual performance may not.

Effective measurement requires asking uncomfortable questions. What would we do differently if this number moved? Who is accountable for acting on this information? Is this metric still relevant, or are we tracking it out of habit? These questions are rarely popular, because they challenge established practices and expose gaps in organizational clarity.

The purpose of measurement is to improve decisions, not to demonstrate that decisions are being made. A small number of well-chosen metrics that connect directly to actions is more valuable than a comprehensive dashboard that no one uses.

The test is simple: take any metric currently tracked and ask what would change if it moved significantly. If the answer is unclear, the metric is probably not serving its intended purpose. It may be creating the appearance of management without its substance.

Have a question worth discussing?

If something here connects to a situation you are working through, reach out.

Start a conversation
Metrics That Obscure Rather Than Reveal | Notes | JHC Consulting