Wednesday, December 18, 2013

Why KPIs are a dangerous fad

You have heard of painting by numbers, but these days the great fad is management by numbers. I call it the metrification of business - although it's just as prevalent in the public sector. If you know what the initials KPI stand for, you'll know what I'm talking about.

I've been a bean-counter all my working life, first as an accountant, then as an economic journalist. So I've long believed in the importance of measurement, of getting the measurement as accurate as possible and of understanding the strengths and weaknesses of particular measures.

Much of my apprenticeship as an economic journalist was spent making sure I understood how all the economic statistics were put together. But as I've watched the enthusiasm for "key performance indicators" and other "metrics" grow, I've become increasingly sceptical about their usefulness.

The rise of metrification has been built on repetition of the seeming truism that what gets measured gets managed. It follows that what isn't measured isn't managed.

It's certainly true that what we measure affects what we do and the way we think about the phenomenon being measured. So in the drive to hold workers accountable - that is, to increase control over them - and improve their performance, it's become fashionable among managers - private and public sector - to measure key aspects of an organisation's performance.

It's only a small step to setting targets for the performance measures and, often, raising those targets from year to year. You might have targets for the performance of organisations, but also for the performance of individuals. And it's only another small step to link performance to remuneration. We pay for results - what could be fairer?

The rapid escalation of executive remuneration over the past 20 years has occurred not so much because of the rise in basic salaries as the proliferation of "performance pay". A study by Mihir Desai, of the Harvard Business School, found that, in America, the proportion of the total pay of senior managers that is based on share prices rose from 20 per cent in 1990 to 70 per cent in 2007.

But, as The Economist reported last year, Desai found this had warped incentives and fostered malfeasance. Managers had won huge payouts simply because the share market had gone up, regardless of whether they had personally added value.

"They have also gamed the system, sometimes illegally, to hit targets that put fat sums in their pockets," it said.

The trouble with this measurement approach to accountability and reward is that, as the American psychologist Martin Seligman has said, "If you don't measure the right thing, you don't get the right thing."

When the push for micro-economic reform was at its height, someone got the bright idea that if you calculated and made public the equivalent of key performance indicators for all the many responsibilities of the state governments, you'd encourage them to compete among themselves to improve their standing in the league tables.

The Productivity Commission has now compiled and published these indicators for many years. But someone who should know warned me they'd become completely unreliable. Why? Because managers in each state had manipulated their results to make them look good against the competition.

In 2005 the website ran a letter from an anonymous public servant reporting their experience with management by KPI.

"Early in June our manager discovered we were a few percentage points away from meeting operational requirements for the financial year. Rather than explain to his boss that staff cannot perform well when there are continual computer problems and weekly changes in procedures and priorities, he instituted a series of ludicrous schemes to improve the statistics," the person wrote.

"Any work that was already out of time was placed on the backburner, not to be touched until after July 1, when it would be counted in the next year's statistics. In other words, work that was overdue would not even be looked at for another fortnight.

"For two days staff did nothing but go through their files searching for cases that could be closed without further action or referred to another area. We achieved absolutely nothing in terms of genuine output for those two days, but our percentage of resolved cases sky-rocketed. We then started on the new work, but only worked on simple cases that could be closed well within the acceptable operational time frame...

"On June 30 our manager proudly announced that we had achieved operational requirements."

I've been around long enough to know that measurement can be a trap. People can be mesmerised by numbers. Because they're objective, people take them to be infallible. They forget (if they ever knew) the assumptions and other limitations on which they're based, they take them to be measuring something they're not and they forget how easily others can manipulate them for their own purposes.

It's easy to measure quantity, but much harder to measure quality. Most jobs are multi-dimensional, but you can't have a KPI covering every dimension. In which case, I can always meet my KPIs by cannibalising some dimension - some aspect of quality - not covered by a KPI.

Einstein said "not everything that counts can be counted". The modern preoccupation with metrics is an attempt to over-simplify the managerial task by confusing quantity with quality.

Sorry, life wasn't meant to be that easy.