Performance Management and Measurement

I've wanted to write this post for a long time. It may never have seen the day if it wasn't for the fact I've been asked to make a presentation on this exact topic in my (formerly undisclosed) Department.

Whenever I need to discuss a subject I know will be controversial (especially when I know my points of views will be "challenging" - and challenged! - by the people in the room), I like to draw extensively on the work of other authors. This way I can say precisely what I want to say, without sounding like I'm the first person to ever make such a statement. True to my habit, this is just what I will do for this post.

First, if performance management is new to you, you must read the chapter entitled "Performance Management : Panacea or Fools’s Gold?" written by Barbara Wake Carroll and David I. Dewar in "The Handbook of Canadian Public Administration" (2002, pp. 413-429). The authors lay the foundations for what performance management is:

“Performance measurement is the collection of information about the performance of programs using some indicator or standard of measurement. Performance monitoring is the review, or tracking, of the measurements. When they are used by a manager to improve programs they form part of a performance management system. [...] Performance management consists of four elements:

• Deciding what is the desired level of performance
• Measuring performance
• Reporting or communicating performance information
• Using performance information to compare actual performance to the agreed performance level”

The authors further explain that there are three levels of focus to performance management in the public sector:

1. “The evaluation of programs or policy at the broad governmental or political level and includes a political consideration of basic objectives
2. The implementation and management of a policy or program
3. The assessment of the performance of individual employees”

(And I would add a fourth level: the management of an organization or business unit.)

Carroll and Dewar then make some distinctions with regards to what can be measured:

• "Process: how something was done
• Outputs: what was done or produced
• Outcomes: what happened in society
• Impact or results: what are the effect of what was done (what has changed as a result of the action) "

(They don't mention "inputs", but I think this one goes without saying as it is typically the easiest to measure and the one that usually gets all the attention.)

Based on my experience in project management, I see four challenges to performance measurement:

1. Availability of data in a timely fashion (i.e. outcomes and outputs sometimes take years to materialize and data is therefore rarely - if ever - available upon completion of a project or implementation of a program);
   
2. Priorities change over time (e.g. ever noticed how budgets become such a hot issue towards the end of fiscal when it's never nearly as big of a deal the rest of the year?);
   
3. Ease of measuring (usually, information relevant to the "management" side of things are easier to measure: inputs, resources, hours of work, expenditures, etc.);
   
4. Linking inputs, processes, outputs, outcomes and impacts (who can really prove that the inputs are directly tied to the outcomes?)
   

For these reasons, critics of performance measurement have voiced similar concerns in different terms (now is probably a good time to resort to quotes!):

• "People revert to metrics out of fear, not out of vision." (Patrick Lencioni)
• "Managers who don't know how to measure what they want settle for wanting what they can measure."(Russell L. Ackoff & Herbert J. Addison)
• "Measurement has its pitfalls and dangers, however, as well as its benefits. The first, well-known danger is that you get what you measure. If measurements are attached to some facets of public service work then those are the things that will get attention, often at the expense of other, equally or more important dimensions of work. […] The second danger, which is a corollary of the first, and the one that concerns us most here, is that things that are difficult to measure may be neglected, while things that are easier to measure will enjoy exaggerated importance. Values, leadership and good people management are among those things that are not yet sufficiently valued, on the excuse that they are difficult to measure." (The Task Force on Values and Ethics)
  

As a former manager, I have always found resorting to metrics quite useful for:

• Improving estimates (learning from experience);
• Measuring efficiencies (do more with less);
• Improving visibility of critical activities (what gets measured gets done);
• Identifying problems in real time (and correct them);
• Spot trends (across projects or units, recurring problems).

But recently I came across an amazing article written by Nicolas Berland, Catherine Chevalier-Kuzla and Samuel Sponem, entitled “On ne gère bien que ce que l’on mesure” published in the "Petit bréviaire des idées reçues en management" (2008). The authors offer what is probably the best dissection of the benefits and pitfalls of measurement I have read. Here are some of the highlights:

Benefits of Measurement:

• Measurement allows synthesizing large quantities of information and clarifying the complex.
• Measurement can make explicit relations between phenomena, and foster learning.
• Measurement makes visible many aspects of the organization.
• Measurement provides a common language and allows shared representation, because measurement is considered objective, i.e. detached from the individual.
• Measurement allows the convergence of goals and allows changing organizations. Indeed, measurement is a support to evaluation systems and is a powerful means of incentive.
• Measurement allows to classify actions, objects, and people compared to each other's, and thus determine the respective contributions and their scale, which in turn allows to decide and to act.
• Measurable objectives allow to control from a distance other people's behaviours, to responsibilize them, and to motivate them.
• Measurement is the antidote to ambiguity; it forces to be clear on vague concepts and forces to act.

Pitfalls of Measurement:

• Measurement has a number of risks: simplification of reality, disconnect with the field, short term focus or short-sightedness, orient behaviours in the wrong direction because of a bad choice of performance indicator.
• Measurement is interested first and foremost in the past.
• Measurement omits important elements of long-term performance of an organization: quality, competence, trust, innovation, etc.
• Furthermore, are easily measurable only variables relative to unambiguous objects. But management is often in the domain of ambiguity.
• Measurement is only as good as the systems and people that captured (garbage in garbage out)
• The danger of motivating people through numbers is that they will disregard what is not measured.
• Sometimes, performance indicators take a life of their own. Measurement is no longer synonym with good management but rather out of hand bureaucracy. What was initially a means to improve the functioning of the organization becomes an end in itself: you must increase (or decrease) the number, no matter the underlying reality. Measurement takes over management.

The key point made by the authors however is when they say that the true value of performance measurement is not in providing answers as much as raising questions. They go on to offer the most honest look at what performance management really accomplishes:

• Measurement can be used to move your agenda forward.
• Measurement is a symbol. It doesn't have to be exact, only to influence people.
• When measurement doesn't produce the expected result, the easiest thing may be to change the measurement.
• In the end what counts is less measurement than the result of the action created by measurement.

The authors close on some philosophical questions about performance management and measurement:

• Where does the obsession for measurement come from?
• What are the real ends of measurement for a manager?
• Do we manage better because we measure more?
• Do we manage poorly when we don't measure?
• Is measurement sufficient for good management?

While you ponder these questions, let me recommend you some more reading:

• Performance Measurement
• Implementing the Vision: Addressing Challenges to Results-Focused Management and Budgeting (Section 4 is particularly worthy of attention)
• Making Performance Management Work ... Better
• What Gets Measured Gets Done
• Why Quantitative Measures Often Make Performance Worse, not Better
• Just because you can measure something with numbers, doesn’t mean that you should
• You Can't Manage What You Don't Measure