KPIs And The Hawthorne Effect – Part 1

By Rick Grantham | Uncategorized

Nov 10

You’ve done it !! kpis and the hawthorne effect


Your Business Intelligence (BI) system is humming along nicely. You have all the necessary gadgets in place – bells and whistles included. Ad hoc reporting keeps your power users happy. Your managers can dive into any metric and explore the data. And dashboards let your executives track their Key Performance Indicators (KPI’s). All is well.


Actually, all is well only until a KPI misses the target (turning red). Now its time to get past the technology and actually manage your metrics. When a KPI misses the target, here is what usually happens.


  • Get on the phone
  • Have more reports run
  • Power-Users run multiple scenarios
  • Call meetings
  • Get the metric on everyone’s radar
  • Make it a standing item in weekly meetings


Basically — you give it a lot of attention – and rightly so. You put a Business Intelligence environment in place so that these items could come to the surface so you can manage them. As a result, the metric improves. You feel good about yourself. And why not? Your attention to the metric helped to improve it. You have the Midas Touch. This is why you get paid the big bucks. There is just one problem – The Hawthorne Effect.


Hawthorne Effect

Hawthorne Effect Definition

The Hawthorne Effect (link to more detail) is a term to describe the Hawthorne Works experiments in the 1920’s and 1930’s. In the experiments, the lighting was increased in a manufacturing plant to gauge the impact on worker productivity and as expected, productivity improved. The Midas Touch. After a period of time productivity returned to normal. When the researchers moved the lights back to normal wattage, an interesting thing happened. Productivity increased again. Basically, productivity increased when any number of things were changed, from variations in lighting to cleaning up the workspace, etc.. But the results were only temporary. Many follow-up studies have verified this effect. People just like it when you pay attention to them, but the improvement is only temporary.


Contrarian View of the Hawthorne Effect


Several years ago, Steven Levitt (of Freakonomics Fame) re-analyzed the data and discovered that the original experiments were not nearly as cut and dry as originally thought.  He suggests that the impact Hawthorne Effect from the original studies is over-amplified and not necessarily backed up by the data.  Click to view the study.


Cut To The Chase


If improvements are seen in your KPI’s only because you are giving them attention, then you should assume that those improvements are temporary. The improvements will be short lived unless you change the underlying process that is creating the results. Said differently, nothing changes for the long-term until you change the process. Most people look at Measurement Systems or KPI’s as an end-product, an alert that allows you to use your management skill to change the metric. However, the wise manager doesn’t see KPI’s as an end product of Business Intelligence, but rather as the first of three cogs that leads to long term improvement.

hawthorne effect


In the graphic above, KPI’s (Measurement/BI System) leads to Analysis (Root-Cause) which leads to to Process Improvement. Once your Process Improvement initiative is complete, a new set of measurement tools should be put in place to ensure that the results are consistent over the long term. Many organizations use Control Charts to track these process improvements.


So What Do You Do From Here?


In Part 2 of this series, we will explore your options.  More to come.

About the Author