Measuring the Efficacy of the World’s Managers

30 Jan 2012  Research & Ideas

Over the past seven years, Harvard Business School's Raffaella Sadun and a team of researchers have interviewed managers at some 10,000 organizations in 20 countries. The goal: to determine how and why management practices differ vastly in style and quality not only across nations, but also across various organizations and industries.

by Carmen Nobel

Firms in the United States, Japan, and Germany tend to be managed especially well, while firms in Brazil, China, and India tend to be managed poorly.

Those are among the initial findings of the World Management Survey (WMS), a huge international research initiative to measure differences in organizational management practices all over the world.

The project was borne of a widely perceived gap in economic research. In business academia, there is an optimistic tendency to assume that managers generally make decisions in the best interest of their firms' performance. Reality is more frustrating. The fact is, some companies are managed beautifully, others dismally.

“During the interview she got a marriage proposal. The manager wanted her to marry his son.”

"It's very tough to believe that there are such wide differences in management out there," says Raffaella Sadun, an assistant professor at Harvard Business School who coleads the WMS with Nicholas Bloom of Stanford University and John Van Reenen of the London School of Economics. "If you really want to be convincing, it's important to have large-scale statistical evidence," she adds.

To that end, over the past seven years, large teams of affiliated WMS analysts have interviewed managers at some 10,000 organizations in 20 countries, setting out to determine how and why management practices differ vastly in style and quality.

Best practices

The researchers used an evaluation tool designed by a leading consultancy firm that broadly rated management practices in three areas: monitoring—how well managers keep track of what's happening in a firm and make good use of that information; targets—how well organizations set appropriate goals and outcomes, and whether they take action if the two are inconsistent; and incentives—whether organizations promoted and rewarded employees based on performance and tried to keep the best performers from quitting.

To collect the data, the researchers hired teams of MBA students who could interview managers in their respective native languages. The respondents chosen for the survey included primarily middle managers in manufacturing plants, although over time the data collection was extended to other industries, such as retail, schools, and hospitals. The approach was to interview those who were high enough on the corporate totem pole to have a good overview of management practices, but low enough that they were familiar with day-to-day operations. The researchers also chose to target primarily small and medium-sized firms, employing between 100 and 5,000 workers, to maximize the chances that the interview would capture salient features of the whole organization, as opposed to the characteristics of single plants.

Rather than limiting the responses with multiple choice or yes/no questions, the interviewers kept the questions open-ended to get a complete picture of the actual practices adopted in the organization. This method was also useful to build an effective bond between the interviewer and the manager, which sometimes led to answers that were both thoughtful and entertaining. (Responding to a question about staff retention, for instance, one manager said, "I spend most of my time walking around cuddling and encouraging people. My staff tells me that I give great hugs.")

"We had very interesting experiences," Sadun recalls. "Once, a student was interviewing a manager, and during the interview she got a marriage proposal. The manager wanted her to marry his son."

Key findings

The researchers have summarized the initial results of the study in a new working paper, Management Practices across Firms and Countries,—also coauthored by Christos Genakos of the Athens University of Economics and Business—and have posted a comprehensive collection of the survey data on the World Management Survey website. The WMS site also features individual policy reports for manufacturing, education, health-care, and retail organizations, providing a management benchmark for executives in any of those fields.

On average, based on the evaluation tool, the research showed that firms in the United States, Japan, and Germany tend to be the best-managed in the world, while firms in Brazil, China, and India scored poorly.

“You have a lot of companies where the transmission of leadership is based on criteria completely unrelated to your ability to lead.”

Of the 20 countries in the survey, the United States received the highest overall management scores in retail, health care, and manufacturing. But US schools scored comparatively lower, coming in fourth behind the United Kingdom, Sweden, and Canada.

"US schools in particular tend to be particularly poor at incentives management—that is, promoting and rewarding high-performing teachers, and retraining and/or firing badly performing teachers," the paper states.

In the United States, India, and China, managerial use of incentives are much more common than the use of monitoring and target-setting, especially in the manufacturing field. In Japan, Sweden, and Germany, monitoring and target-setting far exceed the use of incentives.

But the differences surpassed international boundaries. "There is so much dispersion, even within the same industries and same countries," Sadun says, explaining that the researchers managed to discover certain ownership patterns that help to explain the dispersion.

For instance, government-owned organizations tend to receive low management scores across all the sectors and countries in the study. "They are particularly weak at incentives," the paper explains. "Promotion is more likely to be based on tenure (rather than performance), and persistent low-performers are much less likely to be retrained or moved."

Family-owned businesses scored even lower—particularly those run by a firstborn son who inherited the role of CEO. Family-owned businesses that employed a non-related CEO scored much higher. "The finding there is not so much that family ownership per se is associated with lower scores, but rather family ownership when the selection of the CEO is not meritocratic," Sadun says. "Especially in Europe, you have a lot of companies where the transmission of leadership is based on criteria completely unrelated to your ability to lead."

Also receiving low management scores across the board: organizations at which the company founder was also the CEO. "We systematically find that founders have lower levels of scores," Sadun says. "One possible explanation is that founders are great at the start-up phase because they have the vision and the motivation. But as the firm grows, to do the ongoing management on a daily basis requires a different set of skills."

Market competition appears to be good for management. The research showed that a company's overall management score was directly related to how many competitors a firm faces—the higher number of reported competitors, the higher the management score. And multinational firms scored higher than their domestic counterparts across all the industries and countries in the survey.

Education makes a difference, too. The data showed that the greater the percentage of employees with a college degree in manufacturing and retail firms, the greater the management scores. In hospitals, a greater management score correlated with the percentage of managers who also had clinical experience.

Next steps

With the initial data tracked and scored, the researchers are now working with the US Census Bureau to build an even larger set of management data. Last spring they conducted a survey of more than 48,000 US companies, using the questions and the evaluation tool from the initial WMS survey. The Census Bureau already keeps general track of firms' financial performance in the United States. The WMS team plans to match the management survey data against the financial performance data in order to investigate the link between management quality and the bottom line across an even larger number of organizations.

The researchers have also started to work on management experiments, modeled on the randomized control trials adopted in the medical field. This experimental approach is much more costly and labor- intensive than the survey methodology, Sadun says, but it allows them to identify with much greater precision the causal link between management and performance.

"We want to get at the causality between management and performance," she says. "And to do that, you have to go beyond surveys and start running experiments with firms. That's the most challenging but also the most interesting part of where this research is going."

About the author

Carmen Nobel is senior editor of Harvard Business School Working Knowledge.