Field Evidence on Individual Behavior & Performance in Rank-Order Tournaments

23 Aug 2012  Working Papers

by Kevin J. Boudreau, Constance E. Helfat, Karim R. Lakhani, and Michael Menietti

Executive Summary — Contests abound in everything from amateur and professional sports to arts, architecture, manual labor, and engineering. Just as large-scale online contest platforms that provide ongoing tournament-based work and compensation have emerged, large industrial companies increasingly use them as a complement to in-house research and development. What difference does increased competition make to individual participants? This paper analyzes data from algorithmic programming contests to shed light on the mechanisms that underlie changes in performance in reaction to increased competition. Three mechanisms may account for a performance decline: reduction in effort, increased risk taking, and deterioration in cognitive processing. The study also shows how the ability of competitors affects their reactions to increased competition. Overall, results suggest that a better understanding of behavioral responses in contests can aid both public policy and contest designers. Key concepts include:

• The authors analyze contest data on individual effort, risk taking, and cognitive errors.
• On average, competitors react negatively to an increase in the total number of competitors, and react more negatively to an increase in the number of superstars than non-superstars.
• These negative effects are strongest in a particular subgroup of competitors: those who are highly skilled, but whose abilities put them near to the top rather than at the top in terms of ability.
• For competitors who are near-to-the-top in terms of ability, there is no evidence that the decline in performance outcomes stems from reduced effort or increased risk taking. Instead, errors in logic lead to a decline in performance.
• A small group of very high ability competitors (excluding superstars) reacts positively to increased competition from superstars.
• Very high ability competitors show some evidence of increased effort and no increase in errors of logic, consistent with both economic and psychological explanations.

Author Abstract

Economic analysis of rank-order tournaments has shown that intensified competition leads to declining performance. Empirical research demonstrates that individuals in tournament-type contests perform less well on average in the presence of larger number of competitors in total and superstars. Particularly in field settings, studies often lack direct evidence about the underlying mechanisms, such as the amount of effort, that might account for these results. Here we exploit a novel dataset on algorithmic programming contests that contains data on individual effort, risk taking, and cognitive errors that may underlie tournament performance outcomes. We find that competitors on average react negatively to an increase in the total number of competitors, and react more negatively to an increase in the number of superstars than non-superstars. We also find that the most negative reactions come from a particular subgroup of competitors: those that are highly skilled, but whose abilities put them near to the top of the ability distribution. For these competitors, we find no evidence that the decline in performance outcomes stems from reduced effort or increased risk taking. Instead, errors in logic lead to a decline in performance, which suggests a cognitive explanation for the negative response to increased competition. We also find that a small group of competitors, who are at the very top of the ability distribution (non-superstars), react positively to increased competition from superstars. For them, we find some evidence of increased effort and no increase in errors of logic, consistent with both economic and psychological explanations.

Paper Information

• Full Working Paper Text
• Working Paper Publication Date: August 2012
• HBS Working Paper Number: 13-16
• Faculty Unit: Technology and Operations Management