Summary: A central challenge facing firms is how to incentivize employees. While objective, output-based incentives may be theoretically ideal, in practice they may lead employees to reduce effort on non-incentivized outcomes and may fail in settings where effort is weakly tied to output. We study the effect of subjective incentives (manager performance evaluation) and objective incentives (test score-based) relative to no incentives for teachers using an RCT in 230 Pakistani schools. First, we show that subjective and objective incentives both increase test scores and have similar magnitude effects. However, objective incentives decrease non-test score student outcomes relative to subjective incentives. Second, we show that teachers’ effort response is very different under each scheme, with attendance increasing under subjective and teaching quality decreasing under objective. Finally, we rationalize these effects through the lens of a moral hazard model with multi-tasking. We use within-treatment variation to isolate the causal effect of contract noise and distortion and show that these channels explain most of our reduced form effects.
Subjective versus Objective Incentives and Teacher Productivity
Citation: Andrabi, Tahir and Christina Brown. 2022. “Subjective versus Objective Incentives and Teacher Productivity”, RISE Working Paper Series. 22/092
Tahir Andrabi
Christina Brown
How should schools incentivize teachers when effort is non-verifiable or non-contractable? Contract theory provides an answer. The second best is to incentivize on outcomes of the employee’s production function. However, this introduces two new problems – distortion, over-incentivizing measurable outcomes while ignoring others, and noise, outcomes are a noisy function of employee effort. How do most non-schools actually incentivize workers? They use manager-discretionary (subjective) incentives rather than outcome-based (objective) ones. Raises, promotions, and terminations are subject to manager discretion for most employees. Despite the prevalence of subjective incentives, there is limited causal evidence on the effect of these incentives and whether they could work in the teaching setting.
In this paper, we ask two questions: What is the effect of subjective versus objective incentives on teacher productivity? Are subjective incentives able to help alleviate problems of noise and distortion, which often plague objective incentives? We answer these questions by conducting an 18-month randomized controlled trial with 234 private schools in Pakistan. We randomize schools to provide core teachers with one of three contracts: (i). control: flat raise – all teachers receive a raise of 5% irrespective of performance, (ii). treatment 1: subjective performance raise – teachers receive a raise from 0-10% based on their manager’s rating of their performance, or (iii). treatment 2: objective performance raise – teachers receive a raise from 0-10% based on their students’ midyear and end of year test performance.
Study Design and Findings
Subjective and objective contracts are equally effective at increasing test scores
Both contracts increase test scores by 0.09 sd, which is very similar to average effects from meta-analyzes of performance pay for teachers. These results are consistent across subject and grade and are not driven by rote-memorization type questions. However, we find, in contrast to the test score results, objective and subjective incentives have different effects. Objective incentives negatively affect student socio-emotional development, including a significant decrease in love of learning and an increased likelihood students say they want to change schools. Subjective incentives result in a small positive effect on overall socio-emotional skills. These combined effects suggest that teachers under objective contracts focused exclusively on improving student academic improvement, at the cost of more well-rounded development for students. Whereas, teachers under the subjective contract were able to prioritize both areas.
Subjective and objective incentives lead to changes in classroom practices
As one might expect, subjective incentives spur actions that managers value, and objective incentives spur actions that most quickly and easily translate into test score gains. Subjective incentives lead to increased targeting of individual student needs within the classroom and the use of technology in the classroom. Both teaching practices are one’s principals identified as markers of high-quality teaching. Objective incentive schools see a five-fold increase in class time on test preparation activities. These teachers also exhibit more negative discipline techniques, such as yelling at students.
Teachers believe that subjective incentives are less noisy
We find that teachers believe subjective performance incentives are less noisy than objective incentives, and, therefore, view subjective incentives as more effective at motivating behavior. They view test-score based incentives as much less within their control because so many other factors beyond their effort affect student scores. We also find that teachers in the objective treatment are more likely to prioritize the type of actions which lead to test score gains, at the cost of other areas of student development. Teachers under subjective contract prioritize actions that lead to academic gains and also prioritize administrative tasks, which are likely to be preferred by their manager.
We find that both subjective and objective incentives increase test scores, but objective incentives result in negative effects on socio-emotional development. These student outcomes make sense given the teacher behaviors we see under each incentive. In subjective treatment schools, teachers make small improvements in pedagogy and are involved in more professional development. In objective treatment schools, teachers distort effort toward test preparation. They spend much more time on practice tests and test strategies and use more punitive discipline. While there is heterogeneity in manager application of the subjective treatment arm, we do not find evidence of widespread favoritism or bias.
Study Resources
The following resources are for public use in presentations, papers, lectures, and more under the Creative Commons license BY-ND. Click the images below to view or download individual images, or use the button to download all.
As a condition of use, please cite as: Andrabi, Tahir and Christina Brown. 2022. “Subjective versus Objective Incentives and Teacher Productivity”, RISE Working Paper Series. 22/092
Effects of incentives on student test scores
Effects of incentives on teacher pedagogy
Difference in perceived noise by treatment
Subjective vs. Objective Incentives Presentation