In a recent trip to Pakistan, I visited several types of schools. As in other parts of the word, there were striking differences in quality between and within public and private schools. Some public schools prepare the most skilled individuals in the country, others provide low quality education, and the same goes for private schools. The source of these differences are, on one hand, diversity in family background among the populations that attend each type of schools and, on the other, differences in the characteristics of the schools. Regarding the characteristics of the schools, both educators and economist agree that, at the heart of producing high quality of education, teachers are a fundamental part of the equation. However, the research towards finding what makes an effective teacher has remained elusive.

The fundamental problem in estimating the impact of teachers’ policies is that teachers self-select into the profession and into programs and also students self-select into schools and into the classrooms of specific teachers. Under these conditions, performing a comparison between teachers that participate and those that do not participate in a specific program—such as teacher training—confounds the effects of the program with the initial differences in characteristics between participants and non-participants. Recent developments in impact evaluation, as a technique to find causal effects from policies to outcomes, have been contributing to elucidate the effects of teachers’ policies on learning. 

Five programs around the word

Economists believe that changing incentives will change the action of teaching and therefore, will have impact on learning outcomes; educators believe that training can change pedagogic methods, and therefore, impact learning outcomes. Teachers’ incentive programs are designed so that teachers exercise the optimal effort that will induce the best outcome in students’ learning. Presumably the most important assumption of these programs is that teachers know how to modify their behavior.

The typical incentive program has the following structure. Teachers are promise a bonus it the school perform above certain threshold. Usually the bonus is given to all teachers in the school. The “target” variable is the schools average result in standardized tests.

Several countries have tried some variation of teacher incentive programs. Five recent evaluations¹  produced mixed results. In Kenya, within a context of a randomized control trial, Glewwer et al (2008) do not find any changes in homework or pedagogy from part of teachers. The study shows short term impact in the test linked to the program (the official test), but no impact on an independent test administrated by the evaluation team. Furthermore, the study does not find any impact on drop-out rates.  The authors show some evidence of teaching to the test, which can explain the positive short term results. It is worthwhile to point out that they tested an absolute measure program versus a value added incentive, without finding any affects from either one.

Duflo et al (2007), with a randomized evaluation of a program that incentives presence of the teacher at the schools, find positive impacts on test scores of students (around 0.17 SD); a reduction of teacher absence in 21 perceptual points; and an increase in instructional time in 30%. Muralidharan and Sundararaman (2008) also find positive impact in a randomized program in India in test scores (0.15 SD in first year; 0.22 SD in second year).  The authors show evidence of heterogeneous impacts across household wealth levels. After two years, the authors find that the impact of individual bonuses is larger than group bonuses. In the case of Israel, Lavy (2002) finds positive impact in average test score of the treated schools; increments in the number of credits taken by students, and higher proportion of students for matriculation certificate (specially from low-income). The study also reports reduction in drop-out rates in treated schools. Finally, the study of Mexico does not find any impact on test scores. The authors present convincing evidence that, in this case, the program was not able to incentivize a large proportion of teachers. The reason behind this result is that the score of teacher is a compose index of different attributes—such as experience and academic titles—and a large proportion of teachers reach the necessary points to jump to the next wage level, even if their students do poorly in the tests.

Key elements of programs

It is unclear the nature of the mixed results from the five evaluations. However, one (almost obvious) lesson emerges from these studies: the design of the program is critical for the ultimate result. Three elements seem to play a significant role:

• The outcome variable and attribution. The problem of attribution is to find the relevant variable to evaluate the effort of teachers. First, observability of teachers’ effort is very difficult. Second, usually test scores depend on the effort—and skills—of both teachers and students. A program that targets a variable under direct control of the teacher—such as attendance of the teacher to the school every day—can assess better the attribution of effort.
• The measurement of the outcome variable—either in absolute value or in change value—has several implications in equity and in the structure of incentives. Programs that offer bonus to teacher in schools that pass certain grade (e.g., schools that have 80% or more in a test), do not incentive schools that are far away to the cut-off. In contrast, programs that award changes in the outcome variable—improvements—tend to incentivize the whole spectrum of schools.
• Group or individual incentives: the bonus can go to all teachers in the school or to individual teachers in individual grades.  There are two main challenges of collective rewards. In the first place, the collective reward can lead to free riding problems among teachers. Second, collective rewards can lead to strategic allocation—form part of the school— of the best teachers to key grades. The challenge with individual incentive can induce distrust among teachers and unfair competition.

A final lesson from this emerging research agenda is that clearly we need to understand better the system of incentives to teachers, and we need to increase our knowledge about effective pedagogies. Both perspectives, economist and educators, has to work together in order to understand what makes a good teacher.