Do teacher pay incentives improve pupil test scores?

A meta-analysis published in the American Educational Research Journal looks at the association between teacher pay incentives and pupils’ test scores, and suggests that teacher pay incentives have the potential to improve pupil test scores in some contexts.

Lam D Pham and colleagues analysed effect sizes across 37 studies, 26 of which were conducted in the US. To be included in the meta-analysis, studies had to include a sample comprising teachers and pupils in K-12 education (Year 1 to Year 13) located in a school district or area that had a teacher pay incentive programme. Studies also had to use a randomised controlled trial with a business-as-usual comparison group, and report on pupil outcomes on standardised tests.

Overall, among the US-based studies, the effect of teacher pay incentives on pupil test scores was positive (effect size +0.04), however, this varied across subjects and settings. The average effect size of pay incentives on pupils’ maths test scores (+0.05) was larger than the effect on English test scores (+0.03). Pay incentives for elementary (primary) school teachers were associated with larger effects (+0.10) than middle school teachers (+0.01). In addition, larger pay incentives, and pay incentives that are based on multiple measures of teacher effectiveness, were associated with larger effect sizes.

Source: Teacher merit pay: A meta-analysis (February 2020), American Educational Research Journal

$575 million programme has no impact on pupil outcomes

Findings from an evaluation of a $575 million programme to improve teacher performance found that, while sites implemented new measures of teaching effectiveness and modified personnel policies accordingly, the programme had no impact on pupil outcomes.

The Intensive Partnerships for Effective Teaching initiative, designed and funded by the Bill and Melinda Gates Foundation, aimed to dramatically improve pupil outcomes by improving pupils’ access to effective teaching. Three US school districts and four charter management organisations participated in the programme, which ran between 2009 and 2016.

The final evaluation report, published by the RAND Corporation, found that by the end of 2014-15, outcomes for pupils in the settings that took part in the initiative were not better than outcomes for pupils in similar settings that did not take part. There was no evidence that low-income minority (LIM) pupils had greater access than non-LIM pupils to effective teaching. In addition, it found very few instances of improvement in the effectiveness of teaching overall, and no improvement in the effectiveness of newly hired teachers compared to experienced teachers. The evaluation also found no increase in the retention of effective teachers, although there was some decline in the retention of ineffective teachers in most settings that took part in the initiative.

The report states several possible reasons that the initiative failed to achieve its goals for improving pupil outcome:

  • incomplete implementation of the key policies and practices
  • the influence of external factors, such as state-level policy changes during the initiative
  • insufficient time for effects to appear
  • a flawed theory of action
  • a combination of all these factors.

 

Source:  Improving teaching effectiveness: Final report: The Intensive Partnerships for Effective Teaching through 2015–2016 (2018), RAND Corporation.

Strategies to promote teacher effectiveness

The Institute of Education Sciences has released a new evaluation brief that synthesises findings from two impact studies conducted by the National Center for Education Evaluation (NCEE). One study focused on a strategy of providing teachers  with feedback on their performance for two years (performance feedback), and the other study focused on a strategy of providing teachers with bonuses for four years based on their performance (pay-for-performance). Both strategies were supported by the Teacher Incentive Fund, which provided competitive grants to help US states and districts implement a multi-strategy approach to enhancing teacher effectiveness.

In each study, elementary and middle schools (primary schools) were randomly assigned to implement the strategy (the treatment group) or not (the control group). The performance feedback study included approximately 29,000 pupils and 1,000 teachers in grades 4–8, while the pay-for-performance study included approximately 38,000 pupils and 3,500 teachers in grades 3–8. Pupil outcomes were measured using end-of-year reading and maths scores.

Key findings were as follows:

  • Providing teachers with feedback on their performance for two years improved pupils’ maths achievement after the first year with a difference in scores that corresponds to an effect size of +0.05. The cumulative effect after two years of implementation was similar in magnitude but not statistically significant. The effect on reading in both years was positive but not statistically significant.
  • Providing teachers with bonuses based on their performance for four years improved pupils’ reading achievement after one, two and three years of implementation and pupils’ maths achievement after three years. After each of those periods of implementation, the effect size was +0.04 for reading and +0.06 for maths. However, as noted in the evaluation report, the impacts of pay-for-performance on classroom observation ratings did not appear to explain the impacts on pupil achievement, and in treatment schools, as many as 40% of teachers were unaware that they could earn a performance bonus.

The brief was prepared for NCEE by Andrew Wayne and Michael Garet of American Institutes for Research and Alison Wellington and Hanley Chiang of Mathematica Policy Research.

Source: Promoting educator effectiveness: the effects of two key strategies (March 2018), National Center for Education Evaluation, The Institute of Education Sciences

What is the impact of pay-for-performance?

Teacher Incentive Fund (TIF) grants were awarded in 2010 by the US Department of Education to support performance-based compensation systems for teachers and principals in high-need schools.

In order to assess the impacts of pay-for-performance on educator (teachers and principals) and pupil outcomes, an experimental study design was used in ten US school districts to randomly assign elementary and middle schools to treatment and control groups. Both groups implemented the same performance-based compensation system, but in the control schools, the pay-for-performance element was replaced by a one percent bonus paid to all teachers and principals regardless of performance. A fourth and final report from this evaluation has now been published, covering all four years of the programme (between 2011 and 2015).

Among the key findings are that pay-for-performance had small, positive impacts on pupil achievement by the second year of implementation. From that year onward, reading and maths achievement was higher by 1 to 2 percentile points in schools that offered performance bonuses than in schools that did not. However, it was not entirely clear how this improvement was achieved. The impacts of pay-for-performance on classroom observation ratings did not appear to explain the impacts on pupil achievement, and in treatment schools as many as 40% of teachers were unaware that they could earn a performance bonus.

Source: Evaluation of the Teacher Incentive Fund: Final report on implementation and impacts of pay-for-performance across four years (NCEE 2017-4004),(December 2017), National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, US Department of Education.

The impact of pay-for-performance

A new report from Alison Wellington and colleagues, published by the Institute of Education Sciences, looks at the implementation and impacts in US schools that offered pay-for-performance as part of their 2010 Teacher Incentive Fund (TIF) grants. These grants, now named the Teacher and School Leader Incentive Program, support performance-based compensation systems for teachers and principals in high-need schools.

An experimental study design was used to assess the impacts of pay-for-performance on educator and student outcomes. Elementary and middle schools within the evaluation districts were randomly assigned to treatment and control groups. The treatment schools were to fully implement their performance-based compensation system. The control schools were to implement the same performance-based compensation system with one exception—the pay-for-performance bonus component was replaced with a one percent bonus paid to all educators regardless of performance.

For the 10 evaluation districts that completed three years of TIF implementation (between 2011 and 2014), key findings showed that pay-for-performance had small, significant positive impacts on students’ math and reading achievement. The report notes that after three years of TIF implementation, the average math score was 2 percentile points higher in schools that offered pay-for-performance bonuses than in schools that did not. The average reading score was 1 percentile point higher in schools that offered pay-for-performance bonuses than in schools that did not. This difference was equivalent to a gain of about four additional weeks of learning.

Source: Evaluation of the Teacher Incentive Fund: Implementation and Impacts of Pay-for-Performance After Three Years (2016), Institute of Education Sciences

Teaching in Europe

Eurydice has published a new report about the teaching profession in lower secondary schools (approximately ages 12-15) in Europe. It uses the findings of the the 2013 OECD Teaching and Learning International Survey (TALIS) and Eurydice and Eurostat data on the 28 EU Member States, and seven other European countries. In all, 40 different education systems (including England, Wales, Scotland, and Northern Ireland), and around two million teachers were included.
 
The extensive report examines five areas considered important for policy: (1) demographics and working conditions; (2) initial teacher education and the transition to the teaching profession; (3) continuing professional development; (4) transnational mobility; and (5) attractiveness of the profession.
 
Findings include:
  • A degree is the minimum entry level for teacher training programmes in 15 countries, and 17 countries require a Master’s;
  • The minimum length of initial teacher training is usually between four and six years;
  • Within the EU, 91.2% of teachers have completed an initial teacher training programme;
  • Teaching time is contractually specified in 35 education systems. The majority of countries also centrally regulate the total working time of teachers, which averages 39 hours a week;
  • On average, teaching time constitutes 44% of a teacher’s total working time. England, Estonia, Sweden, Wales, and Northern Ireland are the only education systems that do not contractually specify the number of teaching hours;
  • In 29 education systems CPD is a “professional duty”, although around a third of these (including England, Wales, and Northern Ireland) do not specify how much time should be spent);
  • Incentives to participate in CPD exist in almost two-thirds of the education systems surveyed; and
  • In general, teachers are satisfied or very satisfied with their profession but consider that society does not value it.
 
Source: The Teaching Profession in Europe: Practices, Perceptions, and Policies (2015), Eurydice.