Findings from an evaluation of a $575 million programme to improve teacher performance found that, while sites implemented new measures of teaching effectiveness and modified personnel policies accordingly, the programme had no impact on pupil outcomes.
The Intensive Partnerships for Effective Teaching initiative, designed and funded by the Bill and Melinda Gates Foundation, aimed to dramatically improve pupil outcomes by improving pupils’ access to effective teaching. Three US school districts and four charter management organisations participated in the programme, which ran between 2009 and 2016.
The final evaluation report, published by the RAND Corporation, found that by the end of 2014-15, outcomes for pupils in the settings that took part in the initiative were not better than outcomes for pupils in similar settings that did not take part. There was no evidence that low-income minority (LIM) pupils had greater access than non-LIM pupils to effective teaching. In addition, it found very few instances of improvement in the effectiveness of teaching overall, and no improvement in the effectiveness of newly hired teachers compared to experienced teachers. The evaluation also found no increase in the retention of effective teachers, although there was some decline in the retention of ineffective teachers in most settings that took part in the initiative.
The report states several possible reasons that the initiative failed to achieve its goals for improving pupil outcome:
- incomplete implementation of the key policies and practices
- the influence of external factors, such as state-level policy changes during the initiative
- insufficient time for effects to appear
- a flawed theory of action
- a combination of all these factors.
Source: Improving teaching effectiveness: Final report: The Intensive Partnerships for Effective Teaching through 2015–2016 (2018), RAND Corporation.
In England there is currently a shortage of maths teachers; among the factors that might be influencing this shortage are that departments lose 40% of teachers during their first six years in the profession, and there are higher private sector wages for maths graduates. At the same time, demand for maths teachers has increased due to policy measures to increase participation in maths for 16 to 18 year olds. To examine what impact this has had, the Nuffield Foundation commissioned researchers from FFT Education Datalab to look at how secondary schools have responded to the shortage.
Rebecca Allen and Sam Sims used data from England’s School Workforce Census and found that schools are using their most experienced and well-qualified maths teachers for year groups taking high-stakes exams (GCSEs, A-levels, and GCSE retakes), and using inexperienced maths teachers and teachers who trained in other subjects to fill staffing gaps elsewhere.
In the most disadvantaged schools (those with more pupils eligible for free school meals), pupils across all year groups are more likely to be taught by an inexperienced teacher. At Key Stage 5 (age 16-18) pupils in the most disadvantaged schools are almost twice as likely to have an inexperienced teacher as in the least disadvantaged schools (9.5% versus 5.3%).
Source: How do shortages of maths teachers affect the within-school allocation of maths teachers to pupils? (June 2018), Nuffield Foundation
A study by John Papay and Matthew Kraft from Brown University considers the impact of schools hiring new teachers after the school year has started.
The researchers used a comprehensive administrative dataset from a large, urban school district in the southern United States that includes student, teacher, and test records from the 1999-2000 to the 2009-2010 school years.
They found that students in grades four to eight (Years 5- 9) in classrooms with teachers hired after the start of the school year do worse than their peers with other newly hired teachers (effect size= -0.042 SD in mathematics, -0.026 SD in reading). A substantial part of this effect comes from temporary disruptions that affect teachers and students in the year when a teacher is hired late. In mathematics, but not in reading, they found that schools that hire late lose stronger candidates. Finally, they found that teachers who were hired late leave their schools, and the district, at much greater rates than their peers who are hired on-time, which may have negative impacts on other teachers and students in the school. Thus, delayed hiring prevents schools from hiring, supporting, and retaining effective teachers.
Source: The productivity costs of inefficient hiring practices: Evidence from late teacher hiring (2016), Journal of Policy Analysis and Management
The edTPA is an assessment in the US, introduced in 2013, that evaluates prospective teachers’ classroom performance. It is used by more than 600 teacher education programmes in 40 states, and passing it is a requirement for licensure in 7 states. In an attempt to discern whether the test can accurately determine if teacher candidates who achieve higher scores on this test help their students better than lower-scoring candidates, The National Center for Analysis of Longitudinal Data in Education Research (CALDER) conducted the first independent study of edTPA, and found mixed results.
The study followed 2,300 teacher candidates in Washington State who took the edTPA in 2014. Their scores were correlated with their students’ standardised test scores in reading and maths. The study found that new teachers who passed the edTPA on their first try increased their students’ reading achievement scores more than new teachers who didn’t pass edTPA on their first attempt. There were no differences regarding the effects on students’ math scores.
The authors discuss the complicated implications of these findings for policy and practice. For example, they state that new teachers who fail the test the first time may ultimately become high-performing teachers, and warn of screening them out of the workforce.
Source: Evaluating Prospective Teachers: Testing the Predictive Validity of the edTPA (2016), National Center for Analysis of Longitudinal Data in Education Research (CALDER)
A new working paper from Mathematica Policy Research looks at the role that teachers play in developing non-cognitive skills, the non-tested academic behaviours and mindsets that contribute to children’s long-term success. These behaviours and mindsets include emotional stability, motivation, persistence, and self-control.
Data came from 310 teachers in four US districts who had agreed to have their classes videotaped, complete a teacher questionnaire, and help collect a set of pupil outcomes. The study focused on Grade 4 and 5 (Year 5 and 6) maths classes, although all of the teachers involved were generalists.
The authors examined both “teacher effects” (the teacher themselves) and “teaching effects” (classroom practices) on a range of maths test scores and non-tested outcomes, specifically behaviour in class, happiness in class, and self-efficacy in maths.
They found that individual teachers have large effects on pupils’ self-reported behaviour in class, self-efficacy in maths, and happiness in class that are similar in magnitude to effects on test scores. However, teachers who are effective at improving these outcomes often are not the same as those who raise maths test scores.
The paper concludes that efforts to improve the quality of the teacher workforce should include teachers’ abilities to promote academic behaviours and mindsets.
Source: Teacher and teaching effects on students’ academic behaviors and mindsets (2015), Mathematica.
A US programme intended to boost pupil achievement by providing teachers with two years of professional development, including formal training sessions and meetings with a leadership coach, is showing signs of potential, according to a new RAND Corporation report.
The Leading Educators (LE) Fellowship programme, selects mid-career teachers through a competitive application process. To examine the impact of LE, researchers are comparing pupil achievement gains for teachers who participated in the programme as fellows or mentees with the pupil achievement gains of other teachers.According to RAND, early findings of the programme are mixed, but suggest that it shows promise in improving pupil achievement. Specifically, they report:
- Among fellows, there are both some statistically significant positive and negative programme effects on pupil achievement, with results that vary across states, subject areas, and model specifications.
- Among mentee teachers, for whom sample sizes are larger, there is some suggestive evidence of impacts on pupil achievement — in particular, marginally significant and significant positive programme effects among mentees who teach maths and social studies, respectively, in Louisiana.
- The impact of the programme on teacher retention is unclear, with no consistent pattern of retention impacts across cohorts or states.
The authors note that the current results are based on few years of data and on a small sample of teachers, and results may change when there are more fellows and mentored teachers included in future studies.
Source: Examining the Early Impacts of the Leading Educators Fellowship on Student Achievement and Teacher Retention (2015), RAND Corporation.