The Institute of Education Sciences has released a new evaluation brief that synthesises findings from two impact studies conducted by the National Center for Education Evaluation (NCEE). One study focused on a strategy of providing teachers with feedback on their performance for two years (performance feedback), and the other study focused on a strategy of providing teachers with bonuses for four years based on their performance (pay-for-performance). Both strategies were supported by the Teacher Incentive Fund, which provided competitive grants to help US states and districts implement a multi-strategy approach to enhancing teacher effectiveness.
In each study, elementary and middle schools (primary schools) were randomly assigned to implement the strategy (the treatment group) or not (the control group). The performance feedback study included approximately 29,000 pupils and 1,000 teachers in grades 4–8, while the pay-for-performance study included approximately 38,000 pupils and 3,500 teachers in grades 3–8. Pupil outcomes were measured using end-of-year reading and maths scores.
Key findings were as follows:
- Providing teachers with feedback on their performance for two years improved pupils’ maths achievement after the first year with a difference in scores that corresponds to an effect size of +0.05. The cumulative effect after two years of implementation was similar in magnitude but not statistically significant. The effect on reading in both years was positive but not statistically significant.
- Providing teachers with bonuses based on their performance for four years improved pupils’ reading achievement after one, two and three years of implementation and pupils’ maths achievement after three years. After each of those periods of implementation, the effect size was +0.04 for reading and +0.06 for maths. However, as noted in the evaluation report, the impacts of pay-for-performance on classroom observation ratings did not appear to explain the impacts on pupil achievement, and in treatment schools, as many as 40% of teachers were unaware that they could earn a performance bonus.
The brief was prepared for NCEE by Andrew Wayne and Michael Garet of American Institutes for Research and Alison Wellington and Hanley Chiang of Mathematica Policy Research.
Source: Promoting educator effectiveness: the effects of two key strategies (March 2018), National Center for Education Evaluation, The Institute of Education Sciences
The What Works Clearinghouse (WWC) has released a new intervention report that examines the research on the impact of summer counselling on pupils’ college enrolment and persistence. As the report notes, summer counselling is designed to help college-intending high school graduates complete the steps needed to enrol in college and start their college careers.
The review included five studies of summer counselling that met the WWC’s research standards without reservations. Together, these studies included 13,614 recent high school graduates in 10 locations. In all five studies, pupils were provided with summer counselling for about 1.5 months between high school graduation and college registration. During this time, programmes provided college-intending individuals with information about tasks required for college enrolment, as well as assistance in overcoming unanticipated financial, informational and socioemotional barriers that prevent college entry.
According to the report, the research shows that summer counselling had potentially positive effects on credit accumulation and persistence (extent of evidence: small) and mixed effects on college access and enrolment (extent of evidence: medium to large) for recent high school graduates.
Source: Transition to college intervention report: Summer counseling (March 2018), What Works Clearinghouse, Institute of Education Sciences, US Department of Education.
Teacher Incentive Fund (TIF) grants were awarded in 2010 by the US Department of Education to support performance-based compensation systems for teachers and principals in high-need schools.
In order to assess the impacts of pay-for-performance on educator (teachers and principals) and pupil outcomes, an experimental study design was used in ten US school districts to randomly assign elementary and middle schools to treatment and control groups. Both groups implemented the same performance-based compensation system, but in the control schools, the pay-for-performance element was replaced by a one percent bonus paid to all teachers and principals regardless of performance. A fourth and final report from this evaluation has now been published, covering all four years of the programme (between 2011 and 2015).
Among the key findings are that pay-for-performance had small, positive impacts on pupil achievement by the second year of implementation. From that year onward, reading and maths achievement was higher by 1 to 2 percentile points in schools that offered performance bonuses than in schools that did not. However, it was not entirely clear how this improvement was achieved. The impacts of pay-for-performance on classroom observation ratings did not appear to explain the impacts on pupil achievement, and in treatment schools as many as 40% of teachers were unaware that they could earn a performance bonus.
Source: Evaluation of the Teacher Incentive Fund: Final report on implementation and impacts of pay-for-performance across four years (NCEE 2017-4004),(December 2017), National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, US Department of Education.
The What Works Clearinghouse has released a new practice guide, Preventing Dropout in Secondary Schools , that offers research-based recommendations for reducing dropout rates in middle and secondary schools. The goal is to help educators and administrators learn strategies for identifying at-risk pupils and addressing the challenges they face.
The WWC and an expert panel chaired by Russell Rumberger from the University of California, Santa Barbara synthesised existing research on the topic and combined it with insight from the panel to identify the following four recommendations, which include a rating of the strength of the research evidence supporting each recommendation:
- Monitor the progress of all pupils, and proactively intervene when pupils show early signs of attendance, behaviour, or academic problems (minimal evidence).
- Provide intensive, individualised support to pupils who have fallen off track and face significant challenges to success (moderate evidence).
- Engage pupils by offering curricula and programmes that connect schoolwork with college and career success and that improve pupils’ capacity to manage challenges in and out of school (strong evidence).
- For schools with many at-risk pupils, create small, personalised communities to facilitate monitoring and support (moderate evidence).
Each recommendation provides specific, actionable strategies; examples of how to implement the recommended practices in schools; advice on how to overcome potential obstacles; and a description of the supporting evidence.
Source: Preventing dropout in secondary schools U.S. (September 2017), What Works Clearing House, Institute of Education Sciences Practice Guide
A study published by the Institute of Education Sciences in the US evaluates the impact of the Retired Mentors for New Teachers programme – a two-year programme in which recently retired teachers provide tailored mentoring to new teachers – on pupil achievement, teacher retention and teacher evaluation ratings. The new teachers meet with their mentors weekly on a one-to-one basis and monthly in school-level groups over the course of the two years.
Dale DeDesare and colleagues conducted a randomised controlled trial involving 77 teachers at 11 primary schools in Aurora, Colorado. Within each school, half of the new teachers were randomly assigned to a control group to receive the district’s business-as-usual mentoring support, while the other half received the intervention as well as business-as-usual mentoring support.
The study found that at the end of the first year, pupils who were taught by teachers in the programme group scored 1.4 points higher on the spring Measures of Academic Progress maths assessment than those taught by teachers in the control group, (effect size = +0.064), and this difference was statistically significant. Reading achievement was also higher among pupils taught by teachers in the programme group, however, the difference was not statistically significant (effect size = +0.014 at the end of the first year and +0.07 at the end of the second year). The effect of the programme on teacher evaluation ratings and teacher retention was not significant, although more teachers in the programme group left after two years than in the control group.
Source: Impacts of the retired mentors for new teachers program (REL 2017–225) (March 2017), US Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Central.
The Institute of Education Sciences (IES) in the US has released a new research report on Saxon Math, and findings show mixed results for the programme.
Saxon Math is a curriculum for pupils in grades K-12 (Years 1-13). It uses an incremental structure that distributes content throughout the year. For the IES report, researchers reviewed studies of Saxon Math’s primary courses, which include kindergarten (Year 1) through pre-algebra. Out of 26 studies eligible for review, five studies fell within the scope of the What Work Clearinghouse’s (WWC) primary maths topic area and met WWC design standards. These five studies included 8,855 pupils in grades 1–3 and 6–8 in 149 schools across at least 18 states.
According to the report, the estimated impact of the intervention on outcomes in the mathematics achievement domain was positive and substantively important in two studies and indeterminate in three studies. The authors conclude that Saxon Math has mixed effects on maths test scores of pupils in primary classes.
Source: Saxon Math (May 2017), US Department of Education, Institute of Education Sciences, What Works Clearinghouse