A new working paper, published by Brown University, reports on an online coaching programme that aimed to support maths teachers.
Mathematical Quality of Instruction (MQI) is an observational instrument that helps structure teachers’ and coaches’ reflections about maths teaching. There are four “dimensions” of the instrument – richness of the mathematics, Common Core-aligned pupil practices, working with pupils and mathematics, and teacher errors. In several studies, teachers’ scores on MQI have predicted pupils’ academic achievement gains.
In MQI coaching, teachers selected an element of their practice to work on and filmed one of their lessons. A coach then selected a couple of extracts and chose a comparison stock film clip. The teacher watched the clips and then the two discussed them and developed a plan for improvement.
In the current trial, 142 upper elementary and middle school teachers (Years 4-9) from 51 schools in a mid-western state in the US were assigned to MQI coaching or a control condition. Teachers assigned to MQI coaching took part in a two-day summer school, followed by a bi-weekly coaching cycle for the following academic year.
At the end of that year, the coached teachers showed substantial improvements in their scores on the MQI instrument. They also had improved scores in pupil perceptions of classroom practices. However, there was no measurable impact on state achievement tests (the Smarter Balance Assessment Consortium (SBAC)). In the follow-up year, there were still large impacts on the four MQI dimensions, but none were found on pupils’ assessment of teachers or pupil achievement.
Source: Developing ambitious mathematics instruction through web-based coaching: A randomized field trial. (November 2018). Brown University Working Paper
Findings from an evaluation of a $575 million programme to improve teacher performance found that, while sites implemented new measures of teaching effectiveness and modified personnel policies accordingly, the programme had no impact on pupil outcomes.
The Intensive Partnerships for Effective Teaching initiative, designed and funded by the Bill and Melinda Gates Foundation, aimed to dramatically improve pupil outcomes by improving pupils’ access to effective teaching. Three US school districts and four charter management organisations participated in the programme, which ran between 2009 and 2016.
The final evaluation report, published by the RAND Corporation, found that by the end of 2014-15, outcomes for pupils in the settings that took part in the initiative were not better than outcomes for pupils in similar settings that did not take part. There was no evidence that low-income minority (LIM) pupils had greater access than non-LIM pupils to effective teaching. In addition, it found very few instances of improvement in the effectiveness of teaching overall, and no improvement in the effectiveness of newly hired teachers compared to experienced teachers. The evaluation also found no increase in the retention of effective teachers, although there was some decline in the retention of ineffective teachers in most settings that took part in the initiative.
The report states several possible reasons that the initiative failed to achieve its goals for improving pupil outcome:
- incomplete implementation of the key policies and practices
- the influence of external factors, such as state-level policy changes during the initiative
- insufficient time for effects to appear
- a flawed theory of action
- a combination of all these factors.
Source: Improving teaching effectiveness: Final report: The Intensive Partnerships for Effective Teaching through 2015–2016 (2018), RAND Corporation.
Matthew A Kraft and colleagues conducted a meta-analysis of the causal evidence on the effect of teacher coaching on teaching and learning. Their paper, published in the Review of Educational Research, reviewed 60 studies on teacher coaching programmes conducted after 2006 that measured the impact of teacher coaching on either teaching (measured using tools such as the Classroom Assessment Scoring System or the Early Language and Literacy Classroom Observation) or pupil academic performance (measured by standardised tests).
Their results found that sustained coaching improves both classroom teaching and pupil achievement, with pooled effect sizes of +0.49 standard deviations for teaching and +0.18 standard deviations for academic achievement.
However, the effectiveness of a teacher coaching programme seems to be determined by the number of participants. When studies were divided into programmes that had fewer than 100 participants and those that had more than 100 participants, the impact on teaching was nearly double for the smaller programmes than for programmes with more than 100 participants. For pupil achievement, the smaller programmes showed an impact of nearly three times that of the larger programmes.
Source: The effect of teacher coaching on instruction and achievement: A meta-analysis of the causal evidence (February 2018), Review of Educational Research, Vol 88, Issue 4
Franziska Egert and colleagues in Germany and Amsterdam have conducted a review of the effects of professional development (PD) for early childhood educators on programme quality and children’s educational outcomes.
Studies were only included if they addressed quality of child care or child development, included early childhood teachers (including preschool, kindergarten and centre-based care), were quantitative, were experimental or quasi-experimental, reported effect sizes or data and addressed children 0–7 years old. This yielded 36 studies of 42 programmes evaluating quality ratings, and nine studies of 10 programmes evaluating both quality ratings and pupil outcomes.
Results showed that professional development improved the external quality ratings (as evaluated using the Classroom Assessment Scoring System, Early Language and Literacy Classroom Observation, Environmental Rating Scales and Individualized Classroom Assessment Scoring System) of early childhood education (effect size=+0.68), with programmes providing 45–60 PD hours having the greatest impact on classroom practice as compared to programmes offering fewer or more hours. This was true regardless of whether teachers held a university degree or not. Further, programmes that solely used coaching were almost three times as effective as other programmes. A second meta-analysis of a subset of studies (n=486 teachers, 4,504 children) showed that improvement in the quality of early childhood education programmes was correlated with improvements in child development (effect size=+0.14) as determined by language and literacy scores, maths scores, social-behavioural ratings, and assessment of cognition, knowledge and school readiness.
Source: Impact of in-service professional development programs for early childhood teachers on quality ratings and child outcomes: a meta-analysis (January 2018), Review of Educational Research, Vol 88, Issue 3
An SRI Education evaluation of the New Teacher Center’s (NTC’s) induction model shows some positive results on pupil achievement in mathematics and English language arts (reading, writing, and linguistic/communication skills).
The NTC induction model provides new teachers with two years of coaching from a trained mentor. New teachers meet with their assigned mentor for a minimum of 180 minutes each month and work through a programme of NTC-developed support. The evaluation, conducted by Rebecca Schmidt and colleagues, reports on findings from a three-year randomised controlled trial of NTC’s induction model in two US school districts (one in Florida and the other in Illinois). New teachers in participating schools were randomly assigned to receive either the NTC induction model (the treatment condition) or business-as-usual new teacher support (the control condition).
Pupils in grades 4–8 (Years 5–9) who were taught by teachers who had participated in NTC induction for two years did better in English language arts (effect size = +0.09) and maths (effect size = +0.15) compared to pupils of control teachers.
Source: Impact of the New Teacher Center’s new teacher induction model on teachers and students (June 2017), SRI Education, SRI International
A study published by the Institute of Education Sciences in the US evaluates the impact of the Retired Mentors for New Teachers programme – a two-year programme in which recently retired teachers provide tailored mentoring to new teachers – on pupil achievement, teacher retention and teacher evaluation ratings. The new teachers meet with their mentors weekly on a one-to-one basis and monthly in school-level groups over the course of the two years.
Dale DeDesare and colleagues conducted a randomised controlled trial involving 77 teachers at 11 primary schools in Aurora, Colorado. Within each school, half of the new teachers were randomly assigned to a control group to receive the district’s business-as-usual mentoring support, while the other half received the intervention as well as business-as-usual mentoring support.
The study found that at the end of the first year, pupils who were taught by teachers in the programme group scored 1.4 points higher on the spring Measures of Academic Progress maths assessment than those taught by teachers in the control group, (effect size = +0.064), and this difference was statistically significant. Reading achievement was also higher among pupils taught by teachers in the programme group, however, the difference was not statistically significant (effect size = +0.014 at the end of the first year and +0.07 at the end of the second year). The effect of the programme on teacher evaluation ratings and teacher retention was not significant, although more teachers in the programme group left after two years than in the control group.
Source: Impacts of the retired mentors for new teachers program (REL 2017–225) (March 2017), US Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Central.