Early impact of Diplomas Now

MDRC has published the first results from a randomised controlled trial of Diplomas Now, a whole-school reform initiative. Under the Diplomas Now programme:

  • Schools are reorganised so that small groups of teachers work consistently with the same population of students.
  • There is an intensive peer coaching system for maths and English teachers.
  • Early warning indicators are used to identify students who need different types of support.
  • Additional staff help coordinate the transformation, introduce new practices and structures, provide training and support to school staff members, provide additional services to students, and engage with families and community organisations.

In total, 62 schools (33 middle schools and 29 high schools) from 11 large urban districts were recruited. Thirty-two of the participating schools were randomly assigned to implement the Diplomas Now model (DN schools), and 30 were assigned to continue with “business as usual” (non-DN schools).

So far, the study team has been able to explore early impacts for sixth- and ninth-grade (Year 7 and 10) students moving into DN schools during the first two years of the programme. For this cohort of students, DN schools were more successful than non-DN schools in reducing the number of early warning indicators (a statistically significant 3.6 percentage point reduction). The early warning indicator was a combination of daily attendance of 85% or less, suspensions or expulsions for a total of three or more days, and failing grades in English or maths classes. However, the DN programme made no statistically significant impact on any of these measures separately. The project will continue for several more years.

Source: Addressing Early Warning Indicators: Interim Impact Findings from the Investing in Innovation (i3) Evaluation of Diplomas Now (2016), MDRC.

Is professional development better than being dismissed?

The last issue of Best Evidence in Brief reported on a study in which low-performing teachers were dismissed. A new working paper from the National Bureau of Economic Research reports on an experiment where low-performing teachers were provided with coaching from higher-performing peers.

The experiment took place in Tennessee in 14 elementary and middle schools. Tennessee teachers are observed in the classroom many times each year, and scored on 19 specific skills (eg, questioning, lesson structure and pacing, and managing student behaviour). Schools were randomly assigned to a treatment condition or business-as-usual control group. In the treatment schools, low-performing “target” teachers were matched with high-performing teachers, based on the outcomes of their classroom observations. The high-performing teachers were chosen based on their high scores in skills for which the low-performing teachers had received a low score. The pairs were encouraged to work together on these skills, as well as more generally on observing each other’s teaching, discussing strategies for improvement, and following up on each other’s commitments throughout the year.

After a year, students in treatment schools (whether taught by target or non-target teachers) showed a small improvement (effect size +0.06) on maths and English tests, when compared with students in control schools. Gains by students taught by target teachers were higher (+0.12). These improvements persisted and grew. In the following year, the effect for target teachers was a marginally significant +0.25.

Source: Learning Job Skills from Colleagues at Work: Evidence from a Field Experiment Using Teacher Performance Data (2016), The National Bureau of Economic Research.

Content-focused coaching shows positive effects

This article from Learning and Instruction presents findings from a group-randomised trial investigating the effect of Content-Focused Coaching (CFC).

A key element of CFC is “Questioning the Author (QtA)”, a discussion-based approach to reading comprehension. According to the article, QtA encourages teachers and pupils to work together to construct the meaning of a text during the reading process. Teachers strategically pose questions to pupils at key places in a text that promote understanding, interpretation, and elaborated response, and encourage pupils to share and challenge each other’s ideas to grapple with these questions.

Schools assigned to the treatment condition received a CFC-trained coach, and schools in the comparison condition continued with the literacy coaching that was standard practice in their school. The final sample included 29 US schools serving a high proportion of pupils with English as an Additional Language (EAL) and pupils from low-income families.

Findings showed a positive effect of the CFC programme on observed classroom text discussion quality. Findings also showed a positive effect on pupil reading achievement, as measured on a state assessment test, with stronger effects for EAL pupils compared to their English-proficient peers.

The authors note that additional research is needed to examine the effectiveness and feasibility of adopting CFC on a wider scale.

Source: Literacy Coaching to Improve Student Reading Achievement: A Multi-level Mediation Model (2013), Learning and Instruction, 25.

Key components of successful coaching

Head Start CARES (Classroom-based Approaches and Resources for Emotion and Social Skill Promotion) is a large-scale, US national research demonstration to test a one-year programme to improve pre-kindergarteners’ (age 4–5) social and emotional readiness for school. To facilitate the delivery of the programme, teachers attended training workshops and worked with coaches throughout the school year. In this report from MDRC, researchers present lessons learned from Head Start CARES about coaching social-emotional curricula in a large and complex early childhood education system. Key findings include:

  • Successful coaches exhibited a combination of skills in three important areas: knowledge of the programme, general coaching and consultation skills, and knowledge of and experience in early childhood development and/or teaching.
  • Incorporating coaching into day-to-day practices requires flexibility and is necessary for implementation success.
  • Site-level administrators must be actively engaged in supporting and supervising coaching as well as general implementation processes.

Source: Coaching as a Key Component in Teachers’ Professional Development: Improving Classroom Practices in Head Start Settings (2012), MDRC

Evaluating reading coach quality

A study from the RAND Corporation examines what makes for good reading coaches and coaching. The study included 113 schools from 8 districts in Florida. All used reading coaches to work with school staff to improve their reading teaching and leadership skills. The data showed no relationship between teacher and principal perceptions of coach quality and students’ reading achievement.

The researchers suggest that being an effective literacy coach may require more than content-area expertise and experience teaching children. They identify “understanding how to support adult learners” as a key area of expertise that was sometimes lacking with the coaches in the study.

Source: Reading Coach Quality: Findings from Florida Middle Schools (2012), Literacy Research and Instruction, 51(1).