Targeted Reading Intervention and struggling readers

Targeted Reading Intervention (TRI) is a programme that uses webcam technology to allow kindergarten and first grade (Year 1 and Year 2) teachers to help struggling readers while being observed by a coach who gives them real-time feedback as they work with a pupil. TRI trains teachers in their strategies during a three-day workshop in the summer, with webcam observations and feedback during the school year.

Researchers at the University of Delaware and the University of North Carolina evaluated the effect of TRI in a two-year randomised evaluation  to examine its one-year effects on struggling readers, and to examine if having a teacher teach the programme for two years affected pupil achievement.

The study took place in kindergarten and first grade (Year 1 and Year 2) classrooms in ten schools in high-poverty south eastern rural counties in the US. Subjects were equivalent at baseline on standardised testing in the autumn, and randomisation occurred at the classroom level. During the two years of the study, a total of 50 kindergarten (Year 1) classrooms (26 treatment, 24 control) and 50 first grade (Year 2) classrooms (29 treatment, 21 control) at each school were randomised, and then three struggling readers from each classroom were selected to either the treatment or control condition in each year of the study. In total, 305 pupils were assigned to receive TRI training, and 251 pupils served in the untreated control group. Treatment pupils worked with teachers one-to-one, 15 minutes a day every day for six to eight weeks. Spring post-tests showed that struggling readers who received TRI showed greater gains than struggling readers in the control condition (effect size =+0.26). Longevity of teaching the programme did not show any significant effect on pupil achievement.

Researchers also report on the results for the subset of pupils experiencing the programme who had English as an additional language, which may be found here.

Source: Improving struggling readers’ early literacy skills through a tier 2 professional development program for rural classroom teachers: The Targeted Reading Intervention (June2018), The Elementary School Journal 2018 118:4, 525-548

A systematic review of RCTs in education research

The use of randomised controlled trials (RCTs) in education research has increased over the last 15 years. However, the use of RCTs has also been subject to criticism, with four key criticisms being that it is not possible to carry out RCTs in education; the research design of RCTs ignores context and experience; RCTs tend to generate simplistic universal laws of “cause and effect”; and that they are descriptive and contribute little to theory.

To assess these four key criticisms, Paul Connolly and colleagues conducted a systematic review of RCTs in education research between 1980 and 2016 in order to consider the evidence in relation to the use of RCTs in education practice.

The systematic review found a total of 1,017 RCTs completed and reported between 1980 and 2016, of which just over three-quarters have been produced in the last 10 years. Just over half of all RCTs were conducted in North America and just under a third in Europe. This finding addresses the first criticism, and demonstrates that, overall, it is possible to conduct RCTs in education research.

While the researchers also find evidence to oppose the other key criticisms, the review suggests that some progress remains to be made. The article concludes by outlining some key challenges for researchers undertaking RCTs in education.

Source:  The trials of evidence-based practice in education: a systematic review of randomised controlled trials in education research 1980–2016 (July 2018), Educational Research, DOI: 10.1080/00131881.2018.1493353

$575 million programme has no impact on pupil outcomes

Findings from an evaluation of a $575 million programme to improve teacher performance found that, while sites implemented new measures of teaching effectiveness and modified personnel policies accordingly, the programme had no impact on pupil outcomes.

The Intensive Partnerships for Effective Teaching initiative, designed and funded by the Bill and Melinda Gates Foundation, aimed to dramatically improve pupil outcomes by improving pupils’ access to effective teaching. Three US school districts and four charter management organisations participated in the programme, which ran between 2009 and 2016.

The final evaluation report, published by the RAND Corporation, found that by the end of 2014-15, outcomes for pupils in the settings that took part in the initiative were not better than outcomes for pupils in similar settings that did not take part. There was no evidence that low-income minority (LIM) pupils had greater access than non-LIM pupils to effective teaching. In addition, it found very few instances of improvement in the effectiveness of teaching overall, and no improvement in the effectiveness of newly hired teachers compared to experienced teachers. The evaluation also found no increase in the retention of effective teachers, although there was some decline in the retention of ineffective teachers in most settings that took part in the initiative.

The report states several possible reasons that the initiative failed to achieve its goals for improving pupil outcome:

  • incomplete implementation of the key policies and practices
  • the influence of external factors, such as state-level policy changes during the initiative
  • insufficient time for effects to appear
  • a flawed theory of action
  • a combination of all these factors.

 

Source:  Improving teaching effectiveness: Final report: The Intensive Partnerships for Effective Teaching through 2015–2016 (2018), RAND Corporation.

The effect of teacher coaching on teaching and learning

Matthew A Kraft and colleagues conducted a meta-analysis of the causal evidence on the effect of teacher coaching on teaching and learning. Their paper, published in the Review of Educational Research, reviewed 60 studies on teacher coaching programmes conducted after 2006 that measured the impact of teacher coaching on either teaching (measured using tools such as the Classroom Assessment Scoring System or the Early Language and Literacy Classroom Observation) or pupil academic performance (measured by standardised tests).

Their results found that sustained coaching improves both classroom teaching and pupil achievement, with pooled effect sizes of +0.49 standard deviations for teaching and +0.18 standard deviations for academic achievement.

However, the effectiveness of a teacher coaching programme seems to be determined by the number of participants. When studies were divided into programmes that had fewer than 100 participants and those that had more than 100 participants, the impact on teaching was nearly double for the smaller programmes than for programmes with more than 100 participants. For pupil achievement, the smaller programmes showed an impact of nearly three times that of the larger programmes.

Source:  The effect of teacher coaching on instruction and achievement: A meta-analysis of the causal evidence (February 2018), Review of Educational Research, Vol 88, Issue 4

Numeracy intervention shows promise for pupils struggling with maths

The Education Endowment Foundation has published an evaluation of 1stClass@Number, a 10-week numeracy intervention, delivered by teaching assistants, that provides intensive support for pupils struggling with maths.

A randomised controlled trial was conducted in 133 schools in south and west Yorkshire. Schools each nominated four children in Year 2 to participate, and the schools were then randomly assigned to either receive the intervention or to continue with normal teaching. A team from the University of Oxford evaluated the programme, which was delivered three times a week for 10 weeks in addition to normal mathematics instruction. A process evaluation collected additional data through observations, questionnaires and phone interviews.

Results showed that the intervention had a positive effect on Quantitative Reasoning Tests (effect size = +0.18) compared to pupils in the control group. Among pupils eligible for free school meals, those in the intervention group did not make any additional progress in the Quantitative Reasoning Test compared to control group pupils.

1stClass@Number seemed to have no impact on performance in end of Key Stage 1 maths tests compared to pupils in the control group. However, there was some evidence that the intervention widened the gap in Key Stage 1 maths results between pupils eligible for free school meals and their peers.

Source: 1stClass@Number: Evaluation report and executive summary (July 2018), Education Endowment Foundation

Classroom-based intervention for pupils with autism

Findings from a cluster randomised trial published in the Journal of Consulting and Clinical Psychology suggest that classroom teachers can effectively deliver a programme for young pupils with Autistic Spectrum Disorder (ASD) that results in better outcomes relative to usual school-based education.

Lindee Morgan and colleagues conducted a trial of the Social, Communication, Emotional Regulation and Transactional Support (SCERTS) intervention – a classroom-based, teacher-implemented intervention aimed at improving active engagement, adaptive communication, social skills, executive functioning and problem behaviour in elementary (primary) school pupils with ASD – to assess what improvement pupils in the intervention group made across a variety of measures compared to pupils in the control group. Sixty schools from three US states were randomly assigned to either the intervention or control groups. Teachers in the intervention group were trained in how to deliver the SCERTS programme and received coaching throughout the school year.

Results showed better outcomes for the intervention group than the control group on observed measures of classroom active engagement with respect to social interaction. The intervention group also had better outcomes on measures of adaptive communication, social skills and executive functioning (effect sizes ranged from +0.31 to +0.45).

Source: Cluster randomized trial of the classroom SCERTS intervention for elementary students with autism spectrum disorder (July 2018), Journal of Consulting and Clinical Psychology, Volume 86(7)