The impact of closing failing schools

Stanford University’s Center for Research on Education Outcomes (CREDO) has released findings from a new study on school closures. The report, authored by Chunping Han and colleagues, systematically examines closure of low-performing charter and traditional public schools (TPS) in the US. A main goal of the study was to see whether children whose schools had been closed for poor performance do better or worse in their new schools.

The authors used existing longitudinally linked data that CREDO had developed in partnership with 26 state education agencies. They identified low-performing, full-time, regular (non-alternative) schools and closures in those 26 states from academic year 2006-07 to 2012-13. A total of 1,522 low-performing schools, including 1,204 TPS and 318 charters, were closed in the 26 states during the study period. To measure academic performance across the low-performing schools, the authors used scores from state standardised achievement tests.

Key findings of the study included:

  • A little less than half of displaced closure pupils landed in better schools.
  • In both the charter and traditional public school sectors, low-performing schools with a larger share of black and Hispanic pupils were more likely to be closed than similarly performing schools with a smaller share of disadvantaged minority pupils.
  • The quality of the receiving school made a significant difference in post-closure pupil outcomes. Closure pupils who attended better schools post-closure tended to make greater academic gains than did their peers from not-closed low-performing schools in the same sector, while those ending up in worse or equivalent schools had weaker academic growth than their peers in comparable low-performing settings.

Source: Lights off: Practice and impact of closing low-performing schools (August 2017), Center for Research on Education Outcomes (CREDO)

Open Court Reading receives judgement in multi-year trial

A multi-year scale-up study has examined the effectiveness of Open Court Reading (OCR), a phonics-based curriculum for grades K-6 (Reception to Year 7).

The study, by Michael Vaden-Kiernar and colleagues, and published in the Journal of Research on Educational Effectiveness, was a school-level, cluster randomised trial, involving around 4,500 pupils and 1,000 teachers in 49 elementary schools across the US.

The OCR curriculum includes pupil materials, teacher manuals, diagnostic and assessment tools and test preparation practice guides. In all grades (K-5), the instructional format is a three-part lesson with specific instruction in phonemic awareness and phonics, vocabulary and comprehension skills and writing skills. The programme includes one- or two-day summer workshops at the start of each school year to train teachers on programme implementation, and ongoing support by OCR reading consultants.

An implementation study showed adequate to high levels of fidelity to the programme. However, there were no statistically significant effects on reading performance in Year 1, and a small negative effect (effect size = -0.09) in Year 2. Relative to the “business-as-usual” controls, no positive overall impacts of OCR and mixed impacts for pupil subgroups were found.

Source: Findings from a multiyear scale-Up effectiveness trial of Open Court Reading (June 2017), Journal of Research on Educational Effectiveness

The world’s largest school-based mental health programmes

School-based mental health programmes are now delivered to millions of children, particularly in high-income countries.

J Michael Murphy and colleagues have published a review of those programmes that reach the largest populations. In an article for the Harvard Review of Psychiatry, they profile eight programmes that have shown their effectiveness using randomised controlled trials or quasi-experimental studies. Most of the programmes focus on primary prevention or target specific vulnerable populations, rather than being aimed at supporting children with diagnosed disorders.

For each of the programmes, there is a description of its approach and a review of the supporting evidence. The eight programmes are:

  • Positive Behavioral Interventions & Supports
  • FRIENDS
  • Positive Action
  • Promoting Alternative Thinking Strategies (PATHS)
  • Skills for Life
  • MindMatters
  • Good Behavior Game
  • Cognitive-Behavioral Intervention for Trauma in Schools

Between them, the programmes have reached more than 25 million children, although the authors point out that these have mainly been in high-income countries. There is limited evidence that these programmes could be implemented at scale in low- and middle-income countries.

The authors suggest that, with the growing availability and diversity of programmes, greater attention can now be paid to assessing the processes and practices of implementation that are associated with successful, widely disseminated and sustainable programmes.

Source: Scope, scale, and dose of the world’s largest school-based mental health programs (August 2017), Harvard Review of Psychiatry

The impact of more teachers and smaller classes in the early years

What difference do smaller class sizes, and more teachers, make in early childhood education (ECE)?

A meta-analysis by Jocelyn Bowne and colleagues, published in Educational Evaluation and Policy Analysis, attempts to find some answers. The analysis included evaluations of ECE programs in the US between 1960 and 2007. The evaluations were either experimental studies, used a high-quality quasi-experimental design, or showed baseline equivalence of treatment and control participants. In total, 38 studies were included, all of which looked at children ages 3 to 5 years old attending an ECE center for 10 hours a week or more for at least 4 months. Child-teacher ratios ranged from 5:1 to 15:1 and class sizes from 11 to 25.

The findings were as follows:

  • Above a child–teacher ratio of 7.5:1, changing the ratio had no effect on children’s cognitive and achievement outcomes. Below this, a reduction of the ratio by one child per teacher predicted an effect size of +0.22.
  • For class sizes greater than 15, increasing the size of the class had little effect on children’s cognitive and achievement outcomes. Below this, one child fewer in the class size predicted an effect size of +0.10.

The authors caution that these findings are correlational, rather than causal, so changing class sizes or ratios, certainly at scale, may not lead to these results. However, they conclude that “very small and/or well-staffed classrooms might confer some small benefits for children’s cognitive and academic learning”.

Source: A meta-analysis of class sizes and ratios in early childhood education programs: Are thresholds of quality associated with greater impacts on cognitive, achievement, and socioemotional outcomes? (February 2017), Educational Evaluation and Policy Analysis, Vol 39, Issue 3

Evaluation of Success for All

The Education Endowment Foundation recently published a report on the effectiveness trial of the Success for All (SFA) programme to evaluate its impact on the literacy outcomes of Reception pupils. SFA is a whole-school approach to improving literacy in primary schools. All teachers and senior leaders are involved, with the school receiving a total of 16 training and support days. Teachers receive pedagogical training – for example on effective phonics teaching – and teaching materials such as structured lesson plans. For the school leadership team, there is support in areas such as data management, ability grouping and parental engagement.

Fifty-four schools took part in this effectiveness trial, which was evaluated by Sarah Miller and colleagues from Queen’s University Belfast. Although the intervention was delivered on a whole-school basis, the evaluation focused only on the outcomes of 1,767 pupils starting in Reception, and followed them through to the end of Year 1.

The main analysis found that Reception pupils in SFA schools made more progress than pupils in control schools after two years (effect size = +0.07). The effect was slightly larger for pupils eligible for free school meals (FSM), compared to FSM pupils in control schools after two years (effect size = +0.12). In both cases, the effect was smaller than those found in previous evaluations (this is the third RCT of SFA to be conducted, and the first independent trial of the programme in England). Trials in the US reported effect sizes between +0.15 and +0.30. The report suggests that one possible reason for this was that some schools struggled to implement the programme as intended.

The project delivery team was from the University of York. Robert Slavin, director of the US Center for Research and Reform in Education, is Co-founder and Chairman of the Board of the Success for All Foundation.

Source: Success for All: Evaluation report and executive summary (July 2017), Education Endowment Foundation

Programme components and disadvantaged pupils

Research shows that pupils from low socioeconomic backgrounds are less likely to attend pre-school or to have a home environment incorporating literacy and language activities than their less disadvantaged peers. As a result, children from low socioeconomic backgrounds are less likely to enter school with the social and academic skills needed to set them up for success. Jans Deitrichson and colleagues at the Danish National Centre for Social Research recently performed a meta-analysis aimed at determining what components within academic interventions are the most effective at improving the achievement of primary school students from low-socioeconomic backgrounds.

A total of 101 studies performed between 2000–2014 were included in the meta-analysis. Seventy-six percent were randomised controlled trials and the rest were quasi-experimental studies. Studies had to target pupils from low socioeconomic backgrounds, utilise standardised test results in reading and maths as the outcome measures, and take place in OECD or EU countries, although most were in the US. They also had to contain information that allowed the researchers to calculate effect sizes.

The authors sorted each study’s academic intervention into “component categories” (the methods used). Examples include coaching/ mentoring of pupils, cooperative learning, incentives, small-group tutoring, or a combination of these or other methods. Analysis demonstrated that tutoring, feedback and progress monitoring, and cooperative learning were the components with the largest effect sizes. The authors stated that although the average effect sizes for these components were not large enough to close the achievement gap between high- and low-socioeconomic pupils, they certainly reduced it. They suggest that cost-effectiveness studies should be performed on these programmes to give policymakers and educators a fuller picture of programme benefits.

Source: Academic interventions for elementary and middle school students with low socioeconomic status: A systematic review and meta-analysis (January 2017), Review of Educational Research, Vol 87, Issue 2