Early impact of Diplomas Now

MDRC has published the first results from a randomised controlled trial of Diplomas Now, a whole-school reform initiative. Under the Diplomas Now programme:

  • Schools are reorganised so that small groups of teachers work consistently with the same population of students.
  • There is an intensive peer coaching system for maths and English teachers.
  • Early warning indicators are used to identify students who need different types of support.
  • Additional staff help coordinate the transformation, introduce new practices and structures, provide training and support to school staff members, provide additional services to students, and engage with families and community organisations.

In total, 62 schools (33 middle schools and 29 high schools) from 11 large urban districts were recruited. Thirty-two of the participating schools were randomly assigned to implement the Diplomas Now model (DN schools), and 30 were assigned to continue with “business as usual” (non-DN schools).

So far, the study team has been able to explore early impacts for sixth- and ninth-grade (Year 7 and 10) students moving into DN schools during the first two years of the programme. For this cohort of students, DN schools were more successful than non-DN schools in reducing the number of early warning indicators (a statistically significant 3.6 percentage point reduction). The early warning indicator was a combination of daily attendance of 85% or less, suspensions or expulsions for a total of three or more days, and failing grades in English or maths classes. However, the DN programme made no statistically significant impact on any of these measures separately. The project will continue for several more years.

Source: Addressing Early Warning Indicators: Interim Impact Findings from the Investing in Innovation (i3) Evaluation of Diplomas Now (2016), MDRC.

A whole-school approach to tackling bullying

An article in the July issue of the Scandinavian Journal of Educational Research describes the impact of an evidence-based framework designed to help schools tackle bullying. Based on research on both bullying and educational effectiveness, the framework is a whole-school approach that considers three elements:

  1. The school policy for teaching;
  2. The school learning environment, including behaviour outside the classroom, interaction between teachers, and collaboration with stakeholders including parents and psychologists; and
  3. School evaluation.

A total of 52 schools in Cyprus and Greece were randomly allocated to experimental and control groups. Experimental schools were offered training and support to develop strategies and action plans for confronting and reducing bullying based on the data on what was occurring in their schools. The intervention was implemented for approximately eight months.

The authors found that the approach had a direct effect on improving school factors and both direct and indirect effects on reducing bullying. In both countries, schools that used the approach reduced the extent to which their pupils were being victimised and reduced the extent of bullying compared to the control schools. However, the article acknowledges that the effect of the intervention may partly be attributed to differences in the effort put in by schools in the two groups with regard to implementing their strategies to reduce bullying.

Source: Improving the School Learning Environment to Reduce Bullying: An Experimental Study (2014), Scandinavian Journal of Educational Research , 58(4).

Positive results for Success for All in national evaluations

Success for All (SFA) – a primary literacy approach – was selected to receive a five-year scale-up grant under the US Department of Education’s Investing in Innovation (i3) competition. This report, the second from MDRC’s independent evaluation, discusses the programme’s implementation and impacts in 2012-2013.

The evaluation uses a cluster random assignment design involving 37 schools for children age 5-12 and located in five school districts; 19 schools were randomly selected to receive the SFA programme, while the remaining 18 control group schools did not receive the intervention. Findings showed that first grade (Year 2) pupils who had participated in the SFA programme since kindergarten (Year 1) significantly outperformed children in the control group on two measures of phonics and decoding skills. Outcomes were similar for different categories of children, including African-American, Hispanic, and White children.

In addition, a new article reports the third-year findings of a longitudinal evaluation of SFA in England. The results reveal a statistically significant positive school-level effect for SFA schools compared with control schools on standardised reading measures of word-level and decoding skills. There were also directionally positive but non-significant school-level effects on measures of comprehension and fluency. A total of 18 SFA schools and 18 control schools across England, matched on prior achievement and demographics, were included in the quasi-experimental study.

Sources: The Success for All Model of School Reform: Interim Findings from the Investing in Innovation (i3) Scale-Up (2014), MDRC. Success for All in England: Results From the Third Year of a National Evaluation (2014), SAGE Open 2014 (4).

Tailoring evidence-based reform to different problems

Robert Slavin, professor at the Institute for Effective Education and director of the Center for Research and Reform in Education, focuses his latest blog post not just on reading, writing, and arithmetic, but on the sometimes harder-to-define problems of education, such as managing resources and cultivating relationships.

He says, “Is there anyone out there who thinks that it is not important to identify effective and replicable approaches to teaching reading, algebra, and all the other relatively easy-to-define, easy-to-measure problems of education? Yet solving these does still leave some very important but less-well-defined problems that may take different approaches. These approaches should still be informed by evidence, but perhaps different types of evidence from the design-build-evaluate-disseminate model that usually leads to proven and replicable approaches to reading or algebra, if anything does.”

Schools need more than just autonomy to improve standards

A new report from Australia’s Grattan Institute uses data from two international surveys conducted by the OECD – the Programme for International School Assessment (PISA) and the Teaching and Learning International Survey – to explore a number of issues around standards, including whether giving schools more autonomy can improve achievement.

The report concludes that the link between high autonomy and high performance is weak. Instead, the world’s best-performing school systems articulate the best ways to teach and learn, then implement reform through high-quality systems of teacher development, appraisal, and feedback, among other policies. Autonomous schools in Australia and other countries, they say, are no better at implementing these programmes than are centralised schools.

Behaviour programme shows promise

Researchers from the University of Western Sydney in Australia have published new research detailing a study of Positive Behavior for Learning (PBL), a schoolwide systemic approach to promoting both positive behaviours and student learning. PBL aims to establish strong systems that involve all staff and pupils, and to implement evidence-based practices that support behaviour and learning.

The participants in this study were 2,129 pupils from 18 schools in Australia. A total of 827 boys and 888 girls from four primary and eight secondary schools implementing PBL (the experimental group) were compared with 188 boys and 226 girls from two primary and four secondary schools (the control group).

Using a structural equation modelling technique to test group differences, the authors found that the pupils in schools that had implemented PBL for over nine months had higher scores in both self-reported behavioural and adaptive motivational factors. Specifically, they found higher scores in:

  • Behavioural management input (the pupils’ perceptions of behaviour interventions);
  • Positive behaviours (following school rules);
  • Knowledge about behaviours (schools’ expectations);
  • Effort goal orientation (motivation); and
  • The value of schooling.

They conclude that PBL may benefit all pupils, but more work may be needed for boys.

Source: Seeshing Yeung A, Barker K, Tracey D, and Mooney M, School-wide Positive Behavior for Learning: Effects of Dual Focus on Boys’ and Girls’ Behavior and Motivation for Learning, International Journal of Educational Research 62, 1–10.