New guidance to support schools to put evidence to work in their classrooms

A new guidance report from the Education Endowment Foundation (EEF) aims to give schools the support they need to put evidence to work in their classrooms and implement new programmes and approaches effectively.

The report highlights how good and thoughtful implementation is crucial to the success of any teaching and learning strategy, yet creating the right conditions for implementation – let alone the structured process of planning, delivering and sustaining change – is hard.

The authors offer six recommendations to help schools give their innovations the very best chance by working carefully through the who, why, where, when and how of managing change. These recommendations can be applied to any school improvement decision: programmes or practices, whole-school or targeted approach, internally or externally generated ideas.

The report frames implementation in four stages: explore, prepare, deliver and sustain. It also provides guidance on how schools can create the right environment for change, from supporting staff to getting leadership on board.

Source: Putting evidence to work: a school’s guide to implementation. EEF Guidance Report (February 2018), Education Endowment Foundation

Keep up the good work

In his Huffington Post blog, Robert Slavin, director of the Center for Research and Reform in Education, discusses a study that evaluated a behaviour management programme, First Step to Success, for students with behaviour problems. The programme has been evaluated successfully many times. In this latest study, 200 children in grades 1 to 3 (Years 2 to 4) with serious behaviour problems were randomly assigned to experimental or control groups. On behaviour and achievement measures, students in the experimental group scored much higher, with effect sizes of +0.44 to +0.87.

The researchers came back a year later to see if the outcomes had been maintained. Despite the substantial impacts seen previously, none of three prosocial/adaptive behaviour measures, only one of three problem/maladaptive behaviours, and none of four academic achievement measures now showed positive outcomes. However, the students had passed from teachers who had been trained in the First Step method to teachers who had not.

Dr Slavin says, “Imagine that all teachers in the school learned the program and all continued to implement it for many years. In this circumstance, it would be highly likely that the first-year positive impacts would be sustained and most likely improved over time.” He discusses the implications of the research, and the importance of continuing with successful interventions.

Source: Keep Up the Good Work (To Keep Up the Good Outcomes) (2016), Huffington Post

School ethos important for social-emotional learning

A new article published in the British Educational Research Journal describes a study of Social and Emotional Aspects of Learning (SEAL), a multi-component social-emotional learning programme that is widely used in schools in the UK. Estimates in 2010 suggest that 90% of primary schools and 70% of secondary schools had engaged with SEAL resources, at least to some extent.

For the study, a team of school advisors used a semi-structured observation and interview protocol to rate various aspects of the implementation of SEAL in 49 primary and secondary schools. A total of 2,242 pupils in 29 of these schools completed measures of social experiences and school ethos. School-level achievement and attendance statistics were also collated for all participating schools.
The authors raise the issue of implementation as the programme is built on the premise that each school or setting should find its own way into, and use for, SEAL materials. They note that perceived tensions could be identified between SEAL and other initiatives that may have made competing demands on staff time, effort, and resources. They say these factors may explain the mixed results of previous evaluations.

This evaluation found that the ratings indicative of a whole-school universal approach to SEAL were significantly correlated with school ethos, which in turn mediated associations with pupils’ social experiences, overall achievement, and persistent absence.

Source: Working with ‘Social and Emotional Aspects of Learning’ (SEAL): Associations with School Ethos, Pupil Social Experiences, Attendance, and Attainment (2014), British Educational Research Journal, 40(4).

Do evidence-based programmes work in every context?

Researchers from The Social Research Unit at Dartington, along with the Universities of Exeter and York, have analysed the impact of three evidence-based programmes implemented in Birmingham as part of the city’s “Brighter Futures” strategy. A new article describing their findings illustrates that context matters.

The three programmes evaluated were the Incredible Years BASIC parenting programme, PATHS (Promoting Alternative Thinking Strategies), and the Triple-P parenting programme. In each case, a randomised controlled trial was conducted with validated standardised measures. The findings were as follows:

  • Incredible Years yielded reductions in negative parenting behaviours among parents, reductions in child behaviour problems, and improvements in children’s relationships.
  • In the PATHS trial, modest improvements in emotional health and behavioural development after one year disappeared by the end of year two.
  • There were no effects for Triple-P.

The authors suggest that much can be learned from the strengths and limitations of the Birmingham experience.

Source: The impact of three evidence-based programmes delivered in public systems in Birmingham, UK (2012), International Journal of Conflict and Violence, 6(2).

Applying evidence to social programmes

Jon Baron, president of the US Coalition for Evidence-Based Policy, has written a blog post for The New York Times on using “what works”. He says, “Scientifically rigorous studies – particularly, the ‘gold standard’ of randomised controlled trials – are a mainstay of medicine, providing conclusive evidence of effectiveness for most major medical advances in recent history.

In social spending, by contrast, such studies have only a toehold. Where they have been used, however, they have demonstrated the same ability to produce important, credible evidence about what works – and illuminated a path to major progress.”

Source: Applying Evidence to Social Programs (2012), Economix

Implementation matters

An article in the latest edition of the Oxford Review of Education looks at the importance of studying the implementation of interventions in school settings. Research studies across multiple disciplines, including education, have consistently demonstrated that interventions are rarely implemented as designed and, crucially, that variability in implementation is related to variability in the achievement of expected outcomes.

In this article, the authors call for an increasing emphasis on the “often neglected” study of implementation itself, particularly in school settings. They discuss the importance of implementation throughout all the stages of a programme’s development, from initial testing to widespread dissemination. They argue that, at each stage, studying the way a programme is implemented can provide a range of useful information. This includes learning about the likely effectiveness of the programme in real-world settings, and learning general lessons about how best to implement change in education.

Source: The importance of studying the implementation of interventions in school settings (2012), Oxford Review of Education,38(5)