Evaluation of Success for All

The Education Endowment Foundation recently published a report on the effectiveness trial of the Success for All (SFA) programme to evaluate its impact on the literacy outcomes of Reception pupils. SFA is a whole-school approach to improving literacy in primary schools. All teachers and senior leaders are involved, with the school receiving a total of 16 training and support days. Teachers receive pedagogical training – for example on effective phonics teaching – and teaching materials such as structured lesson plans. For the school leadership team, there is support in areas such as data management, ability grouping and parental engagement.

Fifty-four schools took part in this effectiveness trial, which was evaluated by Sarah Miller and colleagues from Queen’s University Belfast. Although the intervention was delivered on a whole-school basis, the evaluation focused only on the outcomes of 1,767 pupils starting in Reception, and followed them through to the end of Year 1.

The main analysis found that Reception pupils in SFA schools made more progress than pupils in control schools after two years (effect size = +0.07). The effect was slightly larger for pupils eligible for free school meals (FSM), compared to FSM pupils in control schools after two years (effect size = +0.12). In both cases, the effect was smaller than those found in previous evaluations (this is the third RCT of SFA to be conducted, and the first independent trial of the programme in England). Trials in the US reported effect sizes between +0.15 and +0.30. The report suggests that one possible reason for this was that some schools struggled to implement the programme as intended.

The project delivery team was from the University of York. Robert Slavin, director of the US Center for Research and Reform in Education, is Co-founder and Chairman of the Board of the Success for All Foundation.

Source: Success for All: Evaluation report and executive summary (July 2017), Education Endowment Foundation

Evaluation of Challenge the Gap

An evaluation of the Challenge the Gap (CtG) programme for the Education Endowment Foundation found no evidence that the programme increased average achievement for either primary or secondary pupils overall.

Challenge the Gap is a two-year school improvement programme that aims to help schools improve the achievement of their disadvantaged pupils through a professional development programme for staff. The evaluation conducted by The University of Manchester, involved 21,041 pupils from 104 schools (64 primary schools and 39 secondary schools). Around 24% of pupils in the primary schools and 16% in the secondary schools were eligible for free school meals. The evaluation assessed the impact on all participating schools using 2015 Key Stage 2 or Key Stage 4 results. CtG schools were compared to schools with a similar socio-demographic profile.

No evidence was found that CtG increased the average achievement for either primary or secondary school pupils, overall. For children eligible for free school meals (FSM), those in CtG primary schools made two months’ additional progress (average effect size = +0.10) compared to similar children in non-CtG schools. In CtG secondary schools, FSM-eligible pupils made two months’ less progress compared to similar pupils in non-CtG secondary schools (average effect size = -0.10). The smaller number of FSM-eligible pupils in the trial means that these results are less secure than the overall findings.

Source: Challenge the Gap: Evaluation report and executive summary (July 2017), Education Endowment Foundation

Talking in class boosts progress in maths, science and English

An intervention that trained teachers to improve and monitor the quality of classroom talk had a positive impact on primary pupils’ test scores in English, maths and science, a report published by the Education Endowment Foundation (EEF) reveals.

Seventy-six primary schools with higher-than-average proportions of disadvantaged pupils took part in a randomised control trial of the Dialogic Teaching intervention, which is designed to improve the quality of classroom talk as a means of increasing pupils’ engagement, learning and achievement. Year 5 teachers in 38 schools (2,493 pupils), and a teacher mentor from each school, received resources and training from the delivery team and then implemented the intervention over the course of the autumn and spring terms in the 2015/16 school year. A control group of 38 schools (2,466 pupils) continued with business as usual. Following the intervention, pupils were tested in English, maths and science.

The results showed that pupils in the intervention schools did better in the main outcome measures of English (effect size = +0.16), science (+0.12), and maths (+0.09) when compared with pupils in the control schools who didn’t receive the intervention. For pupils who received free school meals, the intervention had a higher impact on maths (+0.16), but around the same for English (+0.12) and science (+0.11). Teachers reported positive effects on pupil engagement and confidence, and on the whole the intervention was highly regarded by participating schools. However, some teachers felt that it would take longer than two terms to fully embed a Dialogic Teaching approach in their classrooms.

The Dialogic Teaching intervention was developed by the Cambridge Primary Review Trust and the University of York. This University of York news story has more.

Source: Dialogic teaching: evaluation report and executive summary (July 2017), Education Endowment Foundation

Scale-up evaluation of Switch-on disappoints

An independent evaluation for the Education Endowment Foundation (EEF) of the Switch-on intervention has found no evidence that it improves the reading outcomes of pupils struggling with literacy at Key Stage 1 (ages 5–7 years) compared to schools’ usual practices.

Switch-on is an intensive, targeted literacy intervention that aims to improve the reading skills of pupils who are struggling with literacy. There are two versions of the intervention: Switch-on Reading and Switch-on Reading and Writing. Both involve specially trained Teaching Assistants (TAs) delivering a tailored programme of literacy support in daily 20-minute sessions over a ten-week period.

Schools selected pupils in Year 3 who were working below age-related expectations at the end of Key Stage 1 and who did not have a high level of special needs. Each of the 184 participating schools was then randomly assigned to receive either Switch-on Reading, Switch-on Reading and Writing, or to continue their usual practices of supporting pupils with reading difficulties. In total, 999 pupils were involved in the trial.

Estimated effect sizes were zero and not statistically significant. The intervention also showed no effect on pupils eligible for free school meals. These findings contradict a previous, smaller EEF-funded evaluation of Switch-on which had shown signs of promise in raising reading outcomes for Year 7 pupils.

Source: Switch-on – effectiveness trial (May 2017), Education Endowment Foundation

Positive progress for maths, but not reading, for a thinking-skills intervention

The Education Endowment Foundation evaluated the impact of the ReflectED programme using a randomised controlled trial involving 1,858 pupils across 30 schools in five areas throughout England over the academic year 2014/15. The evaluation examined the impact on the maths and reading achievement of Year 5 pupils, and also their attitudes toward reading and maths.

The ReflectED programme was developed by Rosendale Primary School to improve pupils’ metacognition — their ability to think about and manage their own learning. This includes the skills of setting and monitoring goals, assessing progress, and identifying personal strengths and challenges.

Year 5 pupils who took part in the trial made an average of four months’ additional progress in maths (ES = +0.30) compared to those in the control groups. The evaluators also found evidence that pupils in the programme developed a more positive attitude toward maths. However, in reading they made two fewer months’ progress than the control group (ES = -0.16) and developed a slightly less positive attitude toward the subject.

The evaluation also found that most schools were already teaching metacognitive and reflective skills similar to those taught in the ReflectED programme, which are likely to have continued in the control group classes. This might have limited the impact that ReflectED had on teachers’ practice and pupils’ outcomes.

Source: ReflectED: Evaluation report and executive summary (2016), Education Endowment Foundation

Breakfast clubs boost reading and mathematics results for primary school pupils

Breakfast clubs that offer pupils in primary schools a free and nutritious meal before school can boost their reading, writing and maths results, according to the results of a randomised controlled trial published by the Education Endowment Foundation.

Over the course of an academic year parents of around 8,600 pupils from 106 primary schools in England with higher than average numbers of disadvantaged pupils were encouraged to send their child to free breakfast clubs. The independent evaluation by researchers at the Institute for Fiscal Studies and the National Children’s Bureau found that for Year 2 children the provision of a breakfast club led to a significant improvement in the main outcome measures of mathematics (Effect Size = +0.15) and reading (+0.10) when compared with schools running “business as usual”. For Year 6 children, the impact on assessments were positive but slightly smaller in reading (+0.10) and mathematics (+0.08). Surprisingly, there were larger improvements for pupils not eligible for free school meals than for those eligible.

The evaluators also reported that pupils’ behaviour and concentration improved. Attendance at school also improved for pupils in breakfast club schools, resulting in about 26 fewer half-days of absence per year for a class of 30. The findings suggest that it is not just eating breakfast that delivers improvements, but attending a breakfast club. This could be due to the content of the breakfast itself, or to other social or educational benefits of the club.

Source: Magic Breakfast: Evaluating school breakfast provision. Evaluation report and executive summary (2016), Education Endowment Foundation