Evaluation of Success for All

The Education Endowment Foundation recently published a report on the effectiveness trial of the Success for All (SFA) programme to evaluate its impact on the literacy outcomes of Reception pupils. SFA is a whole-school approach to improving literacy in primary schools. All teachers and senior leaders are involved, with the school receiving a total of 16 training and support days. Teachers receive pedagogical training – for example on effective phonics teaching – and teaching materials such as structured lesson plans. For the school leadership team, there is support in areas such as data management, ability grouping and parental engagement.

Fifty-four schools took part in this effectiveness trial, which was evaluated by Sarah Miller and colleagues from Queen’s University Belfast. Although the intervention was delivered on a whole-school basis, the evaluation focused only on the outcomes of 1,767 pupils starting in Reception, and followed them through to the end of Year 1.

The main analysis found that Reception pupils in SFA schools made more progress than pupils in control schools after two years (effect size = +0.07). The effect was slightly larger for pupils eligible for free school meals (FSM), compared to FSM pupils in control schools after two years (effect size = +0.12). In both cases, the effect was smaller than those found in previous evaluations (this is the third RCT of SFA to be conducted, and the first independent trial of the programme in England). Trials in the US reported effect sizes between +0.15 and +0.30. The report suggests that one possible reason for this was that some schools struggled to implement the programme as intended.

The project delivery team was from the University of York. Robert Slavin, director of the US Center for Research and Reform in Education, is Co-founder and Chairman of the Board of the Success for All Foundation.

Source: Success for All: Evaluation report and executive summary (July 2017), Education Endowment Foundation

Successful results for Success for All

In 2010, the Success for All Foundation (SFAF) was awarded a $50 million Investing in Innovation (i3) scale-up grant from the US Department of Education, helping to expand its comprehensive school improvement programme. As part of the grant, MDRC carried out an independent evaluation of SFAF’s scale-up initiative. MDRC’s third and final report from the evaluation examines the impact of the Success for All (SFA) reading programme over three years, its incremental cost, and the scale-up process itself.

A total of 37 schools were involved in the study, with 19 randomly chosen to adopt SFA in all year groups, and 18 control schools, which continued to use their existing reading programmes. Key findings included:
  • SFA is an effective vehicle for teaching phonics. In the average SFA school, the programme registered a notable, statistically significant impact on a measure of phonics skills for second-graders (age 7/8) who had been in SFA for all three years, compared with their control group counterparts. Pupils in the average SFA school performed better than the average control group school on tests of reading fluency and comprehension, but not significantly.
  • For a subgroup of special concern to policy makers and practitioners  pupils entering school with low pre-literacy skills  SFA appears to be especially effective. Second-graders (Year 3) in the average SFA school who had started kindergarten (Year 1) in the bottom half of the sample in terms of their knowledge of the alphabet and their ability to sound out words registered significantly higher scores on measures of phonics skills, word recognition, and reading fluency than similar pupils in control group schools. The impact on comprehension for this group was also positive but not statistically significant.
In conclusion, the authors say, “The scale-up findings show that, for a modest investment, SFA reliably improves the decoding skills of students in kindergarten through second grade, and that it is especially beneficial for students who begin in the lower half in these skills.”
Source: Scaling Up the Success for All Model of School Reform: Final Report from the Investing in Innovation (i3) Evaluation (2015), MDRC.

Positive results for Success for All in national evaluations

Success for All (SFA) – a primary literacy approach – was selected to receive a five-year scale-up grant under the US Department of Education’s Investing in Innovation (i3) competition. This report, the second from MDRC’s independent evaluation, discusses the programme’s implementation and impacts in 2012-2013.

The evaluation uses a cluster random assignment design involving 37 schools for children age 5-12 and located in five school districts; 19 schools were randomly selected to receive the SFA programme, while the remaining 18 control group schools did not receive the intervention. Findings showed that first grade (Year 2) pupils who had participated in the SFA programme since kindergarten (Year 1) significantly outperformed children in the control group on two measures of phonics and decoding skills. Outcomes were similar for different categories of children, including African-American, Hispanic, and White children.

In addition, a new article reports the third-year findings of a longitudinal evaluation of SFA in England. The results reveal a statistically significant positive school-level effect for SFA schools compared with control schools on standardised reading measures of word-level and decoding skills. There were also directionally positive but non-significant school-level effects on measures of comprehension and fluency. A total of 18 SFA schools and 18 control schools across England, matched on prior achievement and demographics, were included in the quasi-experimental study.

Sources: The Success for All Model of School Reform: Interim Findings from the Investing in Innovation (i3) Scale-Up (2014), MDRC. Success for All in England: Results From the Third Year of a National Evaluation (2014), SAGE Open 2014 (4).

Encouraging findings for Success for All

A recently published report presents the first-year findings of a three-year longitudinal evaluation of Success for All (SFA), a whole-school literacy approach. As part of a US i3 scale-up grant, third-party evaluator MDRC randomly assigned 37 elementary schools in underprivileged areas across the US to SFA (n=19) or control (n=18) conditions. The study will follow children between the ages of 5 and 8.

First-year findings indicated significantly positive effects of SFA on the Woodcock Word Attack (phonics) scale, but no differences on Woodcock Letter-Word Identification. These findings are in line with prior longitudinal studies, which have found positive effects on Word Attack in the earliest school years, followed by Letter-Word Identification, and then Passage Comprehension by Year 3 and beyond.

Achievement effects were similar for all types of pupils, but not all subgroups had significant differences when they were analysed separately.

Later reports will focus on reading gains over time and on observation and treatment fidelity rating data.

Source: The Success for All Model of School Reform: Early Findings from the Investing in Innovation (i3) Scale-Up (2013), MDRC.