The evidence on achievement gaps over time: Contained but not closing

Research has shown that socioeconomic status (SES) is the highest predictor of children’s academic achievement. Moreover, the achievement gap between low- and high-SES pupils begins early in their schooling. How effective have initiatives been at narrowing the achievement gap? Emma Garcia at the Education Policy Institute in the US and Elaine Weiss at the Broader Bolder Approach to Education examined two cohorts of kindergartners (Year 1), those who started in 1998 and those who started in 2010. They were looking at the relationship between socio-economic status and kindergartners’ cognitive and non-cognitive skills at the start of their school years to see if the achievement gap had narrowed in this twelve-year span.

Using data from the National Center for Education Statistics – Early Childhood Longitudinal Studies of the Kindergarten Classes of 1998-99 and 2010-11, Garcia and Weiss found that the achievement gap did not change between 1998 to 2010 among pupils living in the US’s highest and lowest economic strata, a difference of 1.17 standard deviations in reading and 1.25 standard deviations in maths, despite parents’ increased involvement in educating their children across all SES groups and the implementation of programmes designed to narrow these gaps. Interestingly, they did find that the percentage of children living in poverty grew during that time, yet the achievement gap did not grow, nor did it narrow. They found that greater parental involvement and children’s pre-school attendance contained the gap, but did not do enough to eliminate the overall effects of poverty on pupil achievement.

The researchers then reviewed twelve programmes designed to narrow the achievement gap. The most effective programmes addressed not only academics, but ensured the children were getting proper meals and healthcare and provided other supports for children and their families.

Source: Education inequalities at the school starting gate: Gaps, trends, and strategies to address them (September 2017), Education Policy Institute

What difference does it make?

We regularly quote effect sizes in Best Evidence in Brief as a measure of the impact of an intervention or approach. But what is the impact of a normal school year on children, and how much of that impact is due to the school? A study by Hans Lutyen and colleagues, published in School Effectiveness and School Improvement, attempts to find out.

The study analysed 3,500 pupils from 20 mostly independent (private) English primary schools on four different learning outcomes. These measures, part of the Interactive Computer Adaptive System (InCAS), were reading, general maths, mental arithmetic and developed ability, the last of which measures items such as vocabulary and non-verbal pattern recognition.

Children were measured on these outcomes from Years 1 to 6. Using a regression-discontinuity approach that exploited the discontinuity between the youngest pupils in one year and the oldest pupils in the year below, the researchers were able to identify the overall progress of the children, and the extent to which this was a result of the impact of the school.

The results showed a declining impact of a school year as children got older. The effect size of Year 1 ranged from +1.18 for mental arithmetic to +0.8 for general maths. By Year 6, effect sizes varied from +0.88 for general maths to +0.49 for reading and developed ability.

The effect of schooling itself accounted for an average of between 23.5% and 43.4% of this impact across the four measures. Put another way, the effect size of schooling in Year 1 ranged from +0.55 for reading to +0.31 for developed ability. By Year 6, effect sizes had fallen to between +0.27 for general maths and +0.08 for reading and developed ability.

The researchers suggest that, when setting benchmarks for educational interventions, it is not only important to consider the phase of the educational career, but also the specific measure.

Source: The contribution of schooling to learning gains of pupils in Years 1 to 6 (February 2017), School Effectiveness and School Improvement

Children with reading difficulties may have undiagnosed hearing problems

A report published by the Nuffield Foundation, which compared a group of children with dyslexia to children with a history of repeated ear infections (otitis media with effusion, OME) to see if there were any similarities in their phonological and literacy difficulties, has found that their difficulties may be due in part to undiagnosed hearing problems.

Julia Carroll and Helen Breadmore compared a group of 36 children with dyslexia to 29 children with OME, and also to control groups of typically developing children of the same age and groups of younger children at the same reading level. This made a total sample size of 195 children, ages 8 to 10, from 20 schools in the UK. All of the children completed a series of tests to establish their reading and writing skills and also their phonological skills (ability to manipulate speech sounds) and morphological skills (knowledge of grammatical word structure). Eighteen months later, the children’s reading, spelling and phonological awareness was re-tested, and a hearing screening conducted.

The results showed that the children with dyslexia had different patterns of literacy difficulties than children with OME, although there were some overlaps. The children with dyslexia showed difficulties with both phonological and morphological skills, whereas children with OME had difficulties only on phonological tasks. Both dyslexic children and children with OME had lower levels of reading than the age-matched control children.

The results from the hearing screening eighteen months later found that 9 of the 36 children with dyslexia had mild or moderate hearing impairments, of which their parents and teachers were unaware. The researchers suggest, therefore, that children with reading difficulties should be screened for hearing problems so that they are able to receive more structured support that could help them improve their literacy skills.

Source: Morphological processing in children with phonological difficulties (October 2017), Coventry University and University of Warwick Briefing Paper for The Nuffield Foundation

Open Court Reading receives judgement in multi-year trial

A multi-year scale-up study has examined the effectiveness of Open Court Reading (OCR), a phonics-based curriculum for grades K-6 (Reception to Year 7).

The study, by Michael Vaden-Kiernar and colleagues, and published in the Journal of Research on Educational Effectiveness, was a school-level, cluster randomised trial, involving around 4,500 pupils and 1,000 teachers in 49 elementary schools across the US.

The OCR curriculum includes pupil materials, teacher manuals, diagnostic and assessment tools and test preparation practice guides. In all grades (K-5), the instructional format is a three-part lesson with specific instruction in phonemic awareness and phonics, vocabulary and comprehension skills and writing skills. The programme includes one- or two-day summer workshops at the start of each school year to train teachers on programme implementation, and ongoing support by OCR reading consultants.

An implementation study showed adequate to high levels of fidelity to the programme. However, there were no statistically significant effects on reading performance in Year 1, and a small negative effect (effect size = -0.09) in Year 2. Relative to the “business-as-usual” controls, no positive overall impacts of OCR and mixed impacts for pupil subgroups were found.

Source: Findings from a multiyear scale-Up effectiveness trial of Open Court Reading (June 2017), Journal of Research on Educational Effectiveness

Evaluation of Success for All

The Education Endowment Foundation recently published a report on the effectiveness trial of the Success for All (SFA) programme to evaluate its impact on the literacy outcomes of Reception pupils. SFA is a whole-school approach to improving literacy in primary schools. All teachers and senior leaders are involved, with the school receiving a total of 16 training and support days. Teachers receive pedagogical training – for example on effective phonics teaching – and teaching materials such as structured lesson plans. For the school leadership team, there is support in areas such as data management, ability grouping and parental engagement.

Fifty-four schools took part in this effectiveness trial, which was evaluated by Sarah Miller and colleagues from Queen’s University Belfast. Although the intervention was delivered on a whole-school basis, the evaluation focused only on the outcomes of 1,767 pupils starting in Reception, and followed them through to the end of Year 1.

The main analysis found that Reception pupils in SFA schools made more progress than pupils in control schools after two years (effect size = +0.07). The effect was slightly larger for pupils eligible for free school meals (FSM), compared to FSM pupils in control schools after two years (effect size = +0.12). In both cases, the effect was smaller than those found in previous evaluations (this is the third RCT of SFA to be conducted, and the first independent trial of the programme in England). Trials in the US reported effect sizes between +0.15 and +0.30. The report suggests that one possible reason for this was that some schools struggled to implement the programme as intended.

The project delivery team was from the University of York. Robert Slavin, director of the US Center for Research and Reform in Education, is Co-founder and Chairman of the Board of the Success for All Foundation.

Source: Success for All: Evaluation report and executive summary (July 2017), Education Endowment Foundation

Talking in class boosts progress in maths, science and English

An intervention that trained teachers to improve and monitor the quality of classroom talk had a positive impact on primary pupils’ test scores in English, maths and science, a report published by the Education Endowment Foundation (EEF) reveals.

Seventy-six primary schools with higher-than-average proportions of disadvantaged pupils took part in a randomised control trial of the Dialogic Teaching intervention, which is designed to improve the quality of classroom talk as a means of increasing pupils’ engagement, learning and achievement. Year 5 teachers in 38 schools (2,493 pupils), and a teacher mentor from each school, received resources and training from the delivery team and then implemented the intervention over the course of the autumn and spring terms in the 2015/16 school year. A control group of 38 schools (2,466 pupils) continued with business as usual. Following the intervention, pupils were tested in English, maths and science.

The results showed that pupils in the intervention schools did better in the main outcome measures of English (effect size = +0.16), science (+0.12), and maths (+0.09) when compared with pupils in the control schools who didn’t receive the intervention. For pupils who received free school meals, the intervention had a higher impact on maths (+0.16), but around the same for English (+0.12) and science (+0.11). Teachers reported positive effects on pupil engagement and confidence, and on the whole the intervention was highly regarded by participating schools. However, some teachers felt that it would take longer than two terms to fully embed a Dialogic Teaching approach in their classrooms.

The Dialogic Teaching intervention was developed by the Cambridge Primary Review Trust and the University of York. This University of York news story has more.

Source: Dialogic teaching: evaluation report and executive summary (July 2017), Education Endowment Foundation