Positive progress for maths, but not reading, for a thinking-skills intervention

The Education Endowment Foundation evaluated the impact of the ReflectED programme using a randomised controlled trial involving 1,858 pupils across 30 schools in five areas throughout England over the academic year 2014/15. The evaluation examined the impact on the maths and reading achievement of Year 5 pupils, and also their attitudes toward reading and maths.

The ReflectED programme was developed by Rosendale Primary School to improve pupils’ metacognition — their ability to think about and manage their own learning. This includes the skills of setting and monitoring goals, assessing progress, and identifying personal strengths and challenges.

Year 5 pupils who took part in the trial made an average of four months’ additional progress in maths (ES = +0.30) compared to those in the control groups. The evaluators also found evidence that pupils in the programme developed a more positive attitude toward maths. However, in reading they made two fewer months’ progress than the control group (ES = -0.16) and developed a slightly less positive attitude toward the subject.

The evaluation also found that most schools were already teaching metacognitive and reflective skills similar to those taught in the ReflectED programme, which are likely to have continued in the control group classes. This might have limited the impact that ReflectED had on teachers’ practice and pupils’ outcomes.

Source: ReflectED: Evaluation report and executive summary (2016), Education Endowment Foundation

Breakfast clubs boost reading and mathematics results for primary school pupils

Breakfast clubs that offer pupils in primary schools a free and nutritious meal before school can boost their reading, writing and maths results, according to the results of a randomised controlled trial published by the Education Endowment Foundation.

Over the course of an academic year parents of around 8,600 pupils from 106 primary schools in England with higher than average numbers of disadvantaged pupils were encouraged to send their child to free breakfast clubs. The independent evaluation by researchers at the Institute for Fiscal Studies and the National Children’s Bureau found that for Year 2 children the provision of a breakfast club led to a significant improvement in the main outcome measures of mathematics (Effect Size = +0.15) and reading (+0.10) when compared with schools running “business as usual”. For Year 6 children, the impact on assessments were positive but slightly smaller in reading (+0.10) and mathematics (+0.08). Surprisingly, there were larger improvements for pupils not eligible for free school meals than for those eligible.

The evaluators also reported that pupils’ behaviour and concentration improved. Attendance at school also improved for pupils in breakfast club schools, resulting in about 26 fewer half-days of absence per year for a class of 30. The findings suggest that it is not just eating breakfast that delivers improvements, but attending a breakfast club. This could be due to the content of the breakfast itself, or to other social or educational benefits of the club.

Source: Magic Breakfast: Evaluating school breakfast provision. Evaluation report and executive summary (2016), Education Endowment Foundation

Is the pen mightier than the computer?

The Education Endowment Foundation (EEF) has published a report assessing the impact of Abracadabra (ABRA), a 20-week online literacy programme, on literacy outcomes for Year 1 pupils. ABRA is composed of phonic fluency and comprehension activities based around a series of age-appropriate texts and is designed to be delivered by a teaching assistant to groups of three to five pupils in four 15-minute sessions per week. The EEF evaluation tested the ABRA online intervention alongside a paper-based alternative using the same material.

Fifty-one schools were randomly assigned to receive either a version of the intervention or to act as a control school delivering business as usual. In the schools receiving the intervention, pupils were randomised to receive the online intervention (ABRA), the paper-based intervention, or standard literacy provision.

Positive effects were found for both the online and paper-based interventions. Pupils in the online treatment group (effect size = +0.14) and the paper-based treatment group (ES = +0.23) both showed an improvement in literacy outcomes. The impact was higher for children eligible for free school meals for both ABRA (+0.37) and the paper-based intervention (+0.40). Pupils with below average pre-test outcomes seemed to benefit from ABRA, whereas the paper-based intervention seemed to benefit all pupils. Pupils who received normal literacy provision in the schools where the interventions took place did better than pupils who received normal literacy provision in control schools.

Source: ABRA: Online reading support: Evaluation report and executive summary (2016), Education Endowment Foundation

Saturday school doesn’t orchestrate success

A recent study from the Education Endowment Foundation found disappointing results for a Saturday school designed to improve the reading and maths attainment of underachieving and disadvantaged pupils in Key Stage 2.

Developed by the SHINE Trust and Hallé Orchestra, the intervention provided additional school-based literacy and numeracy lessons, based on musical themes, as well as visits to Hallé rehearsals, performances and other theme-based activities. Twenty-five Saturday sessions, each lasting five hours, were planned for the intervention over the course of an academic year, delivered by qualified teachers, teaching assistants, peer mentors, and professional musicians.

The evaluation, by Victoria White and colleagues from Durham University, consisted of two randomised controlled trials (RCTs)—a pilot trial and a main trial—and a process evaluation. The pilot trial involved 361 Year 5 and 6 pupils in 18 schools; the main trial involved 2,306 Year 4, 5 and 6 pupils in 38 schools.

There was no evidence that the programme had an impact on the reading or mathematics attainment, or attitudes to reading, maths, music, and school, of the children in the trial.

Attendance of eligible pupils was often low and considered as a barrier to successful implementation. Reasons for low attendance included pupils’ lack of availability to attend the Saturday sessions, variable parental engagement with the programme, and limited time at the beginning of the programme for schools to engage children and parents.

The process evaluation revealed a positive picture of involvement and engagement for those pupils who attended the Saturday school activities. Evaluators observed good working relationships between the teachers and pupils, and positive and purposeful learning environments in lessons. All stakeholders felt pupils were making noticeable improvements in behaviour, confidence, and development of social skills.

Source: SHINE on Manchester (2016), Education Endowment Foundation

The value of careers education

The Education Endowment Foundation in the UK has published a review of the international research into careers education, defined as careers-focused school- or college-mediated provision designed to improve students’ education, employment, and/or social outcomes.

Deirdre Hughes and colleagues from the Warwick Institute for Employment Research found that research in the field is weak and fragmented, due mainly to the complexity of differing aspects of careers education being identified and reported in differing ways. Overall, there are significant shortages in quasi-experimental and experimental studies in the career development field.
However, longitudinal studies suggest that the way in which teenagers think about their futures in education and employment has a significant impact on their future as working adults. Teenagers who have underestimated the education required for their desired profession, for example, are statistically more likely to end up not in education, employment, or training. Young people from poorer backgrounds are more likely to have career aspirations that are misaligned with their educational ambitions.

Teenage experience of work—in particular part-time employment—has been associated with improved economic outcomes for young adults. Overwhelmingly, studies identify positive economic outcomes for adults who worked part-time as teenagers while in full-time education. There is evidence of a negative impact on immediate achievement outcomes, although impacts are modest when hours worked are low.

Of the 73 studies included in the review most (46) were carried out in the US, with a smaller number (18) from the UK.

Source: Careers education: International literature review (2016), Education Endowment Foundation

Volunteering improves self-confidence and teamwork, but not achievement

A new study from the Education Endowment Foundation in the UK looks at the impact of a youth social-action project. Delivered by the Youth United Foundation, it involved the creation of new units of uniformed youth organisations (such as The Scout Association, Sea Cadets, or St. John Ambulance) in schools in the North East of England. The groups delivered sessions throughout the year, usually weekly, delivered by trained staff from the youth organisations, sometimes involving adult volunteers, including teachers.

Seventy-one secondary schools were randomly assigned to receive the intervention or not. An initial survey of 3,377 Year 9 students (8th grade in the U.S.) found nearly half wanted to take part in the activities offered, and 663 took part in uniformed group activities during the 2014/15 academic year.

There was no evidence that the intervention had any benefit on children’s academic performance. However, participation in the intervention saw a small improvement in self-reported non-achievement outcomes including self-confidence (effect size = +0.10) and teamwork (+0.04). For children eligible for free school meals, there was no evidence of impact on any outcome.

A process evaluation revealed that students, teachers, and parents thought highly of the intervention. It also highlighted a number of factors that prevented the intervention from being delivered as planned, including a lack of space, time, adult volunteers, and support from senior leadership.

Source: Youth Social Action: Secondary Trial (2016), Education Endowment Foundation