Planning ahead for summer

Heather L Schwartz and colleagues from the RAND Corporation have released a final report on a six-year study of the National Summer Learning Project, an initiative from The Wallace Foundation that was implemented in 2011 in five urban school districts in the US. The summer programmes in these districts were district-led, voluntary summer learning programmes that featured both academic teaching and enrichment opportunities to improve outcomes for low-income pupils.

The overall study combined a randomised controlled trial with correlational analysis and implementation research to examine whether voluntary, district-run summer learning programmes can improve academic, behavioural, and social and emotional outcomes for low-income, urban children in both the short and long terms. The study followed approximately 5,600 pupils from third to seventh grade (Years 4 to 8). Data included surveys, observations and test data.

Findings showed that pupils who received a minimum of 25 hours of mathematics teaching in a summer performed better on the subsequent state maths test, and those receiving 34 hours of English lessons performed better on the subsequent state English language assessment.

These outcomes need to be viewed with caution, however, as pupils who actually attended summer school, as opposed to those who signed up but did not attend, are likely to be more highly motivated and better achieving, introducing possible bias.

Based on their research, the authors offer several recommendations for planning for summer learning, including:

  • Commit in the autumn to a summer programme, and start active planning by January with a programme director who has at least half of his or her time devoted to the job.
  • Prior to the start of the summer programme, professional development for summer teachers should include specific guidance on use of the summer curricula, minimising loss of teaching time, and on checking for pupil understanding.
  • Operate the programme for five to six weeks with three to four hours of academic lessons per day.

A more detailed and comprehensive list of recommendations can be found in the report.

Source: Getting to work on summer learning. Recommended practices for success, 2nd edition (2018), RAND Corporation

The impact of web-based coaching on mathematics instruction

A new working paper, published by Brown University, reports on an online coaching programme that aimed to support maths teachers.

Mathematical Quality of Instruction (MQI) is an observational instrument that helps structure teachers’ and coaches’ reflections about maths teaching. There are four “dimensions” of the instrument – richness of the mathematics, Common Core-aligned pupil practices, working with pupils and mathematics, and teacher errors. In several studies, teachers’ scores on MQI have predicted pupils’ academic achievement gains.

In MQI coaching, teachers selected an element of their practice to work on and filmed one of their lessons. A coach then selected a couple of extracts and chose a comparison stock film clip. The teacher watched the clips and then the two discussed them and developed a plan for improvement.

In the current trial, 142 upper elementary and middle school teachers (Years 4-9) from 51 schools in a mid-western state in the US were assigned to MQI coaching or a control condition. Teachers assigned to MQI coaching took part in a two-day summer school, followed by a bi-weekly coaching cycle for the following academic year.

At the end of that year, the coached teachers showed substantial improvements in their scores on the MQI instrument. They also had improved scores in pupil perceptions of classroom practices. However, there was no measurable impact on state achievement tests (the Smarter Balance Assessment Consortium (SBAC)). In the follow-up year, there were still large impacts on the four MQI dimensions, but none were found on pupils’ assessment of teachers or pupil achievement.

Source: Developing ambitious mathematics instruction through web-based coaching: A randomized field trial. (November 2018). Brown University Working Paper

Maths on a tablet helps low-performing pupils, for a while

Published in the Journal of Educational Psychology, Martin Hassler and colleagues carried out a randomised controlled trial or of a mathematics intervention on tablets (iPads).

The trial involved 283 low-performing second graders (Year 3) spread across 27 urban schools in Sweden. The children were randomised to four groups:

  • A maths intervention called Chasing Planets, consisting of 261 planets on a space map, each with a unique maths exercise (addition or subtraction up to 12). Pupils practised for 20 minutes a day.
  • The maths intervention combined with working memory training, where pupils spent an additional 10 minutes each day on working memory tasks.
  • A placebo group who practised mostly reading tasks on the tablet (again for 20 minutes each day), including Chasing Planets-Reading, which had a similar format to the maths intervention.
  • A control group who received no intervention, not even on improving their skills on the tablets.

The intervention lasted for around 20 weeks, with children completing nine measures at pre- and post-test, and then after six and 12 months.

Both maths conditions scored significantly higher (effect size = +0.53–0.67) than the control and placebo groups on the post-test of basic arithmetic, but not on measures of arithmetic transfer or problem solving. There was no additional benefit of the working memory training. The effects faded at the six-month follow-up (effect size = +0.18–0.28) and even more so after 12 months (effect size = +0.03–0.13).

IQ was a significant moderator of direct and long-term effects, such that children with lower IQ benefited more than higher IQ pupils. Socioeconomic factors did not moderate outcomes.

Source: Short and long-term effects of a mathematics tablet intervention for low performing second graders (November 2018), Journal of Educational Psychology, Vol 110(8)

Low-cost tutoring boosted struggling pupils’ maths results

An evaluation of the Education Endowment Foundation trial of Tutor Trust’s affordable tuition project found that low-cost tutoring in small groups increased maths scores for disadvantaged pupils who are working below age-expected levels in maths.

One hundred and five schools in Manchester and Leeds with double the average numbers of disadvantaged pupils participated in the effectiveness trial of the Tutor Trust project from September 2016 until July 2017. The aim of the project is to improve the maths achievement of disadvantaged pupils by providing small-group tutoring sessions with trained university students and recent graduates.

Year 6 pupils (ages 10–11) who were struggling with maths were selected by their teacher to receive extra support from Tutor Trust tutors, should their school be randomly allocated to the intervention group. The selected pupils in the intervention schools received 12 hours of additional tuition, usually one hour per week for 12 weeks, in groups of three. Pupils in the control schools continued with normal teaching. Achievement was measured using Key Stage 2 maths scores.

The report found that children who received tutoring from Tutor Trust progressed more in maths compared to children in control schools (effect size = +0.19). Among children eligible for free school meals, the effect size was +0.25. There was also some evidence that pupils with lower prior achievement tended to benefit more from the tutoring.

Source: Tutor Trust: Affordable primary tuition evaluation report and executive summary (November 2018), Education Endowment Foundation

Small class size has at best a small effect on academic achievement

Reducing class size is often suggested as a way of improving pupil performance. However evidence from a new Campbell systematic review suggests that reducing class size has at best only a very small effect.

The review summarises findings from relevant studies that measured the effects of class size on academic achievement. A total of 127 studies were analysed, including 45 studies that used data from the US Student Teacher Achievement Ratio (STAR) programme that reduced class sizes substantially in kindergarten to grade 3 (Years 1 to Year 4). However only ten studies, including four of the STAR programme, could be included in the meta-analysis.

Their analysis focused on effects on maths and reading and found a small positive effect of reducing class size on pupils’ reading achievement and a negative, but statistically insignificant, effect on maths. For reading, the weighted average effect size was +0.11, and the weighted average effect size for maths was -0.03.

For the four studies using data from the STAR programme, the researchers found a positive effect of smaller class sizes for both reading and maths. However, the average effect sizes were still very small and do not change the overall finding.

Source: Small class sizes for improving student achievement in primary and secondary schools: a systematic review (October 2018), Campbell Systematic Reviews 2018:10

Effective programmes in primary maths

Marta Pellegrini from the University of Florence and Cynthia Lake, Amanda Inns and Robert E Slavin from Johns Hopkins Center for Research and Reform in Education have released a new report on effective programmes in primary maths. The report reviews research on the mathematics achievement outcomes of all programmes with at least one study meeting the inclusion criteria of the review. A total of 78 studies were identified that evaluated 61 programmes in grades K–5 (Years 1–6).

The studies were very high in quality, with 65 (83%) randomised and 13 (17%) quasi-experimental evaluations. Key findings were as follows:

  • Particularly positive outcomes were found for tutoring programmes.
  • One-to-one and one-to-small group models had equal impacts, as did teachers and paraprofessionals as tutors.
  • Technology programmes showed modest positive impacts.
  • Professional development approaches focused on helping teachers gain in understanding of maths content and pedagogy had no impact on pupil achievement, but more promising outcomes were seen in studies focused on instructional processes, such as cooperative learning.
  • Whole-school reform, social-emotional approaches, maths curricula and benchmark assessment programmes found few positive effects, although there were one or more effective individual approaches in most categories.

The findings suggest that programmes emphasising personalisation, engagement and motivation have most impact in primary maths teaching, while strategies focused on textbooks, professional development for maths knowledge or pedagogy, and other strategies that do not substantially impact pupils’ daily experiences have little impact.

Source: Effective programs in elementary mathematics: A best-evidence synthesis (October 2018), Johns Hopkins University