The impact of web-based coaching on mathematics instruction

A new working paper, published by Brown University, reports on an online coaching programme that aimed to support maths teachers.

Mathematical Quality of Instruction (MQI) is an observational instrument that helps structure teachers’ and coaches’ reflections about maths teaching. There are four “dimensions” of the instrument – richness of the mathematics, Common Core-aligned pupil practices, working with pupils and mathematics, and teacher errors. In several studies, teachers’ scores on MQI have predicted pupils’ academic achievement gains.

In MQI coaching, teachers selected an element of their practice to work on and filmed one of their lessons. A coach then selected a couple of extracts and chose a comparison stock film clip. The teacher watched the clips and then the two discussed them and developed a plan for improvement.

In the current trial, 142 upper elementary and middle school teachers (Years 4-9) from 51 schools in a mid-western state in the US were assigned to MQI coaching or a control condition. Teachers assigned to MQI coaching took part in a two-day summer school, followed by a bi-weekly coaching cycle for the following academic year.

At the end of that year, the coached teachers showed substantial improvements in their scores on the MQI instrument. They also had improved scores in pupil perceptions of classroom practices. However, there was no measurable impact on state achievement tests (the Smarter Balance Assessment Consortium (SBAC)). In the follow-up year, there were still large impacts on the four MQI dimensions, but none were found on pupils’ assessment of teachers or pupil achievement.

Source: Developing ambitious mathematics instruction through web-based coaching: A randomized field trial. (November 2018). Brown University Working Paper

Maths on a tablet helps low-performing pupils, for a while

Published in the Journal of Educational Psychology, Martin Hassler and colleagues carried out a randomised controlled trial or of a mathematics intervention on tablets (iPads).

The trial involved 283 low-performing second graders (Year 3) spread across 27 urban schools in Sweden. The children were randomised to four groups:

  • A maths intervention called Chasing Planets, consisting of 261 planets on a space map, each with a unique maths exercise (addition or subtraction up to 12). Pupils practised for 20 minutes a day.
  • The maths intervention combined with working memory training, where pupils spent an additional 10 minutes each day on working memory tasks.
  • A placebo group who practised mostly reading tasks on the tablet (again for 20 minutes each day), including Chasing Planets-Reading, which had a similar format to the maths intervention.
  • A control group who received no intervention, not even on improving their skills on the tablets.

The intervention lasted for around 20 weeks, with children completing nine measures at pre- and post-test, and then after six and 12 months.

Both maths conditions scored significantly higher (effect size = +0.53–0.67) than the control and placebo groups on the post-test of basic arithmetic, but not on measures of arithmetic transfer or problem solving. There was no additional benefit of the working memory training. The effects faded at the six-month follow-up (effect size = +0.18–0.28) and even more so after 12 months (effect size = +0.03–0.13).

IQ was a significant moderator of direct and long-term effects, such that children with lower IQ benefited more than higher IQ pupils. Socioeconomic factors did not moderate outcomes.

Source: Short and long-term effects of a mathematics tablet intervention for low performing second graders (November 2018), Journal of Educational Psychology, Vol 110(8)

Low-cost tutoring boosted struggling pupils’ maths results

An evaluation of the Education Endowment Foundation trial of Tutor Trust’s affordable tuition project found that low-cost tutoring in small groups increased maths scores for disadvantaged pupils who are working below age-expected levels in maths.

One hundred and five schools in Manchester and Leeds with double the average numbers of disadvantaged pupils participated in the effectiveness trial of the Tutor Trust project from September 2016 until July 2017. The aim of the project is to improve the maths achievement of disadvantaged pupils by providing small-group tutoring sessions with trained university students and recent graduates.

Year 6 pupils (ages 10–11) who were struggling with maths were selected by their teacher to receive extra support from Tutor Trust tutors, should their school be randomly allocated to the intervention group. The selected pupils in the intervention schools received 12 hours of additional tuition, usually one hour per week for 12 weeks, in groups of three. Pupils in the control schools continued with normal teaching. Achievement was measured using Key Stage 2 maths scores.

The report found that children who received tutoring from Tutor Trust progressed more in maths compared to children in control schools (effect size = +0.19). Among children eligible for free school meals, the effect size was +0.25. There was also some evidence that pupils with lower prior achievement tended to benefit more from the tutoring.

Source: Tutor Trust: Affordable primary tuition evaluation report and executive summary (November 2018), Education Endowment Foundation

Effective programmes in primary maths

Marta Pellegrini from the University of Florence and Cynthia Lake, Amanda Inns and Robert E Slavin from Johns Hopkins Center for Research and Reform in Education have released a new report on effective programmes in primary maths. The report reviews research on the mathematics achievement outcomes of all programmes with at least one study meeting the inclusion criteria of the review. A total of 78 studies were identified that evaluated 61 programmes in grades K–5 (Years 1–6).

The studies were very high in quality, with 65 (83%) randomised and 13 (17%) quasi-experimental evaluations. Key findings were as follows:

  • Particularly positive outcomes were found for tutoring programmes.
  • One-to-one and one-to-small group models had equal impacts, as did teachers and paraprofessionals as tutors.
  • Technology programmes showed modest positive impacts.
  • Professional development approaches focused on helping teachers gain in understanding of maths content and pedagogy had no impact on pupil achievement, but more promising outcomes were seen in studies focused on instructional processes, such as cooperative learning.
  • Whole-school reform, social-emotional approaches, maths curricula and benchmark assessment programmes found few positive effects, although there were one or more effective individual approaches in most categories.

The findings suggest that programmes emphasising personalisation, engagement and motivation have most impact in primary maths teaching, while strategies focused on textbooks, professional development for maths knowledge or pedagogy, and other strategies that do not substantially impact pupils’ daily experiences have little impact.

Source: Effective programs in elementary mathematics: A best-evidence synthesis (October 2018), Johns Hopkins University

Findings from an evaluation of Thinking Maths

Evidence for Learning in Australia has published an evaluation of Thinking Maths – a professional learning programme for maths teachers to support pupils’ maths learning during the transition between primary and secondary school (currently Year 7 and Year 8 in South Australia).

The evaluation involved 158 schools in South Australia, which were randomly assigned to the intervention (63 schools) or the control group (104 schools). Teachers participated in 30 hours of face-to-face professional learning delivered at 4–5 week intervals over three school terms. The programme focuses on three areas for better teaching and learning of mathematics: (a) using quality task design, (b) sequencing a conceptual development, and (c) using research-informed effective pedagogies.

Pupils whose teachers received Thinking Maths made additional progress in maths when compared to business-as-usual maths classes (effect size = +0.05). However, there were differences between primary and secondary school pupils: the effect size for secondary pupils (Years 8–10) was -0.16, whereas the effect size for primary pupils (Years 5–7) was +0.14.

Source: Thinking Maths: A professional learning program supporting teachers to engage middle-school students in maths. Evaluation Report and Executive Summary, (September 2018). Evidence for Learning, the Australian Council for Educational Research (ACER)

Lessons from the scale-up of an early maths programme

Pre-K Mathematics is a supplementary mathematics curriculum for pre-k (Reception) children. It focuses on the pre-k classroom and home learning environments of young children, especially those from families experiencing economic hardship. Activities aim to support mathematical development by providing learning opportunities to increase children’s informal mathematical knowledge.

In an article published in Evaluation Review, Jaime Thomas and colleagues report on a cluster-randomised control trial of the scale-up of Pre-K Mathematics in 140 schools in California (70 intervention schools, 70 control). The post-test measured outcomes on the Early Childhood Longitudinal Study, Birth Cohort Mathematics Assessment (ECLS-B) and the Test of Early Mathematics Ability (TEMA-3) at the end of the pre-k year. Results showed that Pre-K Mathematics had positive and significant effects, with an effect size of +0.30 on the ECLS-B and +0.23 on the TEMA-3.

The authors consider how these results differ from previous, smaller studies of the efficacy and effectiveness of Pre-K Mathematics. They find that effect sizes were usually larger in the earlier studies. As studies became larger, more heterogeneous, and less controlled, they tended to yield smaller results.

Source: The Sequential Scale-Up of an Evidence-Based Intervention: A Case Study (August 2018), Evaluation Review