No evidence of impact for a modularisation and self-paced computer-assisted approach to college maths

One of the greatest challenges facing community colleges in the US is that most students’ maths skills are below college level. These students are often referred to developmental maths courses, however, most students never complete the course and fail to earn a college degree.

A study published in Journal of Research on Educational Effectiveness looks at whether a modularised, computer-assisted approach that allows students to move at their own pace through the developmental maths course has any impact on students’ likelihood of completing the developmental maths course, compared with more traditional teaching.

The findings of the randomised trial of 1,400 students found that although the programme was well-implemented, there was no evidence that it was any more or less effective than traditional courses at helping students complete the developmental maths course. The researchers comment that although the results are disappointing, they are important because modularisation and self-paced computer-assisted approaches are popular teaching methods.

Source: A randomized controlled trial of a modularized, computer-assisted, self-paced approach to developmental math (September 2019), Journal of Research on Educational Effectiveness

Digital feedback in primary maths

The Education Endowment Foundation has published an evaluation of Digital Feedback in Primary Maths, a programme that aims to improve primary school teachers’ feedback to pupils.

The intervention uses a tablet application called Explain Everything, diagnostic assessments, and training on effective feedback. The app allows teachers to provide pupils with digitally recorded feedback on a tablet, rather than written feedback. Pupils have the opportunity to review their feedback and develop their work further. By improving teachers’ diagnostic and feedback skills when teaching maths in primary schools, the intervention aims to ultimately improve pupils’ outcomes in maths.

To estimate the impact of Digital Feedback on maths achievement, the evaluation used a randomised controlled trial involving 2,564 pupils in 108 classes across 34 English primary schools. While the intervention took place in each school, classrooms were randomly assigned to the treatment or control group, which carried on with business-as-usual teaching.

The results of the evaluation found no evidence that pupils taking part in the programme made more progress in maths, on average (effect size = -0.04), than the control group. 

Source: Digital feedback in primary maths (September 2019), Education Endowment Foundation

Computer games to improve children’s maths and science achievement

An independent evaluation of Stop and Think: Learning Counterintuitive Concepts has found evidence of a positive impact in maths and science outcomes for pupils in Key Stage 2.

The Learning Counterintuitive Concepts project, funded by the Education Endowment Foundation and Wellcome, aimed to improve science and maths achievement for Year 3 (7–8 year olds) and Year 5 (9–10 year olds) using an intervention called Stop and Think. When learning new concepts in science and maths, pupils must be able to inhibit prior contradictory knowledge and misconceptions to acquire new knowledge successfully. Stop and Think is a computer-assisted learning activity that aims to improve a learner’s ability to adapt to counterintuitive concepts by training them to inhibit their initial response, and instead, give a slower and more reflective answer.

The randomised controlled trial involved 6,672 children from 89 schools across England. The intervention was delivered to the whole class and consisted of 30 sessions delivered for a maximum of 15 minutes, three times a week, for 10 weeks at the start of maths or science lessons.

The results suggest that pupils who participated in Stop and Think made more progress in science and maths on average, compared to children in the business-as-usual control group.  The combined effect size across the two year groups for maths was +0.09 and +0.12 for science.

To check whether this impact was due to the Stop and Think game specifically, or was a result of the extra pupil engagement and motivation arising from having a fun computer-based activity at the start of lessons more generally, schools were offered an alternative computer-based programme that did not include any content from Stop and Think. Intervention-group pupils also made more progress than pupils in this “active” control group. The combined effect sizes for maths and science were +0.13 and +0.15 respectively.

Source: Stop and Think: Learning counterintuitive concepts. Evaluation report (September 2019), Education Endowment Foundation

Examining the evidence on Learning Accounts

Social Programs That Work has released a new evidence summary on Learning Accounts, a demonstration programme in New Brunswick, Canada that provided up to $8,400 in conditional financial aid for post-secondary education to low-income 10th grade (Year 11) pupils. The pupils had to meet certain benchmarks (ie, completion of 10th, 11th, and 12th grade (Years 11-13)) to receive the funding.

The programme was evaluated through a randomised controlled trial with a sample of 1,145 low-income 10th graders in 30 high schools in New Brunswick, Canada. Within each school, the low-income pupils were randomly assigned to a group that was offered participation in the Learning Accounts programme, or to a control group that received usual school services. Survey data was used to measure high school graduation rates, and administrative data was used to examine later graduation from college.

According to the evidence report, over the 10 years following random assignment, the programme produced a 6.5 percentage point increase in the high school graduation rate, and 6.8 percentage point increase in the rate of post-secondary completion.

Source: Learning Accounts (September 2019), Social Programs That Work

Improving the maths and reading skills of children in foster care

A study published in Oxford Review of Education evaluates the effects of TutorBright tutoring on the reading and maths skills of children in family foster care. TutorBright uses one-to-one, at-home tutoring with detailed instructor’s manuals and customised pupil workbooks. Children receive two one-hour tutoring sessions per week, on designated days of the week, for up to 50 hours of tutoring. Children in the waiting list control group were asked to continue with their schooling as usual and not seek additional tutoring or academic support during the school year, and were then offered the tutoring intervention at the end of the school year. TutorBright tutors all had experience with teaching or mentoring, and an undergraduate or master’s degree (completed or in progress).

For the randomised controlled trial, conducted by Andrea J Hickey and Robert J Flynn, child welfare workers nominated foster care children in Ontario, Canada, who met the following criteria: enrolled in grades 1–11 (Years 2–12), fluent in English, currently living in a foster-family setting, and judged likely to remain in care for the duration of the study. Thirty-four children were randomly assigned to tutoring, and 36 to a waiting-list control condition.

The results suggest that the tutored children made greater gains than those in the control group in reading fluency (effect size = +0.16), reading comprehension (ES = +0.34) and maths calculation (ES = +0.39).

Source: Effects of the TutorBright tutoring programme on the reading and mathematics skills of children in foster care: a randomised controlled trial (July 2019), Oxford Review of Education, 45:4

Improving the language and communication of secondary school children with language difficulties

A research report published in the International Journal of Language and Communication Disorders investigates the effectiveness of teaching assistant (TA)-delivered narrative and vocabulary interventions to secondary school children with language difficulties.

Researchers at City University of London and University of Oxford conducted a randomised controlled trial in two outer London boroughs. Across 21 schools, 358 Year 7 underperforming pupils (mean age = 12.8 years) were recruited, and randomised to four groups within each school: vocabulary intervention, narrative intervention, combined narrative and vocabulary intervention, and delayed waiting control group. The narrative programme focused on the understanding and telling of stories, using a story structure to support story generation. Pupils were introduced to different types of stories (fictional, non‐fictional, scripts) and narrative genres. The vocabulary programme focused on developing key concepts and vocabulary items relevant to the curriculum (eg, nutrition) and age-appropriate (eg, careers). A variety of tasks including word associations, categorisation, mind‐mapping and word‐building were used to reinforce word learning.

The language and communication programmes (narrative, vocabulary, and combined narrative and vocabulary) were delivered by TAs in the classroom, three times per week, for 45–60 min each, over six weeks, totalling 18 sessions. Assessments were conducted pre- and post-intervention.

Overall, pupils in the intervention groups made greater improvements on standardised measures of narrative (effect size = +0.296), but not vocabulary skills, compared with control group children.

Source: Improving storytelling and vocabulary in secondary school students with language disorder: a randomized controlled trial (March 2019), International Journal of Language and Communication Disorders, 54:4