A recent article in the American Educational Research Journal describes a scaled-up replication study of a randomised controlled trial evaluating the effects of Number Rockets, a small-group intervention for children aged 6/7 at risk of mathematical difficulties.
A replication study repeats the procedures of an earlier study to see if the effects of the intervention are consistent. If similarly positive results are reproduced, this lends credence to that programme’s efficacy. Replication rates for education research are often low.
In this case, the original study found a positive impact for pupils using Number Rockets in a single US state district under ideal conditions. It was conducted in ten schools and involved 139 pupils (70 in the experimental group, 69 in control) identified as at-risk because of their performance in the lowest fifth on a screening assessment. The current, scaled-up study involved 76 schools across four districts in four US states, with 994 pupils (615 experimental, 379 control) identified by a screening score as being in the sample’s lowest third.
In an effort to more closely replicate real-world implementation, tutors were provided with less support and monitoring than in the original study. The experimental group received the Number Rockets curriculum in groups of 2-3 pupils three or more times a week for 17 weeks in addition to their regular maths curriculum, while the control pupils continued with their current curriculum. The Number Rockets curriculum is scripted, and focuses on concepts and operations with whole numbers, such as addition/subtraction, equality, comparing quantities, number placement on a number line, and also includes 10 minutes of fact practice.
The Number Rockets group outperformed the control group on the TEMA-3 standardised maths test, replicating the findings of the original study. Coincidentally, effect sizes for the experimental group were +0.34 for both the original and current studies.
Source: Intervention for First Graders With Limited Number Knowledge: Large-Scale Replication of a Randomized Controlled Trial (2015), American Educational Research Journal, 52(3).