The use of randomised controlled trials (RCTs) in education research has increased over the last 15 years. However, the use of RCTs has also been subject to criticism, with four key criticisms being that it is not possible to carry out RCTs in education; the research design of RCTs ignores context and experience; RCTs tend to generate simplistic universal laws of “cause and effect”; and that they are descriptive and contribute little to theory.
To assess these four key criticisms, Paul Connolly and colleagues conducted a systematic review of RCTs in education research between 1980 and 2016 in order to consider the evidence in relation to the use of RCTs in education practice.
The systematic review found a total of 1,017 RCTs completed and reported between 1980 and 2016, of which just over three-quarters have been produced in the last 10 years. Just over half of all RCTs were conducted in North America and just under a third in Europe. This finding addresses the first criticism, and demonstrates that, overall, it is possible to conduct RCTs in education research.
While the researchers also find evidence to oppose the other key criticisms, the review suggests that some progress remains to be made. The article concludes by outlining some key challenges for researchers undertaking RCTs in education.
Source: The trials of evidence-based practice in education: a systematic review of randomised controlled trials in education research 1980–2016 (July 2018), Educational Research, DOI: 10.1080/00131881.2018.1493353
A new Campbell Collaboration systematic review has been published, which looks at the impact of Teach for America on learning outcomes.
Teach for America (TFA) is a nationwide teacher preparation programme designed to address the shortage of effective teachers, specifically in high-poverty rural and urban schools across the United States. The systematic review by Herbert Turner and colleagues considered the impact of TFA-prepared teachers relative to novice teachers, and alumni relative to veteran teachers. The impacts studied were for K–12 (Years 1–13) pupil outcomes in maths, English and science.
A total of 24 studies were eligible for the review. However, once the research design, study quality and comparison groups were considered, this was reduced to four qualifying studies.
The review found no significant effect on reading by TFA teachers in their first or second year teaching elementary grades (Years 1–6) when compared with non-TFA novice teachers. There was a small positive impact for pre-K to grade 2 (Reception to Year 3) teachers on reading, but not on maths. However, given the small evidence base, the review counsels that these results should be treated with caution.
Source: What are the effects of Teach for America on math, English language arts, and science outcomes of K–12 students in the USA? (June 2018), A Campbell Systematic Review 2018:7
Research published in JAMA Pediatrics has found there are a wide range of different approaches that can be effective in improving self-regulation skills (the ability to control emotions, avoid inappropriate or aggressive behaviour and engage in self-directed learning) in children and teenagers.
Anuja Pandey and colleagues conducted a systematic review and meta-analysis of evaluations of interventions designed to improve pupils’ self-regulation. Data from 49 studies with a total of more than 23,000 pupils ranging in age from 2 to 17 years was examined. The interventions were classified as curriculum-based programmes (n=21), mindfulness and yoga interventions (n=8), family-based programmes (n=9), exercise-based programmes (n=6) and interventions focused on social and personal skills (n= 6). The researchers found that most interventions (n=33) were successful in improving pupils’ ability to manage behaviour and emotion. A meta-analysis showed there was a positive effect of the interventions, with a pooled effect size of +0.42.
There was no age group in which interventions were more effective. While a curriculum-based approach was most commonly used to deliver interventions, the study found that self-regulation interventions can be effective in family settings targeting parenting practices and sibling relationships.
Source: Effectiveness of universal self-regulation–based interventions in children and adolescents: a systematic review and meta-analysis (April 2018), JAMA Pediatrics Doi:10.1001/jamapediatrics.2018.0232
A systematic review published by the Campbell Collaboration summarises the research on the correlation between reading-related preschool predictors, such as code-related skills and linguistic comprehension, and later reading comprehension skills.
Sixty-four longitudinal studies met the eligibility criteria for the review. These studies spanned 1986 to 2016 and were mostly carried out in the US, Europe and Australia. Overall, the findings of the review found that code-related skills (rhyme awareness, phoneme awareness, letter knowledge and rapid automatised naming) are most important for reading comprehension in beginning readers, but linguistic comprehension (grammar and vocabulary) gradually takes over as children become older. All predictors, except for non-word repetition, were moderately to strongly correlated with later reading comprehension. Non-word repetition had only a weak to moderate correlation to later reading comprehension ability.
These results suggest a need for a broad focus on language skills in preschool-age children in order to establish a strong foundation for reading comprehension.
Source: Preschool predictors of later reading comprehension ability: a systematic review (December 2017), A Campbell Systematic Review 2017:4, Campbell Collaboration
Children with Attention-Deficient Hyperactivity Disorder (ADHD) can have trouble with hyperactivity, impulsivity, inattention and distractibility, all of which can affect language and communication and can lead to low academic performance and antisocial behavior. A systematic review published in the Journal of Child Psychology and Psychiatry seeks to establish the types of language problems children with ADHD experience in order to inform future research into how these language problems contribute to long-term outcomes for children with ADHD.
Hannah Korrel and colleagues examined the last 35 years of ADHD research and identified 21 studies using 17 language measures, which included more than 2,000 participants (ADHD children = 1,209; non-ADHD children = 1,101) for inclusion in the systematic review.
The study found that children with ADHD had poorer performance than non-ADHD children on 11 of the 12 measures of overall language (effect size = 1.09). Children with ADHD also had poorer performance on measures of expressive, receptive and pragmatic language compared with non-ADHD children.
Source: Research Review: Language problems in children with Attention-Deficit Hyperactivity Disorder – a systematic meta-analytic review (2017), Journal of Child Psychology and Psychiatry Volume 58, Issue 6
This Campbell systematic review examines the effectiveness of mindfulness-based interventions (MBIs) implemented in school settings on cognition, behaviour, socio-emotional outcomes and academic achievement. MBIs are interventions that use a mindfulness component, broadly defined as “paying attention in a particular way: on purpose, in the present moment, non-judgmentally”, and is often combined with yoga, cognitive-behavioural strategies, or relaxation-skills training.
A total of 61 studies are included in the review, but only the 35 randomised or quasi-experimental studies are used in the meta-analysis, with a total of 6,207 pupil participants. Most of the studies were carried out in schools in the US (74%), with some in Asia (5%), Europe (16%) and Canada (5%). The interventions ranged in duration (4–28 weeks), number of sessions (6–125 sessions) and frequency of meetings (once every two weeks to five times a week).
The findings show that MBIs in schools have a small positive effect on cognitive outcomes and socio-emotional outcomes, but do not improve behaviour or academic achievement. There was little heterogeneity for all outcomes, apart from behavioural outcomes, suggesting that the interventions produced similar results across studies on cognitive, socio-emotional and academic outcomes, despite the interventions being quite diverse. Overall, Brandy Maynard and colleagues find a lack of support at post-test to indicate that the positive effects on cognitive and socio-emotional outcomes then translate into positive outcomes on behaviour and academic achievement.
Source: Mindfulness-based interventions for improving cognition, academic achievement, behavior, and socioemotional functioning of primary and secondary school students (March 2017), A Campbell Systematic Review 2017:5