A study conducted by Neil Humphrey and colleagues, published in Public Health Research, reports on the findings of a randomised controlled trial of the social and emotional learning intervention, Promoting Alternative Thinking Strategies (PATHS).
PATHS aims to promote children’s social skills via a taught curriculum, which is delivered by the class teacher. A total of 5,218 children in Years 3–5 (ages 7–9) from 45 primary schools in Greater Manchester participated in the trial. Schools were randomly allocated to deliver PATHS for two years or to continue as normal.
The findings of the study suggest that the impact of PATHS was modest and limited. Immediately after the intervention, there was tentative evidence that PATHS made a small improvement on children’s social skills (effect size = +0.09) as assessed by the Social Skills Improvement System. A small improvement in children’s psychological well-being (effect size = +0.07) was also found immediately after the intervention. However, there were no differences between children from PATHS and control schools for any outcomes at the 12- or 24-month post-intervention follow-ups.
Source: The PATHS curriculum for promoting social and emotional well-being among children aged 7–9 years: a cluster RCT. Public Health Research 6 (10).
With the increasing interest in STEM (science, technology, engineering and maths) curricula comes the need for evidence backing these programmes. One such science programme is The BSCS Inquiry Approach, a comprehensive secondary school science approach based on three key concepts: constructivism, coherence and cohesiveness. The materials are built around the 5E process (engage, explore, explain, elaborate and evaluate). Teaching focuses on evaluating pupils’ current understanding and using inquiry methods to move them to higher understandings. Each of the science disciplines (physical science, life science, earth science, and science and society) is composed of four chapters that repeat common themes, which advance over a three-year period. Designing and carrying out experiments in small groups is important in all topics. Teachers receive seven days of professional development each year, including a three-day summer institute and four one-day sessions, enabling sharing of experiences and introducing new content over time.
To determine the effects of The BSCS Inquiry Approach on pupil achievement, BSCS conducted a two-year cluster-randomised study of the intervention that compared pupils in grades 10–11 (Years 11–12) in nine experimental (n=1,509 pupils) and nine control secondary schools (n=1,543 pupils) in Washington State in the US. A total of 45% of pupils qualified for free or reduced-price lunches. At the end of two years, the BSCS pupils scored higher than controls (effect size=+0.09, p<.05) on the Washington State Science Assessments.
Source: An efficacy trial of research-based curriculum materials with curriculum-based professional development, Biological Sciences Curriculum Study, BSCS
Evidence for Learning in Australia has published an evaluation of Thinking Maths – a professional learning programme for maths teachers to support pupils’ maths learning during the transition between primary and secondary school (currently Year 7 and Year 8 in South Australia).
The evaluation involved 158 schools in South Australia, which were randomly assigned to the intervention (63 schools) or the control group (104 schools). Teachers participated in 30 hours of face-to-face professional learning delivered at 4–5 week intervals over three school terms. The programme focuses on three areas for better teaching and learning of mathematics: (a) using quality task design, (b) sequencing a conceptual development, and (c) using research-informed effective pedagogies.
Pupils whose teachers received Thinking Maths made additional progress in maths when compared to business-as-usual maths classes (effect size = +0.05). However, there were differences between primary and secondary school pupils: the effect size for secondary pupils (Years 8–10) was -0.16, whereas the effect size for primary pupils (Years 5–7) was +0.14.
Source: Thinking Maths: A professional learning program supporting teachers to engage middle-school students in maths. Evaluation Report and Executive Summary, (September 2018). Evidence for Learning, the Australian Council for Educational Research (ACER)
The Education Endowment Foundation has published an evaluation of two trials of programmes developed by the University College-London (UCL) Institute of Education investigating approaches to grouping pupils: Best Practice in Setting and Best Practice in Mixed Attainment Grouping.
The main trial, “Best Practice in Setting”, tested an intervention that aimed to get schools to improve their setting practice (grouping pupils in classes by their current achievement levels). A total of 127 schools took part in the trial, which ran over the course of two academic years. Teachers were randomly allocated to sets to prevent “lower” sets from being disproportionately assigned less-experienced teachers, while pupils in Years 7 and 8 were assigned to sets based on independent measures of achievement, rather than more subjective judgements such as behaviour and peer interactions. There were opportunities throughout the year to re-assign pupils to different sets based on their current level of achievement.
The evaluation found no evidence that the intervention improves outcomes in maths (effect size = -0.01) or English (effect size = -0.08). The process evaluation revealed mixed views from participants, and many interviewees thought that what they were being asked to do represented little change from what they already do.
The researchers noted that because school and teacher buy-in was low and attrition rates for follow-up testing were high, half of the schools in the math trial and more than half of the schools in the English trial stopped the intervention before follow-up, and this makes it difficult to conclude anything certain about the impact of Best Practice in Setting.
Source: Best practice in grouping students. Intervention A: Best practice in setting evaluation report and executive summary, (September 2018). Education Endowment Foundation
Best practice in grouping students. Intervention B: Mixed attainment grouping. Pilot report and executive summary, (September 2018). Education Endowment Foundation
Targeted Reading Intervention (TRI) is a programme that uses webcam technology to allow kindergarten and first grade (Year 1 and Year 2) teachers to help struggling readers while being observed by a coach who gives them real-time feedback as they work with a pupil. TRI trains teachers in their strategies during a three-day workshop in the summer, with webcam observations and feedback during the school year.
Researchers at the University of Delaware and the University of North Carolina evaluated the effect of TRI in a two-year randomised evaluation to examine its one-year effects on struggling readers, and to examine if having a teacher teach the programme for two years affected pupil achievement.
The study took place in kindergarten and first grade (Year 1 and Year 2) classrooms in ten schools in high-poverty south eastern rural counties in the US. Subjects were equivalent at baseline on standardised testing in the autumn, and randomisation occurred at the classroom level. During the two years of the study, a total of 50 kindergarten (Year 1) classrooms (26 treatment, 24 control) and 50 first grade (Year 2) classrooms (29 treatment, 21 control) at each school were randomised, and then three struggling readers from each classroom were selected to either the treatment or control condition in each year of the study. In total, 305 pupils were assigned to receive TRI training, and 251 pupils served in the untreated control group. Treatment pupils worked with teachers one-to-one, 15 minutes a day every day for six to eight weeks. Spring post-tests showed that struggling readers who received TRI showed greater gains than struggling readers in the control condition (effect size =+0.26). Longevity of teaching the programme did not show any significant effect on pupil achievement.
Researchers also report on the results for the subset of pupils experiencing the programme who had English as an additional language, which may be found here.
Source: Improving struggling readers’ early literacy skills through a tier 2 professional development program for rural classroom teachers: The Targeted Reading Intervention (June2018), The Elementary School Journal 2018 118:4, 525-548
The use of randomised controlled trials (RCTs) in education research has increased over the last 15 years. However, the use of RCTs has also been subject to criticism, with four key criticisms being that it is not possible to carry out RCTs in education; the research design of RCTs ignores context and experience; RCTs tend to generate simplistic universal laws of “cause and effect”; and that they are descriptive and contribute little to theory.
To assess these four key criticisms, Paul Connolly and colleagues conducted a systematic review of RCTs in education research between 1980 and 2016 in order to consider the evidence in relation to the use of RCTs in education practice.
The systematic review found a total of 1,017 RCTs completed and reported between 1980 and 2016, of which just over three-quarters have been produced in the last 10 years. Just over half of all RCTs were conducted in North America and just under a third in Europe. This finding addresses the first criticism, and demonstrates that, overall, it is possible to conduct RCTs in education research.
While the researchers also find evidence to oppose the other key criticisms, the review suggests that some progress remains to be made. The article concludes by outlining some key challenges for researchers undertaking RCTs in education.
Source: The trials of evidence-based practice in education: a systematic review of randomised controlled trials in education research 1980–2016 (July 2018), Educational Research, DOI: 10.1080/00131881.2018.1493353