The impact of a classroom management programme on children’s mental health

Tamsin Ford and colleagues evaluated the effectiveness of the Incredible Years Teacher Classroom Management (IYTCM) programme. The IYTCM programme aims to improve teachers’ classroom management skills and build strong relationships with students and their parents. Teachers are trained to ignore low-level bad behaviour that often disrupts classrooms and instead develop effective behaviour plans that encourage and promote emotional regulation skills.

The study, published in Psychological Medicine, used a cluster randomised controlled trial, in which children ages four to nine from schools across the southwest of England were randomly allocated to undertake the IYTCM programme or continue their usual practice over a 30-month period (with outcomes assessed at 9, 18, and 30 months). One class in each of 80 schools (40 IYTCM, 40 usual practice; 2,075 children in total) participated. Effects of the intervention on students’ mental health were assessed via the Total Difficulties score from the teacher-report version of the Strengths and Difficulties Questionnaire (SDQ). Data on a range of secondary outcomes (e.g., children’s disruptive behaviour, service use), was also collected in addition to detailing the costs of IYTCM compared to usual practice.

The report concludes that IYTCM may provide a small short-term improvement to children’s mental health, particularly for children who are already struggling. The results of the trial showed there was a small reduction in the SDQ Total Difficulties score at 9 months, but not at 18 or 30 months.

Source: The effectiveness and cost-effectiveness of the Incredible Years® Teacher Classroom Management programme in primary school children: Results of the STARS cluster randomised controlled trial. Psychological Medicine, 1-15.

Effect of preschool home visiting on school readiness

A study published in JAMA Pediatrics examines the sustained effects of a preschool home visiting programme on child outcomes in third grade (Year 4). Karen Bierman and colleagues conducted a randomised controlled trial of the Research-Based and Developmentally Informed Parent home visiting program (REDI-P) on 200 families with preschool children recruited from 24 Head Start centres in Pennsylvania.

Families were assigned to either receive the REDI-P intervention or be sent maths learning games in the post (control group). The intervention focused on improving academic performance and social-emotional adjustment, and reducing children’s problems at home. Families received ten visits from home visitors during preschool and six follow-up visits in kindergarten. Parents received coaching to enhance parent–child relationships and home learning materials to support children’s development and school readiness.

Overall, REDI-P produced sustained benefits four years after the intervention, with children in the REDI-P intervention group needing and using fewer school services than children in the control group. Results showed improvements in academic performance in third grade, measured by direct assessments of child sight-word reading fluency (effect size = +0.28) and teacher-rated academic performance in third grade (effect size= +0.29). The intervention also promoted sustained improvements in children’s social-emotional adjustment, reflected in direct assessments of social understanding (effect size = +0.31). REDI-P also produced reductions in the home problems that parents reported (effect size= −0.28).

Source: Effect of Preschool Home Visiting on School Readiness and Need for Services in Elementary School: A Randomized Clinical TrialJAMA Pediatr. 2018;172(8):e181029.

Professional development and early childhood education and care

A meta-analysis published in Review of Educational Research summarises findings from studies that evaluated the effects of in-service training for early childhood teachers on the quality of early childhood education and care (ECEC) and child outcomes. Overall, data from 36 studies with 2,891 teachers was included in the analysis. For studies to qualify, child care quality had to be measured externally with certified raters at the classroom level.

The analysis, carried out by Franziska Egert and colleagues, revealed that at the teacher level, in-service training had a positive effect on the quality of ECEC, with an effect size of +0.68. Furthermore, a subset of nine studies (including 486 teachers and 4,504 children) that provided data on both quality ratings and child development were analysed, and they showed a small effect at the child level (effect size = + 0.14) and a medium effect at the corresponding classroom level (effect size = +0.45).

Source: Impact of In-Service Professional Development Programs for Early Childhood Teachers on Quality Ratings and Child Outcomes: A Meta-Analysis, Review of Educational Research, 88:3 401 – 433.

How significant is publication bias in educational research?

Research syntheses combine the results of all qualifying studies on a specific topic into one overall finding or effect size. When larger studies with more significant effect sizes are published more often than smaller studies with less significant or even null findings, but are of equal study quality, this is referred to as publication bias. The danger of publication bias is that it does not accurately represent all of the research on a given topic, but instead emphasises the most dramatic.

In this month’s Educational Psychology Review, an article by Jason Chow and Erik Eckholm of Virginia Commonwealth University examines the amount of publication bias present in education and special education journals. They examined the differences in mean effect sizes between published and unpublished studies included in meta-analyses (one kind of research synthesis), whether a pattern emerged regarding individual characteristics common in published vs. unpublished studies, and the number of publication bias tests carried out in these meta-analyses.

From 20 journals, 222 meta-analyses met inclusion criteria for the meta-review, with 29 containing enough information to also be eligible for effect size calculations. The researchers found that for the 1,752 studies included in those meta-analyses, published studies had significantly higher effect sizes than the unpublished studies (ES=+0.64), and studies with larger effect sizes were more likely to be published than those with smaller effect sizes. Fifty-eight percent (n=128) of the meta-analyses did not test for publication bias. The authors discuss the implications of these findings.

Source: Do Published Studies Yield Larger Effect Sizes than Unpublished Studies in Education and Special Education? A Meta-review. Educational Psychology Review 30:3, 727–744

Targeted Reading Intervention and struggling readers

Targeted Reading Intervention (TRI) is a programme that uses webcam technology to allow kindergarten and first grade (Year 1 and Year 2) teachers to help struggling readers while being observed by a coach who gives them real-time feedback as they work with a pupil. TRI trains teachers in their strategies during a three-day workshop in the summer, with webcam observations and feedback during the school year.

Researchers at the University of Delaware and the University of North Carolina evaluated the effect of TRI in a two-year randomised evaluation  to examine its one-year effects on struggling readers, and to examine if having a teacher teach the programme for two years affected pupil achievement.

The study took place in kindergarten and first grade (Year 1 and Year 2) classrooms in ten schools in high-poverty south eastern rural counties in the US. Subjects were equivalent at baseline on standardised testing in the autumn, and randomisation occurred at the classroom level. During the two years of the study, a total of 50 kindergarten (Year 1) classrooms (26 treatment, 24 control) and 50 first grade (Year 2) classrooms (29 treatment, 21 control) at each school were randomised, and then three struggling readers from each classroom were selected to either the treatment or control condition in each year of the study. In total, 305 pupils were assigned to receive TRI training, and 251 pupils served in the untreated control group. Treatment pupils worked with teachers one-to-one, 15 minutes a day every day for six to eight weeks. Spring post-tests showed that struggling readers who received TRI showed greater gains than struggling readers in the control condition (effect size =+0.26). Longevity of teaching the programme did not show any significant effect on pupil achievement.

Researchers also report on the results for the subset of pupils experiencing the programme who had English as an additional language, which may be found here.

Source: Improving struggling readers’ early literacy skills through a tier 2 professional development program for rural classroom teachers: The Targeted Reading Intervention (June2018), The Elementary School Journal 2018 118:4, 525-548

A systematic review of RCTs in education research

The use of randomised controlled trials (RCTs) in education research has increased over the last 15 years. However, the use of RCTs has also been subject to criticism, with four key criticisms being that it is not possible to carry out RCTs in education; the research design of RCTs ignores context and experience; RCTs tend to generate simplistic universal laws of “cause and effect”; and that they are descriptive and contribute little to theory.

To assess these four key criticisms, Paul Connolly and colleagues conducted a systematic review of RCTs in education research between 1980 and 2016 in order to consider the evidence in relation to the use of RCTs in education practice.

The systematic review found a total of 1,017 RCTs completed and reported between 1980 and 2016, of which just over three-quarters have been produced in the last 10 years. Just over half of all RCTs were conducted in North America and just under a third in Europe. This finding addresses the first criticism, and demonstrates that, overall, it is possible to conduct RCTs in education research.

While the researchers also find evidence to oppose the other key criticisms, the review suggests that some progress remains to be made. The article concludes by outlining some key challenges for researchers undertaking RCTs in education.

Source:  The trials of evidence-based practice in education: a systematic review of randomised controlled trials in education research 1980–2016 (July 2018), Educational Research, DOI: 10.1080/00131881.2018.1493353