How much is enough?

There have now been many controlled studies of preventive mental health interventions for young people. For these studies to be useful, practitioners need to know whether the effects shown for a particular intervention are modest, moderate, or large.

Emily Tanner-Smith and colleagues summarised more than 400 mean effect size estimates from 74 meta-analyses that synthesised findings from many trials. All the trials were of programmes aimed at preventing problematic behaviour or emotional problems for young people aged 5-18. The results, published in Prevention Science, indicate that, with few exceptions, the median average effect sizes on various outcomes fell within the range of +0.07 to +0.16. The authors advise that these indicate the level of improvement that has been achieved to date and can serve as a benchmark for assessing the value of new findings.

The report also points out that prevention programmes yielded larger effects on knowledge than on actual behaviour. Providing information to increase knowledge (e.g., about the risks of drug use) is an important component of many programmes, but knowledge does not always correlate strongly with actual behaviour.

Source: Empirically Based Mean Effect Size Distributions for Universal Prevention Programs Targeting School-Aged Youth: A Review of Meta-Analyses (August 2018) Prevention Science

Surprise rewards for good attendance had a surprising consequence

A working paper by Carly Robinson and colleagues, published by the Harvard Kennedy School, reports on an experiment to measure the impact of attendance rewards on pupils.

The trial included 15,629 sixth to twelfth grade pupils (Year 7-13) from 14 school districts in California. All the pupils had previously had perfect attendance in at least one month in the autumn. The pupils were randomly allocated to one of three groups:

  • “Prospective Award” pupils received a letter telling them they would receive a certificate if they achieved perfect attendance in February (the following month).
  • “Retrospective Award” pupils received a letter and certificate telling them they had earned an award for perfect attendance during one month in the autumn term.
  • Control pupils received no communication.

The researchers collected data on the pupils’ attendance in the following month (February). They found there was no impact of offering the prospective reward on subsequent attendance. They also found that offering the retrospective award resulted in pupils attending less school in February. Absences among this group increased by 8% (an average of 0.06 days per pupil). The researchers suggest that the retrospective awards may have sent unintended signals to the pupils, telling them that they were performing better than the descriptive social norm of their peers, and exceeding the institutional expectations for the awarded behaviour.

Source: The Demotivating Effect (and Unintended Message) of Retrospective Awards (July 2018) HKS Faculty Research Working Paper Series RWP18-020.

Examining the effects of parental involvement

A paper by Lisa Boonk and colleagues, published in Educational Research Review, reviews the research literature on the relationship between parental involvement and students’ academic achievement.

To be eligible for the paper, studies had to (a) investigate parental involvement and its relation with academic achievement of learners aged 0 to 18; (b) provide clear descriptions of the parental involvement construct and measurements and type of academic outcome; and (c) be published in the period 2003 to 2017 in a peer-reviewed journal. A total of 75 studies were included.

After reviewing the literature, the authors found that parental involvement variables that show promise according to their correlations with academic achievement are:

  • reading at home
  • parents who hold high expectations/aspirations for their children’s academic achievement and schooling
  • communication between parents and children regarding school
  • parental encouragement and support for learning.

Source: A review of the relationship between parental involvement indicators and academic achievement (June 2018) Educational Research Review.

The impact of a classroom management programme on children’s mental health

Tamsin Ford and colleagues evaluated the effectiveness of the Incredible Years Teacher Classroom Management (IYTCM) programme. The IYTCM programme aims to improve teachers’ classroom management skills and build strong relationships with students and their parents. Teachers are trained to ignore low-level bad behaviour that often disrupts classrooms and instead develop effective behaviour plans that encourage and promote emotional regulation skills.

The study, published in Psychological Medicine, used a cluster randomised controlled trial, in which children ages four to nine from schools across the southwest of England were randomly allocated to undertake the IYTCM programme or continue their usual practice over a 30-month period (with outcomes assessed at 9, 18, and 30 months). One class in each of 80 schools (40 IYTCM, 40 usual practice; 2,075 children in total) participated. Effects of the intervention on students’ mental health were assessed via the Total Difficulties score from the teacher-report version of the Strengths and Difficulties Questionnaire (SDQ). Data on a range of secondary outcomes (e.g., children’s disruptive behaviour, service use), was also collected in addition to detailing the costs of IYTCM compared to usual practice.

The report concludes that IYTCM may provide a small short-term improvement to children’s mental health, particularly for children who are already struggling. The results of the trial showed there was a small reduction in the SDQ Total Difficulties score at 9 months, but not at 18 or 30 months.

Source: The effectiveness and cost-effectiveness of the Incredible Years® Teacher Classroom Management programme in primary school children: Results of the STARS cluster randomised controlled trial. Psychological Medicine, 1-15.

Effect of preschool home visiting on school readiness

A study published in JAMA Pediatrics examines the sustained effects of a preschool home visiting programme on child outcomes in third grade (Year 4). Karen Bierman and colleagues conducted a randomised controlled trial of the Research-Based and Developmentally Informed Parent home visiting program (REDI-P) on 200 families with preschool children recruited from 24 Head Start centres in Pennsylvania.

Families were assigned to either receive the REDI-P intervention or be sent maths learning games in the post (control group). The intervention focused on improving academic performance and social-emotional adjustment, and reducing children’s problems at home. Families received ten visits from home visitors during preschool and six follow-up visits in kindergarten. Parents received coaching to enhance parent–child relationships and home learning materials to support children’s development and school readiness.

Overall, REDI-P produced sustained benefits four years after the intervention, with children in the REDI-P intervention group needing and using fewer school services than children in the control group. Results showed improvements in academic performance in third grade, measured by direct assessments of child sight-word reading fluency (effect size = +0.28) and teacher-rated academic performance in third grade (effect size= +0.29). The intervention also promoted sustained improvements in children’s social-emotional adjustment, reflected in direct assessments of social understanding (effect size = +0.31). REDI-P also produced reductions in the home problems that parents reported (effect size= −0.28).

Source: Effect of Preschool Home Visiting on School Readiness and Need for Services in Elementary School: A Randomized Clinical TrialJAMA Pediatr. 2018;172(8):e181029.

Professional development and early childhood education and care

A meta-analysis published in Review of Educational Research summarises findings from studies that evaluated the effects of in-service training for early childhood teachers on the quality of early childhood education and care (ECEC) and child outcomes. Overall, data from 36 studies with 2,891 teachers was included in the analysis. For studies to qualify, child care quality had to be measured externally with certified raters at the classroom level.

The analysis, carried out by Franziska Egert and colleagues, revealed that at the teacher level, in-service training had a positive effect on the quality of ECEC, with an effect size of +0.68. Furthermore, a subset of nine studies (including 486 teachers and 4,504 children) that provided data on both quality ratings and child development were analysed, and they showed a small effect at the child level (effect size = + 0.14) and a medium effect at the corresponding classroom level (effect size = +0.45).

Source: Impact of In-Service Professional Development Programs for Early Childhood Teachers on Quality Ratings and Child Outcomes: A Meta-Analysis, Review of Educational Research, 88:3 401 – 433.