Evaluation of “Code Clubs”

The National Foundation for Education Research (NFER) has published the results of a randomised controlled trial and process evaluation of Code Clubs – a UK network of after-school clubs where children aged 9–11 learn to program by making games, animations, websites and applications. Code Club UK produces material and projects that support the teaching of Scratch, HTML/CSS and Python. The clubs, which are supported by volunteers, usually run for one hour a week after school during term time.

The evaluation, conducted by Suzanne Straw and colleagues, assessed the impact of Code Clubs on Year 5 pupils’ computational thinking, programming skills and attitudes towards computers and coding. Twenty-one schools in the UK took part in the trial which used a pupil-randomised design to compare pupil outcomes in the intervention and control groups. Intervention group pupils attended Code Club during the 2015/16 academic year, while control group pupils continued as they would do normally.

The results of the evaluation showed that attending Code Club for a year did not impact on pupils’ computational thinking any more than might have occurred anyway, but did significantly improve their coding skills in Scratch, HTML/CSS and Python. This was true even when control children learned Scratch as part of the computing curriculum in school. Code Club pupils reported increased usage of all three programming languages – and of computers more generally. However, the evaluation data suggests that attending Code Club for a year does not affect how pupils view their abilities in a range of transferable skills, such as following instructions, problem solving, learning about new things and working with others.

Source: Randomised controlled trial and process evaluation of code clubs (March 2017), National Foundation for Educational Research (NFER)

What is the evidence to support reading interventions?

A review from the National Center for Education Evaluation and Regional Assistance in the US assesses the evidence base supporting reading interventions in grades 1–3 (Years 2–4 in the UK) to improve reading outcomes for pupils struggling with typical classroom reading lessons.

The findings are based on studies of 20 interventions conducted in the US that Russell Gersten and colleagues identified that met the What Works Clearinghouse evidence standards. Of these 20 interventions, 19 produced positive or potentially positive effects in at least one area of reading. Interventions in grade 1 (Year 2) produced lower effects in reading comprehension (+0.39) than in word and pseudo-word reading (+0.45), but higher effects than in passage reading fluency (+0.23). For grade 2 and 3 (Years 3 and 4) interventions, the weighted mean effects in reading comprehension (+0.33) were lower than those for both word and pseudo-word reading (+0.46) and passage reading fluency (+0.37). The strongest and most consistent effects were found in word and pseudo-word reading for all three grades.

Although the evidence supports the efficacy of reading interventions, the review points out that the majority of interventions evaluated are interventions for individual pupils, as opposed to small-group interventions which are more typical in school settings. In addition, most of the interventions include high levels of ongoing support for teachers.

Source: What is the evidence base to support reading interventions for improving student outcomes in grades 1–3? (April 2017), US Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southeast (REL 2017–271)

Re-engaging disconnected youth

This report by Melanie Skemer and colleagues at MDRC presents implementation and early impact results from a random assignment evaluation of the Young Adult Internship Program (YAIP), a subsidised employment programme for young people (ages 16 to 24) in New York City who are disconnected from school and work. YAIP offers participants a 10- to 12-week paid internship along with services such as job training and individual support.

MDRC reports that from July 2013 to March 2014, nearly 2,700 young people were assigned at random to either a programme group, which was offered YAIP services, or to a control group, which was not offered those services. MDRC is measuring outcomes for both groups over time to assess whether YAIP services lead to better outcomes. Data sources include administrative records on wages and postsecondary enrolment, subsidised employment payroll records and surveys conducted approximately 4, 12 and 30 months after participants entered the study.

Key findings from the report include:

  • Participation rates were high: over three-quarters of young people assigned to the programme group worked in a subsidised internship and 86 percent of those young people completed the internship.
  • Programme group members were more likely than control group members to report receiving employment services, as well as advice or support and mentorship from staff members at an agency or organisation. However, substantial numbers of control group members also reported receiving help in these areas.
  • Programme group members were more likely than the control group members to work in the year following random assignment, but the quarterly employment rates of the two groups converged after the YAIP internships ended.

MDRC plans to release a report in 2018 that will present YAIP’s final impact results, with a longer-term follow-up of 30 months, as well as the results of a benefit-cost analysis.

Source: Reengaging New York City’s disconnected youth through work: implementation and early impacts of the young adult internship program. (April 2017), OPRE Report 2017-22, MDRC

Does encouraging contingent talk benefit language development?

A study published in the Journal of Child Psychology and Psychiatry examines whether language outcomes for low socio-economic status (SES) children can be improved by encouraging contingent talk (how often the parent talks about objects in the child’s current focus of attention) through a low-intensity intervention.

In a randomised controlled trial with high- and low-SES families, 142 children aged 11 months and their parents were randomly allocated to either a contingent talk intervention or a dental health control. Families in the intervention watched a video about contingent talk and were asked to practice it for 15 minutes a day for a month. Families were visited in their homes twice when children were 11, 12, 18 and 24 months. Questionnaires were also collected by mail at 15 months. Parent communication was assessed at 11 months (baseline) and after one month. Infant communication was assessed at baseline, 12, 15, 18 and 24 months.

At baseline, the amount of contingent talk children hear is found to be associated with SES, with lower-SES parents engaging in less contingent talk. At post-test (when children were 12 months old) all parents who had taken part in the intervention engaged in more contingent talk, regardless of their socioeconomic status. Lower-SES parents in the intervention group reported that their children produced more words at 15 and 18 months. However, effects of the intervention didn’t persist at 24 months. So while parents’ contingent talk is increased through the intervention, and this is effective in promoting vocabulary growth for lower-SES infants in the short term, these effects are not long-lasting. The study concludes that follow-up interventions may be necessary to produce benefits lasting to school entry.

Source: A randomised controlled trial to test the effect of promoting caregiver contingent talk on language development in infants from diverse socioeconomic status backgrounds (April 2017), The Journal of Child Psychology and Psychiatry doi:10.1111/jcpp.12725

Using data to help ensure equity in suspensions

A new guide is available from the Institute of Education Sciences to help educators in the US to use data to determine if any ethnic groups are being disproportionately suspended or expelled within a school or district, and if so, how to use data to promote equity among all ethnic groups.

The guide is divided into two sections. The first describes how to use multiple data to analyse if a group is being disproportionately suspended or expelled, and how to determine the effectiveness of any interventions that might be in place. It also describes the data that can be used to analyse factors that may be contributing to any disproportion. In cases where a school or district determines there are inequalities that may be unjust, the second section outlines a process that helps promote equitable discipline, called Plan-Do-Study-Act. One district’s successful experience using the Plan-Do-Study-Act process is described in detail. The back of the guide contains websites and resources related to equity in school discipline and quality improvement processes.

Source: School discipline data indicators: A guide for districts and schools (April 2017), US Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Northwest (REL 2017–240)

Mindfulness-based interventions in schools

This Campbell systematic review examines the effectiveness of mindfulness-based interventions (MBIs) implemented in school settings on cognition, behaviour, socio-emotional outcomes and academic achievement. MBIs are interventions that use a mindfulness component, broadly defined as “paying attention in a particular way: on purpose, in the present moment, non-judgmentally”, and is often combined with yoga, cognitive-behavioural strategies, or relaxation-skills training.

A total of 61 studies are included in the review, but only the 35 randomised or quasi-experimental studies are used in the meta-analysis, with a total of 6,207 pupil participants. Most of the studies were carried out in schools in the US (74%), with some in Asia (5%), Europe (16%) and Canada (5%). The interventions ranged in duration (4–28 weeks), number of sessions (6–125 sessions) and frequency of meetings (once every two weeks to five times a week).

The findings show that MBIs in schools have a small positive effect on cognitive outcomes and socio-emotional outcomes, but do not improve behaviour or academic achievement. There was little heterogeneity for all outcomes, apart from behavioural outcomes, suggesting that the interventions produced similar results across studies on cognitive, socio-emotional and academic outcomes, despite the interventions being quite diverse. Overall, Brandy Maynard and colleagues find a lack of support at post-test to indicate that the positive effects on cognitive and socio-emotional outcomes then translate into positive outcomes on behaviour and academic achievement.

Source: Mindfulness-based interventions for improving cognition, academic achievement, behavior, and socioemotional functioning of primary and secondary school students (March 2017), A Campbell Systematic Review 2017:5