The effects of a volunteer mentoring programme on reading outcomes

This article from the Journal of Early Childhood Research presents findings of a randomised controlled trial evaluation of the effects of a volunteer mentoring programme on reading outcomes among struggling readers aged eight to nine years. The trial involved children from 50 primary schools who received two 30-minute mentoring sessions per week from volunteer mentors that involved paired reading activities.

The evaluation showed that the programme was effective in improving decoding skills, reading rate, and reading fluency. However, no evidence was found of the programme having an effect on reading comprehension or reading confidence and enjoyment of reading. The findings make an important contribution to the existing evidence in this area, and show that mentoring programmes that use non-specialist volunteers, rather than teachers or highly trained mentors, can be effective in improving some core reading but may be less effective in improving reading comprehension.

Source: The effects of a volunteer mentoring programme on reading outcomes among eight- to nine-year-old children: A follow up randomised controlled trial (2012), Journal of Early Childhood Research, 10(2)

Improving college readiness and enrollment for disadvantaged populations

This paper from the Society for Research on Educational Effectiveness synthesises the evidence on the effectiveness of programmes designed to improve college readiness and enrollment for disadvantaged populations in the US. The purpose of the paper is to provide guidance for policymakers and practitioners implementing college access programmes, and to identify important gaps in the scientific evidence base that warrant further research.

The authors note that their findings are still preliminary. However, they do identify two early conclusions:

  • Measures of completed coursework are the best pre-college predictors of college graduation. The authors encourage evaluators to consider including these outcome measures in their evaluations of college access programmes.
  • The sharp differences in the size of estimated impacts between quasi-experimental designs (QEDs) and randomised controlled trials raise questions about the extent to which QEDs are identifying causal impacts.

Source:Effects of college access programs on college readiness and enrollment: A meta-analysis, Society for Research on Educational Effectiveness

Randomised controlled trial of the Teens and Toddlers programme

This report from the Department for Education presents findings of a randomised controlled trial (RCT) to evaluate the impact of the Teens and Toddlers (T&T) programme, which aims to reduce teenage pregnancy by raising the aspirations and educational attainment of 13- to 17-year-old girls at most risk of leaving education early, social exclusion, and becoming pregnant.

The T&T programme, which consisted of weekly three-hour sessions over 18 to 20 weeks, combined group-based learning with work experience in a nursery. The RCT measured the impact of the programme on a specific set of outcomes while it was taking place, immediately afterwards, and one year later. Immediately after the intervention, there was no evidence of a positive impact on the three primary outcomes:

  • use of contraception;
  • expectation of teenage parenthood; and
  • general social and emotional development.

However, there was evidence of improved self-esteem and sexual-health knowledge, which were secondary outcomes. One year later, the only impact was that the teenagers were less likely to have low self-esteem.

Source: Randomised controlled trial of the ‘teens and toddlers’ programme (2012), Department for Education

Little incentive for rewarding teacher teams

In the last issue of Best Evidence in Brief, we included a PISA in Focus review on performance-based pay for teachers. This US study from the RAND Corporation also looks at performance pay, but specifically at the effects of rewarding teams of teachers. The study, which used a randomised design, included 159 teams of teachers teaching pupils in grades 6 to 8 (KS3) in nine schools. Teachers on selected teams had the opportunity to earn a bonus based on their pupil’s growth in achievement in mathematics, English language arts, science, and social studies.

The study showed that the intervention had no effect on pupil achievement, teacher practices, or teacher attitudes. Pupils taught by teacher teams who were offered incentives scored slightly better on some standardised tests, but the differences were small and not statistically significant.

Source: No evidence that incentive pay for teacher teams improves student outcomes (2012), RAND Corporation

Summer school literacy programme stops pupils slipping back

Pupils who went to summer school had improved their literacy performance at the end of the summer. This randomised trial, from Early Childhood Research Quarterly, examined the effects of a summer literacy programme on struggling readers in the US where summer holidays are longer.

Overall, pupils who didn’t attend summer school showed mean declines in the reading of nonsense words (a standard test of fluency) of approximately five words per minute over the summer. Children who attended summer school at the end of Kindergarten (Year 1) had a fluency gain of approximately 12 words-per-minute. Pupils at the end of Grade 1 (Year 2) had a fluency gain of 7.5 words per minute.

The findings are generally consistent with previous studies of summer school effects and the summer learning outcomes of children, and suggest that summer school can be a useful strategy to support learning over the summer months.

Source: Summer school effects in a randomized field trial (2012), Early Childhood Research Quarterly 28(1)

A practical guide to low-cost randomised controlled trials

This guide from the Coalition for Evidence-Based Policy summarises five well-conducted, low-cost randomised controlled trials (RCTs) carried out in real-world community settings. As the guide states, RCTs are regarded as the strongest method for evaluating programme effectiveness. Evaluations of the Triple P (Positive Parenting Programme) system and New York City’s teacher incentive programme are reviewed. The purpose of the guide is to illustrate the feasibility and value of low-cost RCTs for policy makers and researchers.

Source: Rigorous program evaluations on a budget: How low-cost randomized controlled trials are possible in many areas of social policy (2012), Coalition for Evidence-Based Policy