Randomised controlled trial of the Fostering Changes programme

This randomised controlled trial was conducted in order to investigate the effectiveness of the Fostering Changes programme, which provides practical support and training for foster carers in the UK. The evaluation was carried out on 63 carers across four Greater London local authorities; 34 were randomly allocated to the Fostering Changes training group and 29 to the control group.

Results of the trial showed that there was a significant improvement in the behaviour of children of carers in the Fostering Changes group, compared to children of carers in the control group. There was a large effect on carer-defined problems and a small-to-moderate effect on emotional and behavioural difficulties. The quality of attachment between looked-after child and carer was significantly improved and there were significant positive changes in carer confidence and parenting practices that were related to the skills obtained as a result of the Fostering Changes training course.

Source: Randomised Controlled Trial of the fostering changes programme (2012), Department for Education

New study finds technology improves grammar learning

Hand-held technology can help to improve primary pupils’ learning of grammar, according to a new study by researchers at the Institute for Effective Education (IEE). A randomised evaluation of the use of Questions for Learning (QfL), a technology-enhanced, self-paced learning tool, was conducted in more than 40 primary schools. In QfL, each pupil responds to progressively more difficult questions that are presented on wireless hand-held devices at the rate that the pupil answers them. This allows both more advanced and weaker pupils to answer in a private way at a pace appropriate to them.

Pupils in classes who used QfL showed significant gains in grammar compared with pupils in the control group. This improvement was greater in schools that used QfL at least three days each week, and for low- and average-achieving pupils. If these results held over a school year, these pupils would make between three and four months of additional progress. Both teachers and pupils enjoyed using the strategy for formative assessment, believed it improved pupil achievement in grammar, and would recommend its use for other pupils and for other subjects.

Source: Effects of technology-enhanced formative assessment on achievement in primary grammar (2012), Institute for Effective Education

The effects of a volunteer mentoring programme on reading outcomes

This article from the Journal of Early Childhood Research presents findings of a randomised controlled trial evaluation of the effects of a volunteer mentoring programme on reading outcomes among struggling readers aged eight to nine years. The trial involved children from 50 primary schools who received two 30-minute mentoring sessions per week from volunteer mentors that involved paired reading activities.

The evaluation showed that the programme was effective in improving decoding skills, reading rate, and reading fluency. However, no evidence was found of the programme having an effect on reading comprehension or reading confidence and enjoyment of reading. The findings make an important contribution to the existing evidence in this area, and show that mentoring programmes that use non-specialist volunteers, rather than teachers or highly trained mentors, can be effective in improving some core reading but may be less effective in improving reading comprehension.

Source: The effects of a volunteer mentoring programme on reading outcomes among eight- to nine-year-old children: A follow up randomised controlled trial (2012), Journal of Early Childhood Research, 10(2)

Improving college readiness and enrollment for disadvantaged populations

This paper from the Society for Research on Educational Effectiveness synthesises the evidence on the effectiveness of programmes designed to improve college readiness and enrollment for disadvantaged populations in the US. The purpose of the paper is to provide guidance for policymakers and practitioners implementing college access programmes, and to identify important gaps in the scientific evidence base that warrant further research.

The authors note that their findings are still preliminary. However, they do identify two early conclusions:

  • Measures of completed coursework are the best pre-college predictors of college graduation. The authors encourage evaluators to consider including these outcome measures in their evaluations of college access programmes.
  • The sharp differences in the size of estimated impacts between quasi-experimental designs (QEDs) and randomised controlled trials raise questions about the extent to which QEDs are identifying causal impacts.

Source:Effects of college access programs on college readiness and enrollment: A meta-analysis, Society for Research on Educational Effectiveness

Randomised controlled trial of the Teens and Toddlers programme

This report from the Department for Education presents findings of a randomised controlled trial (RCT) to evaluate the impact of the Teens and Toddlers (T&T) programme, which aims to reduce teenage pregnancy by raising the aspirations and educational attainment of 13- to 17-year-old girls at most risk of leaving education early, social exclusion, and becoming pregnant.

The T&T programme, which consisted of weekly three-hour sessions over 18 to 20 weeks, combined group-based learning with work experience in a nursery. The RCT measured the impact of the programme on a specific set of outcomes while it was taking place, immediately afterwards, and one year later. Immediately after the intervention, there was no evidence of a positive impact on the three primary outcomes:

  • use of contraception;
  • expectation of teenage parenthood; and
  • general social and emotional development.

However, there was evidence of improved self-esteem and sexual-health knowledge, which were secondary outcomes. One year later, the only impact was that the teenagers were less likely to have low self-esteem.

Source: Randomised controlled trial of the ‘teens and toddlers’ programme (2012), Department for Education

Little incentive for rewarding teacher teams

In the last issue of Best Evidence in Brief, we included a PISA in Focus review on performance-based pay for teachers. This US study from the RAND Corporation also looks at performance pay, but specifically at the effects of rewarding teams of teachers. The study, which used a randomised design, included 159 teams of teachers teaching pupils in grades 6 to 8 (KS3) in nine schools. Teachers on selected teams had the opportunity to earn a bonus based on their pupil’s growth in achievement in mathematics, English language arts, science, and social studies.

The study showed that the intervention had no effect on pupil achievement, teacher practices, or teacher attitudes. Pupils taught by teacher teams who were offered incentives scored slightly better on some standardised tests, but the differences were small and not statistically significant.

Source: No evidence that incentive pay for teacher teams improves student outcomes (2012), RAND Corporation