An evaluation of PACE Center for Girls

Megan Millenky and colleagues from MDRC have released a new reporton an evaluation of PACE Center for Girls. PACE, a Florida-based organisation, provides academic and social services to at-risk middle and high school girls. According to the report, PACE operates daily, year-round; on a typical day, girls attend academic classes and receive additional support such as individual counselling, academic advice, and referrals to other services.

The research team used a random assignment design to evaluate the impact of PACE. From August 2013 to November 2015, a sample of 1,125 girls were enrolled in the study (673 in the programme group, and 452 in the control group). Data sources included administrative records, a survey, and interviews.

Key findings from the study were as follows:

  • The programme group received more academic and social services — and received them more often from a professional source — than the control group.
  • Over a one-year period, PACE increased school enrolment and attendance for the girls it served, compared with the control group. Girls in the programme group were also more likely to be “on track” academically than those in the control group.
  • Girls in both the programme and control groups appeared goal-orientated and hopeful about their futures and reported relatively low levels of risky behaviour one year after study enrolment.
  • The cost of PACE’s holistic package of services is, on average, $10,400 per pupil more than the cost of the services received by control group members through academic and social services provided in the community. The additional cost is largely driven by PACE’s extensive social services; the cost of academic services is similar to those of Florida public schools.

The authors note that further follow-up research would be necessary to see whether PACE affects longer-term academic and delinquency outcomes and to complete a full benefit-cost analysis.

Source: Focusing on girls’ futures: Results from the evaluation of PACE Center for Girls (January 2019), MDRC

Interventions to improve student behaviour

The Campbell Collaboration has published a policy brief looking at six systematic reviews of school-based interventions with students who demonstrate at-risk behaviour.

It found that school intervention programmes are marginally effective at ensuring that more students attend school, and at curtailing harmful student behaviours.

Students participating in 28 programmes addressing chronic truancy improved their attendance by nearly five days per year, although in most of the programmes student attendance was still below 90%.

A review of 73 violence prevention programmes found that students showed significantly lower levels of aggressive and disruptive behaviour, including a 7% reduction in fighting on school grounds.

Twelve studies of interventions aimed at improving classroom-wide behaviour found that students in the programmes showed less disruptive behaviour than their peers.

The 44 anti-bullying interventions studied in 16 countries showed average decreases of 20-23% in bullying and 17-20% in victimisation.

Evidence from 12 studies of sexual violence prevention programmes found that students had increased awareness of sexual violence and approaches to conflict resolution, but there was no effect on levels of violent behaviour or victimisation.

Source: Effects of School Based Interventions to Improve Student Behaviour: A Review of Six Campbell Systematic Reviews (2016), The Campbell Collaboration.

Practical support for preventing gang and youth violence

The Early Intervention Foundation has released two new reports on gang and youth violence, based on international evidence. The authors emphasise the importance of early intervention and of providing high-quality, evidence-based support to children and young people at risk of involvement, delivered in the right way by the right people.

The first report looks to identify who is potentially at risk of involvement in gangs or youth violence. Findings are grouped into five domains – individual, peer group, community, school, and family – with the strongest risk factors associated with the individual. This includes behavioural risk factors (eg, violent activity, exposure to and consumption of drugs and alcohol) and explanatory risk factors (eg, psychological issues such as symptoms of ADHD, hyperactivity, self-esteem, levels of aggression, and an inability to say no to peer pressure). The likelihood of involvement increased in line with the number of risk factors.

The second report identifies what types of programmes or interventions appear to be most effective in preventing involvement. Skills-based and family-focused programmes were found to be amongst the most robustly evaluated and effective types of programme. Mentoring, community-based, and sports-based programmes to tackle youth crime and violence appeared promising, but have a limited evidence base. In contrast, approaches based on deterrence and discipline (eg, boot camps) were ineffective, and may even make things worse (eg, increase the likelihood of offending).

Sources: Preventing Gang and Youth Violence: A Review of Risk and Protective Factors (2015), Early Intervention Foundation, and What Works to Prevent Gang Involvement, Youth Violence and Crime: A Rapid Review of Interventions Delivered in the UK and Abroad (2015), Early Intervention Foundation.

Promising results for Career Academies

Career Academies is a US dropout prevention strategy for young people considered most at risk of dropping out of secondary school. The programme integrates academic curricula with career themes, including health care, finance, technology, communications, and public service, and includes work experience through partnerships with local employers.

The What Works Clearinghouse (WWC) has released an intervention report that looks at research on the programme. In particular, the WWC analysed a randomised controlled trial that included approximately 1,400 young pupils in the US who applied to an academy before their ninth- or tenth-grade years (Year 10 or Year 11). The academies were located in eight urban areas in six states.

Based on the research, the WWC found Career Academies to have potentially positive effects on completing school and no discernible effects on staying in school or progressing in school for secondary school aged young people.

Source: WWC Intervention Report: Career Academies (2015), What Works Clearinghouse.

First-year effects of the Communities in Schools programme

MDRC has released a report describing the first-year results of a randomised study of the Communities in Schools (CIS) programme. This is a programme designed to prevent at-risk middle and high school pupils in the US from dropping out by providing them with academic, behavioural, and emotional supports through an organised, in-school, case-managed system.

The study took place in 28 schools during the 2012-2013 school year. The sample included 2,230 pupils, of which 1,140 were assigned to the CIS group, and 1,090 were assigned to receive the regular support services provided by their schools. Both groups were predominantly ethnic minority and low income, and similar in terms of attendance rate, academic achievement, and EAL status, the only difference being that the experimental group was 2.8% more likely to receive free- or reduced-price lunches.

Following one year of services, CIS pupils were more likely than the controls to report having positive relationships with adults outside the home or school setting, to report positive peer relations, and to view education as valuable. However, the case-managed group did not demonstrate more gains in attendance, academics, or discipline than the control group.

The authors discuss areas for improving the programme and will examine the second year of data to continue to assess findings.

Source: Case Management for Students at Risk of Dropping Out: Implementation and Interim Impact Findings from the Communities In Schools Evaluation (2015), MDRC.

Mind-set interventions on a larger scale

A paper from researchers at Stanford University and the University of Texas describes how the institutions delivered brief online mind-set interventions to 1,594 pupils in 13 schools.

The authors reported that previous research on mind-set demonstrated the worth of small-scale interventions, and they set out to gather evidence on whether such interventions are practical on a larger scale.

Mind-set interventions seek to replace unhelpful fixed-mind-set attitudes where intelligence is viewed as unchangeable and praise focuses on passive aspects of achievement (“you are lucky to be so smart/talented”) with growth-mind-set attitudes where intelligence can grow and effort attracts praise (“you worked hard to come up with a good answer”).

Pupils allocated to intervention groups were given two 45-minute sessions two weeks apart (intervention groups had either growth-mind-set or sense-of-purpose sessions or both). Pupils allocated to the control group were given similar sessions but these “lacked the key psychological message that intelligence is malleable.”

One third of the study participants were considered to be at risk of dropping out of high school. Pupils in this subgroup improved their core subject grade point average after the mind-set intervention. Achievement of satisfactory grades (A to C) increased by 6% among the at-risk subgroup; there was no difference in satisfactory grades in the control group.

The authors concluded that their study demonstrated “the potential for academic-mind-set interventions to be effective on a wide scale.”

Source: Mind-set interventions are a scalable treatment for academic underachievement (2015), Psychological Science