Successful results for Success for All

In 2010, the Success for All Foundation (SFAF) was awarded a $50 million Investing in Innovation (i3) scale-up grant from the US Department of Education, helping to expand its comprehensive school improvement programme. As part of the grant, MDRC carried out an independent evaluation of SFAF’s scale-up initiative. MDRC’s third and final report from the evaluation examines the impact of the Success for All (SFA) reading programme over three years, its incremental cost, and the scale-up process itself.

 
A total of 37 schools were involved in the study, with 19 randomly chosen to adopt SFA in all year groups, and 18 control schools, which continued to use their existing reading programmes. Key findings included:
  • SFA is an effective vehicle for teaching phonics. In the average SFA school, the programme registered a notable, statistically significant impact on a measure of phonics skills for second-graders (age 7/8) who had been in SFA for all three years, compared with their control group counterparts. Pupils in the average SFA school performed better than the average control group school on tests of reading fluency and comprehension, but not significantly.
  • For a subgroup of special concern to policy makers and practitioners  pupils entering school with low pre-literacy skills  SFA appears to be especially effective. Second-graders (Year 3) in the average SFA school who had started kindergarten (Year 1) in the bottom half of the sample in terms of their knowledge of the alphabet and their ability to sound out words registered significantly higher scores on measures of phonics skills, word recognition, and reading fluency than similar pupils in control group schools. The impact on comprehension for this group was also positive but not statistically significant.
In conclusion, the authors say, “The scale-up findings show that, for a modest investment, SFA reliably improves the decoding skills of students in kindergarten through second grade, and that it is especially beneficial for students who begin in the lower half in these skills.”
 
Source: Scaling Up the Success for All Model of School Reform: Final Report from the Investing in Innovation (i3) Evaluation (2015), MDRC.

First-year effects of the Communities in Schools programme

MDRC has released a report describing the first-year results of a randomised study of the Communities in Schools (CIS) programme. This is a programme designed to prevent at-risk middle and high school pupils in the US from dropping out by providing them with academic, behavioural, and emotional supports through an organised, in-school, case-managed system.

The study took place in 28 schools during the 2012-2013 school year. The sample included 2,230 pupils, of which 1,140 were assigned to the CIS group, and 1,090 were assigned to receive the regular support services provided by their schools. Both groups were predominantly ethnic minority and low income, and similar in terms of attendance rate, academic achievement, and EAL status, the only difference being that the experimental group was 2.8% more likely to receive free- or reduced-price lunches.

Following one year of services, CIS pupils were more likely than the controls to report having positive relationships with adults outside the home or school setting, to report positive peer relations, and to view education as valuable. However, the case-managed group did not demonstrate more gains in attendance, academics, or discipline than the control group.

The authors discuss areas for improving the programme and will examine the second year of data to continue to assess findings.

Source: Case Management for Students at Risk of Dropping Out: Implementation and Interim Impact Findings from the Communities In Schools Evaluation (2015), MDRC.

The varying impact of an earlier years programme

A paper from MDRC analyses variation in the effects of the Head Start programme in the United States using data from the Head Start Impact Study.

Head Start is the largest US federal programme for early years development of disadvantaged children and has served more than 30 million children since 1965.

The MDRC paper confirms previous studies that suggested substantial variation in the effects of Head Start in relation to the individual, subgroup, and between Head Start Centers.

The main findings were:

  • Head Start improved cognitive outcomes in children with the lowest cognitive skills and tended to reduce disparities between children in key cognitive outcomes.
  • Dual-language and Spanish-speaking children with low pretest scores gained the most from Head Start.
  • Much of the positive effect of Head Start came from mitigating for limited prior English; the positive effect on children with limited English persisted for at least three years.

The added value of Head Start compared with local alternatives varied substantially between Centers and reflected differences in provision (such as hours of care, teacher education, and classroom quality).

Some Head Start Centers were much more effective than alternatives (including parental care) and others were much less effective than alternatives.

Source: Quantifying Variation in Head Start Effects on Young Children’s Cognitive and Socio-Emotional Skills Using Data from the National Head Start Impact Study (2015), MDRC.

Interventions for four-year-olds help three-year-olds, too

A new report from MDRC presents the findings for three-year-old children from the Head Start CARES demonstration. This was a large randomised controlled trial in more than 100 Head Start centres across the US. It tested the effects of three different approaches for improving children’s social-emotional competencies: The Incredible Years, PATHS, and Tools of the Mind – Play. The main trial looked at the impact on four-year-old children and this further analysis examines the impact on three-year-olds who were in the same classes.

The results show:

  • Overall, the approaches increased teachers’ social-emotional instruction, but did not affect other aspects of practice or classroom climate in mixed-age classes. The approaches also improved teacher reports of three-year-olds’ social behaviours and closeness with teachers.
  • The Incredible Years did not produce a statistically significant improvement in teachers’ use of classroom and behaviour-management strategies, but it improved teacher reports of three-year-olds’ social behaviours and closeness with teachers.
  • PATHS improved teachers’ social-emotional instruction and Tools of the Mind – Play improved scaffolding of children’s play. However, there was little evidence that the approaches improved teacher reports of three-year-olds’ social-emotional outcomes.

The findings suggest that it is possible for the benefits of social-emotional interventions to extend to three-year-olds, even when the interventions are designed primarily for four-year-olds. In this study, these benefits were driven primarily by The Incredible Years.

Source: Impacts of Social-Emotional Curricula on Three-Year-Olds: Exploratory Findings from the Head Start CARES Demonstration (2014), MDRC.

Boosting graduation rates ASAP

MDRC has released a randomised assignment study of the City University of New York’s (CUNY) Accelerated Study in Associate Program (ASAP).

ASAP is a three-year-long programme for students who need remedial courses to complete community college requirements. In the US, community colleges provide higher education from the age of 18, awarding certificates, diplomas, and associate’s degrees, but not bachelor’s degrees. Community college students who need remedial courses have low graduation rates, but the results of this study showed that students who participated in ASAP were almost twice as likely to graduate as students in the control group (who did not participate in ASAP). The results of the study form the largest finding of any community college reform in MDRC’s history.

A total of 896 community college students at three of CUNY’s seven campuses participated in the study – 451 received ASAP and 445 received the usual college services. Students were matched on low-income, the need for one or two remedial courses, credits earned, residency, willingness to attend college full time, minimum GPA (Grade Point Average), and major subject choice.

ASAP components included:

  • Students were required to attend college full time, take remedial courses early, and graduate in three years.
  • Students were provided with tutors specific to ASAP.
  • Students received a tuition waiver covering the difference between the financial aid provided and the cost of tuition and fees.

ASAP’s total cost per student was initially $14,000 more than for students who used the usual college services. But MDRC calculated that at the third-year point, the cost of earning a degree was lower for ASAP students than for control. This was because so many more students graduated through ASAP than did students who used the usual college services.
ASAP is being implemented in six of CUNY’s seven colleges.

Source: Doubling Graduation Rates: Three-Year Effects of CUNY’s Accelerated Study in Associate Programs (ASAP) for Developmental Education Students (2015), MDRC.

Small Schools of Choice improve graduation rates and college enrolment

A new update on New York high schools Small Schools of Choice has shown that the positive effects of the approach continue into postsecondary education.

In the 1990s, New York City instituted a high school reform effort called Small Schools of Choice (SSCs). SSCs have 100 students in each year group and emphasise academic rigour and strong student/faculty relations. Students apply to their preferred schools and are selected by lottery.

Researchers at MDRC examined the effects on postsecondary education of attending SSCs in a longitudinal study of more than 12,000 mostly disadvantaged students. They found that students at SSCs had higher graduation rates and were more likely to attend college the following year than the control group of students who applied to SSCs but attended other schools.

Attending an SSC increased graduation rates in the four cohorts studied by an average of 9.4 percentage points. Students at SSCs were more likely than the control groups to remain enrolled in college more than three years later. SSCs made these gains at a 14-16% lower cost per graduate than the control schools (mostly due to higher rates of graduation and fewer students needing a fifth year of high school to graduate).

Source: Headed to College: The Effects of New York City’s Small High Schools of Choice on Postsecondary Enrollment (2014), MDRC