An evaluation of PACE Center for Girls

Megan Millenky and colleagues from MDRC have released a new reporton an evaluation of PACE Center for Girls. PACE, a Florida-based organisation, provides academic and social services to at-risk middle and high school girls. According to the report, PACE operates daily, year-round; on a typical day, girls attend academic classes and receive additional support such as individual counselling, academic advice, and referrals to other services.

The research team used a random assignment design to evaluate the impact of PACE. From August 2013 to November 2015, a sample of 1,125 girls were enrolled in the study (673 in the programme group, and 452 in the control group). Data sources included administrative records, a survey, and interviews.

Key findings from the study were as follows:

  • The programme group received more academic and social services — and received them more often from a professional source — than the control group.
  • Over a one-year period, PACE increased school enrolment and attendance for the girls it served, compared with the control group. Girls in the programme group were also more likely to be “on track” academically than those in the control group.
  • Girls in both the programme and control groups appeared goal-orientated and hopeful about their futures and reported relatively low levels of risky behaviour one year after study enrolment.
  • The cost of PACE’s holistic package of services is, on average, $10,400 per pupil more than the cost of the services received by control group members through academic and social services provided in the community. The additional cost is largely driven by PACE’s extensive social services; the cost of academic services is similar to those of Florida public schools.

The authors note that further follow-up research would be necessary to see whether PACE affects longer-term academic and delinquency outcomes and to complete a full benefit-cost analysis.

Source: Focusing on girls’ futures: Results from the evaluation of PACE Center for Girls (January 2019), MDRC

Helping youth transition to adulthood

A new report by Cynthia Miller and colleagues at MDRC examines four-year results from a national evaluation of YouthBuild. The report describes YouthBuild as a programme that attempts to improve prospects for less-educated young people, serving over 10,000 individuals each year at over 250 organisations nationwide. Each organisation provides hands-on, construction-related or other vocational training, educational services, case management, counselling, service to the community, and leadership-development opportunities, to low-income young people ages 16 to 24 who did not complete secondary school.

MDRC evaluated the YouthBuild programme using a randomised controlled trial. Study participants were either invited to enrol in YouthBuild (the intervention group) or referred to other services in the community (the control group). A total of 75 programmes across the country were included, with a sample of nearly 4,000 young people who enrolled in the study between 2011 and 2013. Data included in-person observations, survey data, and administrative records.

Key findings of the evaluation included:

  • YouthBuild increased the receipt of high school equivalency credentials.
  • YouthBuild increased enrolment in college, largely during the first two years. Very few young people had earned a degree after four years, and the programme had a very small effect on degree receipt.
  • YouthBuild increased survey-reported employment rates, wages, and earnings, but did not increase employment as measured with employer-provided administrative records, which might not include certain kinds of employment and other types of informal work.
  • YouthBuild increased civic engagement, largely via participation in YouthBuild services. It had no effects on other measures of positive youth development.

Overall, the authors say the effects observed through four years indicate that the programme provides a starting point for redirecting otherwise disconnected young people.

Source: Laying a foundation: Four-year results from the national YouthBuild evaluation (May 2018), MDRC

Reviewing the evidence on career and technical education

A new report by Rachel Rosen and colleagues at MDRC reviews the available research evidence supporting various types of career and technical education (CTE) programmes, examining both the amount of evidence available in each area and its level of rigour. The report details several CTE programme types (eg, instruction and training, apprenticeships and readiness skills training) and provides a literature review of the available evidence to support each programme type.

Key findings were as follows:

  • The most evidence exists for CTE course work and training. In that area, there are multiple studies suggesting that participation in CTE can improve pupils’ outcomes. In addition, multiple studies found that career-related certificates and associate’s degrees are linked to increased wages.
  • Several career pathway models, particularly career academies and early college high schools, are also supported by strong, rigorous studies that provide evidence of positive benefits for pupils.
  • The evidence for other models and for individual programme components is weaker. The authors suggest that these models and components probably need to be evaluated further.

Source: Career and technical education current policy, prominent programs, and evidence (September 2018), MDRC

Evidence and policy

In a review of important 2017 releases, MDRC recently referenced a memo to policymakers with recommendations for increasing research use and applying evidence to all policy decisions, both educational and otherwise.

Recommendations included:

  • Programmes and policies should be independently evaluated. To ensure high-quality evaluations, they should be directly relevant to policy, free of political or other influences and credible to subjects and consumers.
  • The government should provide incentives for programmes to apply evidence results to improve their performance.
  • Utilise a tiered evidence strategy, such as is used in the Every Student Succeeds Act, to set clear guidelines for standards.
  • Existing funding sources should be applied to generate evidence. A 1% set-aside was recommended.
  • Federal and state agencies should be allowed to access and share their data for evaluation purposes.

Source: Putting evidence at the heart of making policy (February 2017), MDRC

Evidence on the long-term effects of home visiting programmes

Children from low-income families are more likely than those from higher-income families to have poor social, emotional, cognitive, behavioural and health outcomes. One approach that has helped parents and their young children is home visiting, which provides information, resources and support to expectant parents and families with young children.

This MDRC brief summarises prior evidence on the effects of four evidence-based models of home visiting using information from seven studies of families with children aged 5- to 21-years-old. Specifically, the brief looks at what the effects of home visiting are for families as children get older, and how the monetary benefits of home visiting compare with their costs.

  • The key findings of the report include:
    Evidence-based home visiting has improved outcomes for parents and children across a wide range of child ages, outcome areas, and national models.
  • Evidence-based home visiting appears to be cost-effective in the long term.
  • The largest benefits from evidence-based home visiting come through reduced spending on government programmes and increased individual earnings.

The information in this brief will inform the design of a study to assess the long-term effects of home visiting. It will suggest where this long-term follow-up study can seek to replicate prior results, where it can try to fill gaps in current knowledge, and which outcomes are important to measure in order to assess the benefits and costs of home visiting.

Source: Evidence on the long-term effects of home visiting programs: Laying the groundwork for long- term follow-up in the mother and infant home visiting program evaluation (MIHOPE). (September 2017), OPRE Report 2017-73, MDRC

Project-based learning

A working paper from MDRC builds on and updates a literature review of project-based learning (PBL) published in 2000. Focused primarily on articles and studies that have emerged in the last 17 years, the working paper discusses the principles of PBL, how PBL has been used in K–12 (Year 1–13) settings, the challenges teachers face in implementing it, how school and local factors influence its implementation and what is known about its effectiveness in improving learning outcomes.

The report suggests that the evidence for PBL’s effectiveness in improving pupil outcomes is “promising, but not proven”.  The biggest challenge to evaluating the effectiveness of PBL, the researchers suggest, is a lack of consensus about the design of PBL and how it fits in with other teaching methods. Some studies have found positive effects associated with the use of PBL. However, without a clear vision of what a PBL approach should look like, it is difficult for teachers and schools to assess the quality of their own implementation and know how to improve their approach. They also suggest that PBL implementation is particularly challenging because it changes pupil–teacher interactions and requires a shift from teacher-directed to pupil-directed inquiry and requires non-traditional methods of assessment.

The paper concludes with recommendations for advancing the PBL research literature in ways that will improve PBL knowledge and practice.

Source: Project-Based Learning: a literature review (October 2017), A MDRC Working Paper