More evidence for growth mindset

The findings of an MDRC evaluation of a growth mindset intervention have suggested a positive impact on pupils’ academic performance.

To test whether a growth mindset intervention could improve pupils’ academic performance, the National Study of Learning Mindsets implemented a randomised controlled trial of a low-cost growth mindset intervention specifically designed for ninth grade (Year 10) pupils. A total of 11,888 pupils from 63 high schools across the US took part in the intervention, which included two 25-minute self-administered online training modules on the topic of brain development. Pupils in the intervention group were given modules about growth mindset and were asked to answer reflective questions in a survey. Instead of learning about the brain’s malleability, pupils in the control group learned about basic brain functions, and they were also asked to answer survey questions.

The results of the evaluation found a positive impact on pupils’ average grade point average (GPA) (effect size = +0.04), as well as their maths GPA (effect size = +0.05). Other results from the evaluation suggest that:

  • The intervention changed pupils’ self-reported mindset beliefs, their attitudes towards efforts and failure and their views on academic challenges.
  • Immediately after the intervention, students were more likely to attempt more challenging academic tasks.
  • Pupils who were lower performing at pre-test benefited more than their higher-performing peers.

Source: Using a growth mindset intervention to help ninth-graders: An independent evaluation of the National Study of Leaning Mindsets (November 2019), MDRC

An evaluation of PACE Center for Girls

Megan Millenky and colleagues from MDRC have released a new reporton an evaluation of PACE Center for Girls. PACE, a Florida-based organisation, provides academic and social services to at-risk middle and high school girls. According to the report, PACE operates daily, year-round; on a typical day, girls attend academic classes and receive additional support such as individual counselling, academic advice, and referrals to other services.

The research team used a random assignment design to evaluate the impact of PACE. From August 2013 to November 2015, a sample of 1,125 girls were enrolled in the study (673 in the programme group, and 452 in the control group). Data sources included administrative records, a survey, and interviews.

Key findings from the study were as follows:

  • The programme group received more academic and social services — and received them more often from a professional source — than the control group.
  • Over a one-year period, PACE increased school enrolment and attendance for the girls it served, compared with the control group. Girls in the programme group were also more likely to be “on track” academically than those in the control group.
  • Girls in both the programme and control groups appeared goal-orientated and hopeful about their futures and reported relatively low levels of risky behaviour one year after study enrolment.
  • The cost of PACE’s holistic package of services is, on average, $10,400 per pupil more than the cost of the services received by control group members through academic and social services provided in the community. The additional cost is largely driven by PACE’s extensive social services; the cost of academic services is similar to those of Florida public schools.

The authors note that further follow-up research would be necessary to see whether PACE affects longer-term academic and delinquency outcomes and to complete a full benefit-cost analysis.

Source: Focusing on girls’ futures: Results from the evaluation of PACE Center for Girls (January 2019), MDRC

Helping youth transition to adulthood

A new report by Cynthia Miller and colleagues at MDRC examines four-year results from a national evaluation of YouthBuild. The report describes YouthBuild as a programme that attempts to improve prospects for less-educated young people, serving over 10,000 individuals each year at over 250 organisations nationwide. Each organisation provides hands-on, construction-related or other vocational training, educational services, case management, counselling, service to the community, and leadership-development opportunities, to low-income young people ages 16 to 24 who did not complete secondary school.

MDRC evaluated the YouthBuild programme using a randomised controlled trial. Study participants were either invited to enrol in YouthBuild (the intervention group) or referred to other services in the community (the control group). A total of 75 programmes across the country were included, with a sample of nearly 4,000 young people who enrolled in the study between 2011 and 2013. Data included in-person observations, survey data, and administrative records.

Key findings of the evaluation included:

  • YouthBuild increased the receipt of high school equivalency credentials.
  • YouthBuild increased enrolment in college, largely during the first two years. Very few young people had earned a degree after four years, and the programme had a very small effect on degree receipt.
  • YouthBuild increased survey-reported employment rates, wages, and earnings, but did not increase employment as measured with employer-provided administrative records, which might not include certain kinds of employment and other types of informal work.
  • YouthBuild increased civic engagement, largely via participation in YouthBuild services. It had no effects on other measures of positive youth development.

Overall, the authors say the effects observed through four years indicate that the programme provides a starting point for redirecting otherwise disconnected young people.

Source: Laying a foundation: Four-year results from the national YouthBuild evaluation (May 2018), MDRC

Reviewing the evidence on career and technical education

A new report by Rachel Rosen and colleagues at MDRC reviews the available research evidence supporting various types of career and technical education (CTE) programmes, examining both the amount of evidence available in each area and its level of rigour. The report details several CTE programme types (eg, instruction and training, apprenticeships and readiness skills training) and provides a literature review of the available evidence to support each programme type.

Key findings were as follows:

  • The most evidence exists for CTE course work and training. In that area, there are multiple studies suggesting that participation in CTE can improve pupils’ outcomes. In addition, multiple studies found that career-related certificates and associate’s degrees are linked to increased wages.
  • Several career pathway models, particularly career academies and early college high schools, are also supported by strong, rigorous studies that provide evidence of positive benefits for pupils.
  • The evidence for other models and for individual programme components is weaker. The authors suggest that these models and components probably need to be evaluated further.

Source: Career and technical education current policy, prominent programs, and evidence (September 2018), MDRC

Evidence and policy

In a review of important 2017 releases, MDRC recently referenced a memo to policymakers with recommendations for increasing research use and applying evidence to all policy decisions, both educational and otherwise.

Recommendations included:

  • Programmes and policies should be independently evaluated. To ensure high-quality evaluations, they should be directly relevant to policy, free of political or other influences and credible to subjects and consumers.
  • The government should provide incentives for programmes to apply evidence results to improve their performance.
  • Utilise a tiered evidence strategy, such as is used in the Every Student Succeeds Act, to set clear guidelines for standards.
  • Existing funding sources should be applied to generate evidence. A 1% set-aside was recommended.
  • Federal and state agencies should be allowed to access and share their data for evaluation purposes.

Source: Putting evidence at the heart of making policy (February 2017), MDRC

Evidence on the long-term effects of home visiting programmes

Children from low-income families are more likely than those from higher-income families to have poor social, emotional, cognitive, behavioural and health outcomes. One approach that has helped parents and their young children is home visiting, which provides information, resources and support to expectant parents and families with young children.

This MDRC brief summarises prior evidence on the effects of four evidence-based models of home visiting using information from seven studies of families with children aged 5- to 21-years-old. Specifically, the brief looks at what the effects of home visiting are for families as children get older, and how the monetary benefits of home visiting compare with their costs.

  • The key findings of the report include:
    Evidence-based home visiting has improved outcomes for parents and children across a wide range of child ages, outcome areas, and national models.
  • Evidence-based home visiting appears to be cost-effective in the long term.
  • The largest benefits from evidence-based home visiting come through reduced spending on government programmes and increased individual earnings.

The information in this brief will inform the design of a study to assess the long-term effects of home visiting. It will suggest where this long-term follow-up study can seek to replicate prior results, where it can try to fill gaps in current knowledge, and which outcomes are important to measure in order to assess the benefits and costs of home visiting.

Source: Evidence on the long-term effects of home visiting programs: Laying the groundwork for long- term follow-up in the mother and infant home visiting program evaluation (MIHOPE). (September 2017), OPRE Report 2017-73, MDRC