Examining the evidence on Learning Accounts

Social Programs That Work has released a new evidence summary on Learning Accounts, a demonstration programme in New Brunswick, Canada that provided up to $8,400 in conditional financial aid for post-secondary education to low-income 10th grade (Year 11) pupils. The pupils had to meet certain benchmarks (ie, completion of 10th, 11th, and 12th grade (Years 11-13)) to receive the funding.

The programme was evaluated through a randomised controlled trial with a sample of 1,145 low-income 10th graders in 30 high schools in New Brunswick, Canada. Within each school, the low-income pupils were randomly assigned to a group that was offered participation in the Learning Accounts programme, or to a control group that received usual school services. Survey data was used to measure high school graduation rates, and administrative data was used to examine later graduation from college.

According to the evidence report, over the 10 years following random assignment, the programme produced a 6.5 percentage point increase in the high school graduation rate, and 6.8 percentage point increase in the rate of post-secondary completion.

Source: Learning Accounts (September 2019), Social Programs That Work

Improving the maths and reading skills of children in foster care

A study published in Oxford Review of Education evaluates the effects of TutorBright tutoring on the reading and maths skills of children in family foster care. TutorBright uses one-to-one, at-home tutoring with detailed instructor’s manuals and customised pupil workbooks. Children receive two one-hour tutoring sessions per week, on designated days of the week, for up to 50 hours of tutoring. Children in the waiting list control group were asked to continue with their schooling as usual and not seek additional tutoring or academic support during the school year, and were then offered the tutoring intervention at the end of the school year. TutorBright tutors all had experience with teaching or mentoring, and an undergraduate or master’s degree (completed or in progress).

For the randomised controlled trial, conducted by Andrea J Hickey and Robert J Flynn, child welfare workers nominated foster care children in Ontario, Canada, who met the following criteria: enrolled in grades 1–11 (Years 2–12), fluent in English, currently living in a foster-family setting, and judged likely to remain in care for the duration of the study. Thirty-four children were randomly assigned to tutoring, and 36 to a waiting-list control condition.

The results suggest that the tutored children made greater gains than those in the control group in reading fluency (effect size = +0.16), reading comprehension (ES = +0.34) and maths calculation (ES = +0.39).

Source: Effects of the TutorBright tutoring programme on the reading and mathematics skills of children in foster care: a randomised controlled trial (July 2019), Oxford Review of Education, 45:4

Improving the language and communication of secondary school children with language difficulties

A research report published in the International Journal of Language and Communication Disorders investigates the effectiveness of teaching assistant (TA)-delivered narrative and vocabulary interventions to secondary school children with language difficulties.

Researchers at City University of London and University of Oxford conducted a randomised controlled trial in two outer London boroughs. Across 21 schools, 358 Year 7 underperforming pupils (mean age = 12.8 years) were recruited, and randomised to four groups within each school: vocabulary intervention, narrative intervention, combined narrative and vocabulary intervention, and delayed waiting control group. The narrative programme focused on the understanding and telling of stories, using a story structure to support story generation. Pupils were introduced to different types of stories (fictional, non‐fictional, scripts) and narrative genres. The vocabulary programme focused on developing key concepts and vocabulary items relevant to the curriculum (eg, nutrition) and age-appropriate (eg, careers). A variety of tasks including word associations, categorisation, mind‐mapping and word‐building were used to reinforce word learning.

The language and communication programmes (narrative, vocabulary, and combined narrative and vocabulary) were delivered by TAs in the classroom, three times per week, for 45–60 min each, over six weeks, totalling 18 sessions. Assessments were conducted pre- and post-intervention.

Overall, pupils in the intervention groups made greater improvements on standardised measures of narrative (effect size = +0.296), but not vocabulary skills, compared with control group children.

Source: Improving storytelling and vocabulary in secondary school students with language disorder: a randomized controlled trial (March 2019), International Journal of Language and Communication Disorders, 54:4

Results of a large randomised controlled trial of growth mindset

In a recent issue of Best Evidence in Brief, we reported on an English study of a growth mindset intervention, which found no evidence that it led to additional progress in literacy or numeracy. Now a US randomised controlled trial published in the journal Nature has found that a short, online, self-administered growth mindset intervention may improve achievement among lower-achieving students and increase overall enrollment in advanced math courses.

The study, conducted by David Yeager and colleagues, was the largest ever randomised controlled trial of growth mindset in US schools, with 12,000 ninth graders (Year 10) in 65 schools involved.

Students were individually randomised to either a control or intervention group. The intervention group was asked to complete two 25-minute online courses, taken three weeks apart. Students were given information about how the brain works and the latest research on growth mindset – then they completed activities such as explaining what they had learned from the course to students in the year below. Students in the control group were given a similar programme with information on how the brain worked, but no information on growth mindset.

Following the intervention, students’ grade point average (GPA) in their core classes of maths, science, English, and social studies, were collected. (In the US, grade point averages run from 4.0, which is an A, to 1.0, which is a D. There is no E grade. The score below D is an F.)

The study found that:

  • GPA scores for lower-achieving students in the intervention group rose by 0.1 points relative to peers in the control group (effect size = +0.11).
  • The proportion of lower-achieving students with D or F averages dropped by 5%.

Both higher- and lower-achieving students were more likely to take an advanced maths class in 10th grade (Year 11) – meaning enrollment in these courses rose from 33% to 36% in the 41 schools that shared this data.

Source: A national experiment reveals where a growth mindset improves achievement (August 2019) Nature.

Using expressive writing to reduce test anxiety

Test anxiety can have negative impacts on pupils’ performance and psychological health. This study published in PLoS One examined whether expressive writing could be beneficial to alleviate test anxiety. Lujun Shen and colleagues conducted a randomised controlled trial among high school pupils in China who were facing The National Higher Education Entrance Examination (Gaokao), which is considered a crucial exam.

The study randomly selected 200 pupils (aged 16-17) from three high schools in Xinxiang city. Pupils were first assessed for eligibility. A sample of 75 pupils was recruited into the study for having a high level of test anxiety. Next, 38 of the pupils were allocated into an expressive writing group, and 37 of them were allocated to a control writing group. Pupils in the expressive writing group were instructed to write for 20 minutes about the positive emotions they had each day, consecutively for 30 days. Pupils in the control writing group were instructed to write about their daily activities consecutively for the same period of time.

Pupils were assessed using the Test Anxiety Scale (TAS) during the recruitment (late April), and after the end of the writing (early June). The study also analysed summaries of the writing manuscripts of the 38 expressive writing group pupils for qualitative data. The findings were as follows:

  • The expressive writing group scored significantly lower than the control writing group in the Test Anxiety Scale post-test.
  • There were no significant gender differences in the post-test TAS scores.
  • Qualitative analysis of the writing found more elements of positive emotion in the last ten days of expressive writing compared to the first ten days among the expressive writing group.

The authors suggest that expressive writing is an easy, inexpensive, and convenient method to cope with anxiety because it does not require a psychological counsellor nor a specific location.

Source: Benefits of expressive writing in reducing test anxiety: A randomized controlled trial in Chinese samples (February 2018), PLoS One, 5(13).

Helping students to graduate ASAP

A study published in American Economic Journal: Applied Economics presents the results from a randomised controlled trial of the City University of New York’s (CUNY) Accelerated Study in Associate Programs (ASAP) initiative on students’ academic progress and success. This latest paper considers the long-term impact of the programme (we covered the original study previously in Best Evidence in Brief).

The CUNY ASAP programme is a comprehensive three-year programme aimed at helping more students to graduate from community college more quickly than they otherwise would (in the US, community colleges provide higher education from the age of 18). It aims to remove the barriers to academic success often faced by low-income students and comprises the following components:

  • Students are required to attend college full time, take remedial courses early, and graduate in three years.
  • Each student is provided with a dedicated ASAP advisor.
  • Students receive a tuition waiver covering the difference between the financial aid provided and the cost of tuition and fees. They are also provided with free passes for public transport and free use of textbooks.
  • Students can enrol in courses with other ASAP students in convenient schedules.

The results of the study showed that ASAP had positive impacts on full-time enrolment and credit accumulation. It had an estimated 18 percentage point effect on three-year graduation rates, increased six-year graduation rates by an estimated 10 percentage points, and helped students to graduate more quickly than students in the control group.

Source: Supporting community college students from start to degree completion: Long-term evidence from a randomized trial of CUNY’s ASAP” (July 2019), American Economic Journal: Applied Economics, 11 (3).