DLL por favor!

Projections estimate that by 2030, children with English as an Additional Language will comprise 25% of US students, 77% of whom will speak Spanish. Yet there is little evidence-based research addressing what works in literacy for the Spanish-speaking population. Trisha Borman and colleagues recently reported the first-year results of a randomised study of Descubriendo La Lectura (DLL), an individually administered Spanish literacy programme for first graders (Year 2) struggling with literacy in their native Spanish, examining its effects on both Spanish and English literacy.

DLL incorporates the research-proven practices of 1:1 tutoring, using a student’s native language to improve their second language, intervening early (in first grade), using data to track and guide progress, professional development, research-proven practices, and teacher collaboration.

Subjects were first-grade students in 22 schools in 3 states, statistically matched at baseline and randomly assigned to receive DLL either in the 2016 school year (experimental group, n=78), or the 2017 school year (delayed treatment control group, n=74). Students qualified for DLL if they spoke Spanish at home and scored below 25% on the IdO (a test that assesses literacy). Students were also pretested and post-tested using the Iowa Test of Basic Skills (ITBS) and Logramos (the Spanish equivalent of the ITBS). The experimental group received 30-minute lessons daily with students and teachers meeting 1:1, progressing at their own pace until they qualified to leave the programme and were post-tested. This process took 12-20 weeks, depending on the student.

Results favoured the DLL group, with statistically significant effect sizes on the Spanish Logramos averaging +0.55 (p<.001). On the English ITBS, the mean effect size was +0.17, which was not significant.  There were larger positive outcomes on measures made by the developers. The programme will continue to be studied in two subsequent cohorts.

Source: Addressing Literacy Needs of Struggling Spanish-Speaking First Graders: First-Year Results From a National Randomized Controlled Trial of Descubriendo la Lectura (July 2019) AERA Open

Nudging proves difficult to scale up

Small-scale trials of “nudge” approaches, where, for example, students are encouraged via a series of text messages to apply for financial support for college, have shown positive results. These trials have involved a few thousand students, but could the approach be scaled up to state or national level?

A working paper from the National Bureau of Economic Research reports the results of two large randomised controlled trials that collectively reached over 800,000 students in the US. Kelli A. Bird and colleagues tested the impact of a national and state-level campaign to encourage students (average age 18.6 years) to apply for financial aid for college, with multiple treatment arms to investigate different potential mechanisms and approaches.

The trials found no impacts on financial aid receipt or college enrollment overall or for any student subgroups. There was no evidence that different approaches to message framing, delivery, or timing, or offers of one-on-one advice, affected the efficacy of any of the campaigns.

The researchers suggest three reasons why the scaled-up approach may not have been effective:

  • Most previous studies involved a local partner with closer connections to and knowledge of the students. Local partners may know something important about their students and students may react differently to messages from organisations in their communities.
  • A global scale-up results in messaging content that is more generic and less personalised to students.
  • The students in this study may have had better information about the financial support available to them than previous cohorts, so the intervention made less impact.

Source: Nudging at Scale: Experimental Evidence from FAFSA Completion Campaigns (August 2019) NBER Working Paper No. 26158

The impact of refugees on local children

Following an earthquake in Haiti in 2010, more than 4,000 refugee children entered Florida’s school system, most of them in four school districts. What impact did their arrival have on existing students in those schools?

In an article in the Journal of Labor Economics, David Figlio and Umut Özek draw on student-level administrative data from Florida that provides detailed information on all students enrolled in a public school between 2002–3 and 2011–12. These include reading scores on the Florida Comprehensive Assessment Test (FCAT) and maths scores for students between grades 3 and 10 (Years 4-11), as well as a wealth of student characteristics.

Using data from the year before the earthquake, and the two years after, they found that there was a neutral to positive impact on the children already in the schools. In particular, in the spring of 2010, each percentage point increase in refugee concentration was associated with 0.6%–0.7% of a standard deviation increase in reading test scores, 0.3%–0.4% of a standard deviation increase in maths test scores, and 0.2–0.6 percentage points fewer disciplinary incidents. They found that these results — neutral or slightly positive impacts — were consistent across sub-groups of students (by age, place of birth, race/ethnicity, spoken language, and socio-economic status). The researchers caution that these results are not necessarily generalisable, since, for example, Florida has one of the most equitable school funding distributions in the US and, with a substantial existing population of Haitian immigrants, may have systems and approaches in place that helped to mitigate any potential impact on local students.

Source: Unwelcome Guests? The Effects of Refugees on the Educational Outcomes of Incumbent Students (July 2019) Journal of Labor Economics

Does Girls Active lead to active girls?

A study published in the International Journal of Behavioral Nutrition and Physical Activity looks at the results of an intervention aimed at improving the activity levels of adolescent girls.

The randomised controlled trial by Deirdre Harrington and colleagues took place in 20 secondary schools in Leicester. Ten schools received Girls Active and ten schools continued with usual practice. Developed by the Youth Sport Trust, Girls Active is focused on providing a support framework to schools to review their physical activity, sport, and PE teaching to ensure they are relevant and attractive to all adolescent girls, but with a particular focus on 11–14 year olds. The programme includes a range of resources for schools, including self-evaluation, training, mentoring, and funding for developing school capacity.

In total, 1,752 girls aged 11-14 participated. The primary outcome measure (at baseline, 7 months, and 14 months) was moderate to vigorous physical activity (MVPA), as recorded on wrist-worn accelerometers. Secondary outcomes included overall physical activity, light physical activity, sedentary time, body composition, and psychosocial outcomes. The results showed small improvements in MVPA in comparison with control schools after 7 months, but none after 14 months. Subgroup analysis showed that the intervention was effective at 14 months in larger schools, but caused an MVPA decrease in smaller schools. There was no pattern in the secondary outcomes, and any differences were slight.

Source: Effectiveness of the ‘Girls Active’ school-based physical activity programme: A cluster randomised controlled trial (April 2018) International Journal of Behavioral Nutrition and Physical Activity

Results of a large randomised controlled trial of growth mindset

In a recent issue of Best Evidence in Brief, we reported on an English study of a growth mindset intervention, which found no evidence that it led to additional progress in literacy or numeracy. Now a US randomised controlled trial published in the journal Nature has found that a short, online, self-administered growth mindset intervention may improve achievement among lower-achieving students and increase overall enrollment in advanced math courses.

The study, conducted by David Yeager and colleagues, was the largest ever randomised controlled trial of growth mindset in US schools, with 12,000 ninth graders (Year 10) in 65 schools involved.

Students were individually randomised to either a control or intervention group. The intervention group was asked to complete two 25-minute online courses, taken three weeks apart. Students were given information about how the brain works and the latest research on growth mindset – then they completed activities such as explaining what they had learned from the course to students in the year below. Students in the control group were given a similar programme with information on how the brain worked, but no information on growth mindset.

Following the intervention, students’ grade point average (GPA) in their core classes of maths, science, English, and social studies, were collected. (In the US, grade point averages run from 4.0, which is an A, to 1.0, which is a D. There is no E grade. The score below D is an F.)

The study found that:

  • GPA scores for lower-achieving students in the intervention group rose by 0.1 points relative to peers in the control group (effect size = +0.11).
  • The proportion of lower-achieving students with D or F averages dropped by 5%.

Both higher- and lower-achieving students were more likely to take an advanced maths class in 10th grade (Year 11) – meaning enrollment in these courses rose from 33% to 36% in the 41 schools that shared this data.

Source: A national experiment reveals where a growth mindset improves achievement (August 2019) Nature.