Nudging proves difficult to scale up

Small-scale trials of “nudge” approaches, where, for example, students are encouraged via a series of text messages to apply for financial support for college, have shown positive results. These trials have involved a few thousand students, but could the approach be scaled up to state or national level?

A working paper from the National Bureau of Economic Research reports the results of two large randomised controlled trials that collectively reached over 800,000 students in the US. Kelli A. Bird and colleagues tested the impact of a national and state-level campaign to encourage students (average age 18.6 years) to apply for financial aid for college, with multiple treatment arms to investigate different potential mechanisms and approaches.

The trials found no impacts on financial aid receipt or college enrollment overall or for any student subgroups. There was no evidence that different approaches to message framing, delivery, or timing, or offers of one-on-one advice, affected the efficacy of any of the campaigns.

The researchers suggest three reasons why the scaled-up approach may not have been effective:

  • Most previous studies involved a local partner with closer connections to and knowledge of the students. Local partners may know something important about their students and students may react differently to messages from organisations in their communities.
  • A global scale-up results in messaging content that is more generic and less personalised to students.
  • The students in this study may have had better information about the financial support available to them than previous cohorts, so the intervention made less impact.

Source: Nudging at Scale: Experimental Evidence from FAFSA Completion Campaigns (August 2019) NBER Working Paper No. 26158

Text messages add up to improvement

A new study by the Education Endowment Foundation has found that text messages sent to parents increased maths achievement and lowered absenteeism.

The Parent Engagement Project (PEP) was designed to raise achievement by encouraging parents to engage with their children’s learning. Parents were sent regular texts about upcoming tests, whether homework had been submitted, and what their children were learning. On average, each parent received 30 texts during the academic year. The cluster randomised controlled trial involved 15,697 students in Years 7, 9, and 11 from 36 English secondary schools during the 2014/15 school year.

Children who received the intervention showed a small, significant, positive impact (an effect size of +0.033) on their maths achievement. There was also a significant reduction in their absenteeism (-0.054), even though none of the texts were about attendance.

Although the effects of the intervention were small, the programme is inexpensive and relatively easy for schools to implement. Parents were generally satisfied with the frequency, timing, and content of the texts. A teacher survey and interviews showed that schools were enthusiastic about the programme and liked its immediacy and low cost.

Source: Texting Parents (2016), Education Endowment Foundation

Texting parents helped with early literacy

A study of a programme that sent literacy-related advice via text messages to parents of preschool children showed that it improved both the parents’ literacy behaviour and the children’s early literacy.

READY4K! is an eight-month-long text messaging programme for parents of preschool children. Parents receive texts that cover literacy skills, encourage them to participate, and provide follow-up tips. In the study by the National Bureau of Economic Research, 519 parents in California were randomly assigned to receive the programme or a series of “placebo” texts (eg, about school enrollment) during the 2013-14 school year.

The texts increased the frequency with which parents read books to children and other literacy activities (effects up to 0.35 standard deviations higher). According to teachers, texted parents asked more questions about their child’s learning (up to 0.19 standard deviations higher) than parents who were in the control group and their children performed better on early measures of literacy – lower-case alphabet knowledge and letter sounds (up to 0.34 standard deviations higher).

The authors say that the widespread use, low cost, and ease of scalability of text messaging make it an attractive approach for supporting parenting practices.

Source: One Step at a Time: The Effects of an Early Literacy Text Messaging Program for Parents of Preschoolers (2014), NBER

Text messages change perceptions, but not behaviour

An experiment in Oklahoma examined the effect of giving pupils mobile phones, and then texting them messages to encourage them to participate at school. Almost 2,000 young people (aged 11-13) took part, all in schools with high percentages of children eligible for free or reduced-price lunches. The pupils were randomly allocated to four different groups – two groups received daily text messages, another received credits for reading books, and the last was a control group. The text messages exhorted the value of schooling, for example, “Each year high school students make $21,023. College graduates make $58,613. Do the math.”

Students changed their views about the value of education, and reported that they put in more effort at school. However, it made no difference to their actual attendance or behaviour at school, nor to their achievement. The authors suggest that this may be because the benefits are so far in the future that it is not worth the pupils expending effort now, or because they lack the self-control to commit to studying or going to class.

Source: Information and Student Achievement: Evidence from a Cellular Phone Experiment (2013), Harvard.

Text messaging does not affect children’s grammatical development

Researchers from Coventry University carried out a longitudinal study to investigate whether “text speak” had any detrimental impact on grammatical development and other related literacy and language skills over the course of a year. They assessed the spelling, grammar, understanding of English, and IQ of three groups of children and young people (83 primary school children, 78 secondary school children, and 49 undergraduates), and compared those skills with a sample of their text messages.

There was no evidence of any significant relationships between poor grammar in text messages and their understanding of written or spoken grammar. For the primary school children, there was an association between punctuation errors in text messages and spelling ability. Children who made fewer punctuation errors when texting tended to be better at spelling and quicker to process writing than those who made more errors in their text messages.

For the undergraduate group, there was some evidence of a link between punctuation errors in text messages and the spelling ability and grammatical understanding of participants. However, this link was weak, and researchers concluded that it was probably related to children’s IQ score.

Source: Text messaging and grammatical development (2012), Nuffield Foundation