What does the evidence say about technology use?

New educational technology programmes are being released faster than researchers can evaluate them. The National Bureau of Economic Research in the US has written a working paper, Education Technology: An Evidence-Based Review, which discusses the evidence to date on the use of technology in the classroom, with the goal of finding decision-relevant patterns.

Maya Escueta and colleagues compiled publicly available quantitative research that used either randomised controlled trials or regression discontinuity designs (where pupils qualify for inclusion in a programme based on a cut-off score at pre-test). All studies had to examine the effects of an ed-tech intervention on any education-related outcome. Therefore, the paper included not only the areas of technology access, computer-assisted learning and online courses, but also the less-often-studied technology-based behavioural interventions.

Authors found that:

  • Access to technology may or may not improve academic achievement at the K-12 level (Years 1–13), but does have a positive impact on the academic achievement of higher education students (ES=+0.14).
  • Computer-assisted learning, when equipped with personalisation features, was an effective strategy, especially in maths.
  • Behavioural intervention software, such as text-message reminders or e-messages instructing parents how to practise reading with their children, showed positive effects at all levels of education, and was also a cost-effective approach. Four main uses for behavioural intervention software emerged: encouraging parental involvement in early learning activities, communication between the school and parents, successfully transitioning into and through higher education, and creating mindset interventions. Research is recommended to determine the areas where behavioural intervention software is most impactful.
  • Online learning courses had the least amount of research to examine and showed the least promise of the four areas. However, when online courses were accompanied by in-person teaching, the effect sizes increased to scores comparable to fully in-person courses.

 Source: Education technology: an evidence-based review (August 2017), National Bureau of Economic Research (NBER) Working Paper No. 23744

Does increasing time spent in class improve pupil performance?

A study by Huebener and colleagues examined whether increasing the amount of time pupils spend in the classroom affects their performance.

The authors used PISA scores to analyse the effect of increasing the time spent in class by two hours per week over a five-year period for ninth-grade students in Germany (average age = 15 years old). During the additional classroom time, pupils were taught new content.

Their findings indicate that while increasing the time spent in class did improve pupils’ average performance, effect sizes were small. The increase in lesson time was shown to increase average PISA test scores in reading, maths and science (effect size between +0.04 and +0.06 for one additional hour per week). However, these results differ according to pupil ability, with a widening gap in performance between low- and high-performing pupils. The researchers suggest this is because the additional teaching time was used to teach new content, and that lower-performing pupils may not be able to cope with this additional content. They recommend that when policymakers consider adding additional classroom time, they consider how this time is spent. Different pupils have different learning needs, so the content of the extra lessons, rather than the time, is more important to improving pupil performance.

Source: Increased instruction hours and the widening gap in student performance (August 2017), Labour Economics, Volume 47

Talking in class boosts progress in maths, science and English

An intervention that trained teachers to improve and monitor the quality of classroom talk had a positive impact on primary pupils’ test scores in English, maths and science, a report published by the Education Endowment Foundation (EEF) reveals.

Seventy-six primary schools with higher-than-average proportions of disadvantaged pupils took part in a randomised control trial of the Dialogic Teaching intervention, which is designed to improve the quality of classroom talk as a means of increasing pupils’ engagement, learning and achievement. Year 5 teachers in 38 schools (2,493 pupils), and a teacher mentor from each school, received resources and training from the delivery team and then implemented the intervention over the course of the autumn and spring terms in the 2015/16 school year. A control group of 38 schools (2,466 pupils) continued with business as usual. Following the intervention, pupils were tested in English, maths and science.

The results showed that pupils in the intervention schools did better in the main outcome measures of English (effect size = +0.16), science (+0.12), and maths (+0.09) when compared with pupils in the control schools who didn’t receive the intervention. For pupils who received free school meals, the intervention had a higher impact on maths (+0.16), but around the same for English (+0.12) and science (+0.11). Teachers reported positive effects on pupil engagement and confidence, and on the whole the intervention was highly regarded by participating schools. However, some teachers felt that it would take longer than two terms to fully embed a Dialogic Teaching approach in their classrooms.

The Dialogic Teaching intervention was developed by the Cambridge Primary Review Trust and the University of York. This University of York news story has more.

Source: Dialogic teaching: evaluation report and executive summary (July 2017), Education Endowment Foundation

The effects of head teacher training on pupil achievement

An NBER Working Paper examines the impact of implementing management training for head teachers on pupil achievement. The management training focused on lesson planning, data-driven teaching and teacher observation and coaching (approximately 300 hours over two years). Using a school-level randomised experiment, 58 schools in Houston, Texas, were randomised to receive either the training intervention or to serve as a business-as-usual control group.

The study found that offering management training to head teachers led to increased test scores across low-stakes tests in a range of subjects in year one (effect size = +0.19). For high-stakes test scores in maths and reading, the effect size was lower (+0.10). However, the training intervention had no impact on high-stakes tests in year two.

The training was most beneficial for head teachers who were less experienced, had better maths skills, had more internal locus of control, had higher levels of “grit” and remained in the school for both years of the study.

The intervention showed most impact on teachers in the schools who were more experienced and more educated. The intervention showed most impact for pupils who were new to the school, white or Hispanic and economically well-off.

Source: Management and student achievement: Evidence from a randomized field experiment (May 2017), NBER Working Paper No. 23437, National Bureau of Economic Research


Unhealthy school lunches nibble away academic performance

A National Bureau of Economic Research (NBER) Working Paper looks at the effect of offering healthier school lunches on end-of-year test scores for pupils in California.

Michael L Anderson and colleagues analysed data collected over a five-year period (academic years 2008/2009 to 2012/2013) from around 9,700 schools that reported test scores on California’s Standardized Testing and Reporting exam, a state-wide test given to all pupils in grades 2 to 11 (Years 3 to 13). In order to determine the link between food quality and pupil achievement, they also collected data from the California Department of Education on school districts’ meal vendors for the same time span. Over that five-year period, about 12% of California schools contracted an outside company to provide lunch for at least one school year. The nutritional value of the school lunch menus was analysed by nutritionists at the Nutrition Policy Institute using the Healthy Eating Index.

The results of the study found that in years when schools offered a healthy lunch menu, pupil test scores were on average higher (effect size +0.03 to +0.04). In addition, test scores for pupils who qualified for reduced-price or free school lunches, (and therefore more likely to eat the healthy lunches), increased by about 40% in comparison to pupils who didn’t receive free school lunches.  The positive effect of healthy lunches on academic achievement persisted for the duration of a long-term contract.

Source: School lunch quality and academic performance (March 2017), NBER Working Paper No. 23218, National Bureau of Economic Research (NBER)

Children who enjoy reading over three years ahead in the classroom

Research published by the National Literacy Trust highlights the link between enjoyment of reading and achievement, with children who enjoy reading more likely to do better at reading – over three years ahead in the classroom – of their peers who don’t enjoy it.

The findings are based on data from 42,406 children aged 8 to 18 who participated in a National Literacy Trust survey at the end of 2016. At age 10, children who enjoy reading have a reading age 1.3 years higher than their peers who don’t enjoy reading, rising to 2.1 years for 12-year-olds. At age 14, children who enjoy reading have an average reading age of 15.3 years, while those who don’t enjoy reading have an average reading age of just 12 years, a difference of 3.3 years.

The survey also indicates that three-quarters (78%) of UK primary school children enjoy reading, with girls more likely to enjoy reading than boys. Overall, 64.9% of girls enjoying reading either very much or quite a lot compared with 52.4% of boys, and this gap increases with age. At ages 8 to 11, 82.8% of girls and 72.4% of boys said they enjoyed reading. By ages 14 to 16, this figure has dropped to 53.3% of girls and 35.7% of boys reporting that they enjoy reading.

Source: Celebrating reading for enjoyment: findings from our annual literacy survey 2016 (June 2017), National Literacy Trust