The effects of financial incentives on standardised testing

Can financial incentives improve outcomes on standardised tests? In an unpublished paper, John List, Jeffrey Livingston and Susanne Neckermann examined whether motivating pupils to gain the knowledge needed to succeed on the International Student Admissions Test (ISAT) would result in higher test scores.

Subjects were 3rd–8th grade (Year 4–Year 9) pupils (n=226E, 226C) who were judged to be at risk of not passing the ISAT and were receiving tutoring to improve their scores in nine elementary and middle schools in a suburb outside of Chicago. Using a system designed by the ISAT’s developers, authors created short benchmark tests designed to measure the knowledge needed to be successful on the ISAT. Pupils, their parents and their tutors were informed that they would receive up to $90 if pupils’ performance improved on these benchmark tests and if they met other academic and behavioural goals.

Results showed that pupils demonstrated significant gains on the incentivised benchmark testing compared to control pupils (effect size =+0.29), indicating they had the knowledge to pass the ISAT. However, they did not demonstrate significant gains compared to controls for the ISAT itself (effect size=+0.05), for which they did not receive any financial incentive. This was true regardless of whether incentives were provided immediately or were delayed. The authors conclude that pupils may not be motivated to show what they know on standardised testing that holds no personal stake for them.

Similar results were found in a review of international experiments evaluating financial incentives in education by Robert Slavin, posted on The Best Evidence Encyclopedia. The emerging consensus is that financial incentives can have an impact on easily counted outcomes for which the incentives were directly given (such as attendance), but not on more general outcomes that should flow from the incentivised outcomes (such as achievement).

Source:  Do students show what they know on standardized tests? working papers (2016),  from the selected works of Jeffrey A Livingston, Bentley University

Do teacher observations make any difference to pupil performance?

An evaluation published by the Education Endowment Foundation (EEF) has found that introducing more frequent and structured lesson observations – where teachers observe their colleagues and give them feedback – made no difference to pupils’ GCSE maths and English results.

A randomised controlled trial of the whole-school intervention Teacher Observation was conducted in 82 secondary schools in England, which had high proportions of pupils who had ever been eligible for free school meals. In total, the study involved 14,163 pupils – 7,507 pupils (41 schools) in the intervention, and 6,656 pupils (41 schools) in the control.

Maths and English teachers in the intervention schools were asked to take part in at least six structured 20-minute peer observations over a two-year period (with a suggested number of between 12 and 24). Teachers rated each other on specific elements of a lesson, such as how well they managed behaviour, engaged pupils in learning, or used discussion techniques.

The evaluation, which was conducted by a team from the National Foundation for Educational Research (NFER), found that Teacher Observation had no impact on pupils’ GCSE English and maths scores compared to those of pupils in control schools (effect size = -0.01).

Source: Teacher Observation: Evaluation report and executive summary (November 2017), Education Endowment Foundation

New research on dual-language immersion programmes

A new research brief by Jennifer L Steele and colleagues, published by the RAND Corporation, presents new research on dual-language immersion (DLI) programmes. These programmes provide both native English speakers and children learning English as an additional language (EAL) with general academic teaching in two languages from kindergarten (Year 1) onwards.

In partnership with the American Councils on International Education and the Portland Public Schools in Oregon (PPS), the authors conducted a random-assignment study of DLI education. The goal was to estimate the causal effects of the district’s DLI programmes on pupil performance over time in reading, mathematics and science, and on EAL pupils’ reclassification as English proficient.

PBS allocates immersion slots using a random-assignment lottery process for those who apply to the programmes. The study focused on 1,625 DLI lottery applicants in the kindergarten cohorts from 2004–2005 to 2010–2011. Pupil achievement was tracked until 2013–2014.

Key findings of the study were as follows:

  • PPS pupils randomly assigned to dual-language immersion programmes outperformed their peers on state reading tests by 13% of a standard deviation in grade 5 (Year 6) and by 22% of a standard deviation in grade 8 (Year 9).
  • Immersion-assigned pupils did not show statistically significant benefits or deficits in terms of mathematics or science performance.
  • There were no clear differences in the effects of dual-language immersion according to pupils’ native language.
  • EAL pupils assigned to dual-language immersion were more likely than their peers to be classified as English proficient by grade 6 (Year 7). This effect was mostly attributed to EAL pupils whose native language was the same as one of the two languages taught.

Source: Dual-language immersion programs raise student achievement in English (2017), RAND Corporation Research Brief, RB-9903

Praise to raise self-esteem?

An observational longitudinal study published in Child Development tests whether receiving overly positive, inflated praise from a parent eventually fosters low self-esteem and even narcissism, rather than raising it as might be expected.

The study involved 120 children recruited from schools in the Netherlands and their parents. Children were aged 7 to 11. Children completed questionnaires in school at four six-month intervals, and levels of narcissism and self-esteem were measured using the Childhood Narcissism scale and the Global Self-Worth Subscale of the Self-Perception Profile for Children.

Eddie Brummelman and colleagues found that children with lower levels of self-esteem at the beginning of the study received more inflated praise from parents, which in turn led to lower self-esteem at the later test points. Inflated praise also predicted higher narcissism over time, but only in children with high initial levels of self-esteem.

Source: When parents’ praise inflates, children’s self-esteem deflates (November 2017), Child Development, Volume 88, Issue 6 doi:10.1111/cdev.12936

Montessori preschool boosts academic results and reduces income achievement gap

A longitudinal study published in Frontiers in Psychology examined how children in Montessori schools changed over three years compared with children in other pre-school settings.

The Montessori model involves both child-directed, freely-chosen activity and academic content. Angeline Lillard and colleagues compared educational outcomes for children allocated places by a random lottery to either Montessori pre-schools (n=70) or non-Montessori pre-school settings (n=71) in Connecticut, US. The research team carried out a variety of assessments with the children over a three-year period, from when the children were three until they were six.

The researchers found that over time children in Montessori pre-schools performed better on measures of academic achievement (Woodcock–Johnson IIIR Tests of Achievement effect size = +0.41) and social understanding, while enjoying their school work more, than those in conventional pre-school settings. They also found that in Montessori classrooms, children from low-income families, who typically don’t perform as well in school, showed similar academic performance as children from higher-income families. Children with low executive function similarly performed as well as those with high executive function.

The findings, they suggest, indicate that well-implemented Montessori education could be a way to help disadvantaged children to achieve their academic potential.

Source: Montessori Preschool Elevates and Equalizes Child Outcomes: A Longitudinal Study (October 2017), Frontiers in Psychology

Can teaching be simplified?

Technology that simplifies teaching by providing teachers with “off-the-shelf” lessons may increase pupil achievement, particularly if the teachers are supported in using them, according to a working paper published by the National Bureau of Economic Research (NBER).

The study, conducted by Kirabo Jackson and Alexey Makarin, provided middle school maths teachers with online lessons from the Mathalicious curriculum – an inquiry-based maths curriculum for grades 6 to 12 (Years 7 to 13) grounded in real-world topics and situations. Maths teachers from 170 schools across Virginia, US, took part and were randomly assigned to one of three conditions: full treatment (online access to Mathalicious lessons along with supports to promote their use); lesson-only (online access to Mathalicious lessons only); or control (business-as-usual).

While positive effects on pupil achievement in maths were seen in both the full treatment and lesson-only conditions, results were only significant for the full-treatment group. Providing teachers with online access to the lessons along with supports to promote their use increased pupil maths achievement by an effect size of +0.09 (p<.05). Test scores for pupils in the lesson-only group were non-significantly higher than those of the control group (effect size = +0.04).

Source: Can Online Off-The-Shelf Lessons Improve Student Outcomes? Evidence from A Field Experiment (January 2017), National Bureau of Economic Research (NBER), NBER Working Paper No. 22398