Is personalised learning effective?

A new research brief by John F Pane and colleagues at the RAND Corporation asks the question: “Does personalised learning improve pupil learning more than other educational approaches?” As part of their report, the authors present findings from an evaluation of personalised learning (PL) schools conducted by RAND Corporation researchers for the Bill and Melinda Gates Foundation.

The research team analysed maths and reading scores for approximately 5,500 pupils in 32 US schools that received funding from the Next Generation Learning Challenges (NGLC) initiative to support highly personalised approaches to learning. These schools took the Northwest Evaluation Association (NWEA) Measures of Academic Progress (MAP) mathematics and reading assessments for one academic year: autumn 2014 to spring 2015. The research team compared the achievement of pupils in PL schools with matched peers attending non-PL schools and national norms.

Key findings from the research brief include:

  • Early evidence suggests that PL can improve achievement for pupils, regardless of their starting level of achievement.
  • Benefits of PL may take some time to emerge. Analyses suggest that effects may be more positive after schools have experience implementing PL.
  • To date, the field lacks evidence on which practices are most effective or what policies must be in place to maximise the benefits.

The authors note that additional research is needed using more rigorous experimental studies.

Source: How Does Personalized Learning Affect Student Achievement? RAND Corporation, 2017.

A thirty-year look at studies on computer-assisted maths

During the past 30 years, thousands of articles have been written about technology’s effects on pupil achievement. In order to quantify technology’s effects on maths achievement, Jamaal Young at the University of Texas conducted a meta-analysis of all of the meta-analyses on the topic during the last three decades. His second-order meta-analysis was comprised of 19 meta-analyses representing 663 primary studies, more than 141,000 pupils and 1,263 effect sizes. Each meta-analysis that was included had to address the use of technology as a supplement to instruction, use pupil maths achievement as an outcome measure, report an effect size or enough data to calculate one, have been published after 1985 and be accessible to the public.

The author found that all technology enhancements positively affected pupil achievement, regardless of the technology’s purpose. However, technology that helped pupils perform computational functions had the greatest effects on pupil achievement, while combinations of enhancements demonstrated the least effects on pupil achievement. The author found that study quality and the type of technology used in the classroom were the main influencers on effect sizes. The highest-quality studies had the lowest effect sizes, which he attributes to their more rigorous analysis procedures. The high-quality reviews gave an overall effect size for the use of technology of +0.16 (compared with +0.38 for low- and +0.46 for medium-quality reviews).

Source: Technology-enhanced mathematics instruction: A second-order meta-analysis of 30 years of research (November 2017), Educational Research Review, Volume 22

Can teaching be simplified?

Technology that simplifies teaching by providing teachers with “off-the-shelf” lessons may increase pupil achievement, particularly if the teachers are supported in using them, according to a working paper published by the National Bureau of Economic Research (NBER).

The study, conducted by Kirabo Jackson and Alexey Makarin, provided middle school maths teachers with online lessons from the Mathalicious curriculum – an inquiry-based maths curriculum for grades 6 to 12 (Years 7 to 13) grounded in real-world topics and situations. Maths teachers from 170 schools across Virginia, US, took part and were randomly assigned to one of three conditions: full treatment (online access to Mathalicious lessons along with supports to promote their use); lesson-only (online access to Mathalicious lessons only); or control (business-as-usual).

While positive effects on pupil achievement in maths were seen in both the full treatment and lesson-only conditions, results were only significant for the full-treatment group. Providing teachers with online access to the lessons along with supports to promote their use increased pupil maths achievement by an effect size of +0.09 (p<.05). Test scores for pupils in the lesson-only group were non-significantly higher than those of the control group (effect size = +0.04).

Source: Can Online Off-The-Shelf Lessons Improve Student Outcomes? Evidence from A Field Experiment (January 2017), National Bureau of Economic Research (NBER), NBER Working Paper No. 22398

Insights on personalised learning

As part of a recent study for the Bill and Melinda Gates Foundation, RAND Corporation researchers have tried to identify what personalised learning (PL) looks like in a small sample of schools that are using PL approaches schoolwide.

This report describes the concept and implementation of personalised learning, along with some of the challenges, and considers how PL affects achievement in these schools. To measure how PL affects achievement, To measure how PL affects achievement, John F Pane and colleagues analysed maths and reading scores for all pupils in the sample (approximately 5,500 pupils) who took the Northwest Evaluation Association Measures of Academic Progress assessments. They found positive effect sizes of approximately +0.09 in maths and +0.07 in reading relative to a comparison group of similar pupils.

Based on the findings from the study, the researchers offer the following recommendations for implementing PL:

  • Provide teachers with resources and time to pilot new teaching approaches and gather evidence of how well they work.
  • Provide teachers with time and resources to collaborate on developing curriculum material and on reviewing and scoring pupil work.
  • Identify a school staff member who is comfortable with technology and has curriculum expertise to serve as a “just-in-time” resource for teachers.
  • Provide resources and support for school staff to help them choose the most appropriate digital or non-digital curriculum materials.
  • Provide resources and support for school staff to integrate multiple data systems.

Source: Informing progress: Insights on personalized learning implementation and iffects (July 2017), RAND Corporation

Does playing chess improve maths ability?

An article published in Learning & Behavior examines whether learning to play chess can help improve children’s mathematical ability. To test this hypothesis, Giovanni Sala and Fernand Gobet, from the University of Liverpool, conducted two studies with primary school children in schools in Italy.

The first experiment involved 233 children from eight schools (mean age = 8.5 years). The experimental group (N=53) attended 25 hours of chess lessons during school hours (although not necessarily during maths lessons), along with regular school activities, and were then given a test to assess their mathematical ability and a questionnaire to assess their metacognitive ability. The results were compared to both an active control group (who were similarly taught to play draughts) and a passive control group (who continued with regular school activities). The results showed no significant difference between the three groups in mathematical or metacognitive ability.

For the second experiment, 52 children (mean age = 9.32 years) in three classes of a primary school in Italy participated. Classes were randomly assigned to the three experimental conditions, but this time the active control group learned the game of Go instead of draughts, and both the chess and Go instruction replaced some of the time originally dedicated to learning maths (approximately 15 hours). The results showed no significant effects of learning chess on mathematical ability. Children in the passive control group seemed to benefit slightly more than those learning chess or Go. There was no difference between the three experimental groups on metacognitive ability.

The study concludes that the results of the two experiments do not support the hypothesis that learning chess benefits children’s mathematical ability. The effects of chess, if any, appear to be minimal and too limited to provide any educational advantage over traditional teaching methods.

Source: Does chess instruction improve mathematical problem-solving ability? Two experimental studies with an active control group (June 2017), Learning & Behavior doi:10.3758/s13420-017-0280-3

Report finds mixed effects for Saxon Math

The Institute of Education Sciences (IES) in the US has released a new research report on Saxon Math, and findings show mixed results for the programme.

Saxon Math is a curriculum for pupils in grades K-12 (Years 1-13). It uses an incremental structure that distributes content throughout the year. For the IES report, researchers reviewed studies of Saxon Math’s primary courses, which include kindergarten (Year 1) through pre-algebra. Out of 26 studies eligible for review, five studies fell within the scope of the What Work Clearinghouse’s (WWC) primary maths topic area and met WWC design standards. These five studies included 8,855 pupils in grades 1–3 and 6–8 in 149 schools across at least 18 states.

According to the report, the estimated impact of the intervention on outcomes in the mathematics achievement domain was positive and substantively important in two studies and indeterminate in three studies. The authors conclude that Saxon Math has mixed effects on maths test scores of pupils in primary classes.

Source: Saxon Math (May 2017), US Department of Education, Institute of Education Sciences, What Works Clearinghouse