Insights on personalised learning

As part of a recent study for the Bill and Melinda Gates Foundation, RAND Corporation researchers have tried to identify what personalised learning (PL) looks like in a small sample of schools that are using PL approaches schoolwide.

This report describes the concept and implementation of personalised learning, along with some of the challenges, and considers how PL affects achievement in these schools. To measure how PL affects achievement, To measure how PL affects achievement, John F Pane and colleagues analysed maths and reading scores for all pupils in the sample (approximately 5,500 pupils) who took the Northwest Evaluation Association Measures of Academic Progress assessments. They found positive effect sizes of approximately +0.09 in maths and +0.07 in reading relative to a comparison group of similar pupils.

Based on the findings from the study, the researchers offer the following recommendations for implementing PL:

  • Provide teachers with resources and time to pilot new teaching approaches and gather evidence of how well they work.
  • Provide teachers with time and resources to collaborate on developing curriculum material and on reviewing and scoring pupil work.
  • Identify a school staff member who is comfortable with technology and has curriculum expertise to serve as a “just-in-time” resource for teachers.
  • Provide resources and support for school staff to help them choose the most appropriate digital or non-digital curriculum materials.
  • Provide resources and support for school staff to integrate multiple data systems.

Source: Informing progress: Insights on personalized learning implementation and iffects (July 2017), RAND Corporation

Study of personalised learning shows promising evidence – or does it?

The US National Education Policy Center’s Think Twice Think Tank Review Project recently reviewed a RAND study on personalised learning. The RAND study examined the effects of three school-wide personalised learning initiatives on pupil achievement to try to find evidence linking specific learning strategies to achievement outcomes.

RAND defined “personalised learning” (PL) as incorporating five specific characteristics including data-supported pupil goals accessible to teachers and pupils, and personalised learning of each pupil’s choice with in-school support and learning outside school.

The RAND study compared the MAP reading and maths scores of 11,000 pupils in 62 schools who had been using a personalised learning approach for two years to the scores of pupils matched at baseline to serve as a comparison. Researchers found higher achievement scores for the PL group, especially at primary age. In addition, the study showed that personalised learners’ scores increased at a greater rate than the nation’s scores. Overall, researchers deemed PL promising practice.

However, the Think Twice Think Tank Review Project disagrees. In a review of the study researchers felt that the study’s limitations prevented it from demonstrating true evidence of promising practice. First, reviewers noted that only pupil involvement in analysing their own data and goal setting was associated with consistent gains. They pointed out that two of the attributes ascribed to the success of PL – flexible learning environments and student grouping – were also used in schools not using PL. They noted that the largest departure from usual classroom practice (competency-based progression) was not used in the majority of the experimental schools, casting doubt on its pertinence. In addition, reviewers were dubious about the generalisability of the findings because 90% of the study schools were charter schools.

Think Tank reviewers concluded that the study suggests there may indeed be personalised practices associated with test score gains, but that the practices in the three experimental models weren’t drastically different than practices in untreated schools. The study’s limitations cast doubt on the models’ generalisation.

Source: Continued Progress Promising Evidence on Personalized Learning (2015), RAND Corporation, and Review of Continued Progress: Promising Evidence on Personalized Learning (2016), NEPC.

The characteristics of excellent schools and teachers

A new report published by Pearson, Exploring Effective Pedagogy in Primary Schools: Evidence from Research, builds on the findings of the Effective Pre-school, Primary and Secondary Education (EPPSE) study. The researchers looked at practices in 125 English primary schools.

Standardised assessments were used to measure children’s academic attainment in reading and maths in Years 1 and 5. Classroom practices and processes were studied using two classroom observation instruments.

The researchers identified 11 pedagogic strategies that were important in good schools: organisation, shared objectives, homework, classroom climate, behaviour management, collaborative learning, personalised teaching and learning, making links explicit, dialogic teaching and learning, assessment for learning, and whole-class teaching.

In addition, teachers in excellent schools excelled in organisational skills; positive classroom climate; personalised highly-interactive approaches to teaching and learning; use of dialogic teaching and learning; and more frequent and effective use of whole-class teaching.

Source: Exploring Effective Pedagogy in Primary Schools: Evidence from Research (2014), Pearson.

The gene genie

Researchers at King’s College, the University of Warwick, and the University of New Mexico have published a new paper exploring the role of genes in educational achievement. They wanted to test the hypothesis that genetic differences (heritability) in educational achievement persist throughout compulsory education, as assessed by GCSEs at age 16.

The authors used data on 11,117 twins born in England and Wales between 1994 and 1996 recruited into the Twins Early Development Study and considered genetics, shared or common factors, and non-shared or unique environmental components. They found that heritability was substantial (58% of the variation) for overall GCSE performance for compulsory core subjects. In contrast, the overall effects of the shared environment, which includes all family and school influences shared by twins growing up in the same family and attending the same school, accounted for about 36% of the variance of mean GCSE scores.

They suggest that the significance of these findings is that individual differences in educational achievement can be attributed much more to genetics than to school or family environment, and conclude that this supports personalised learning.

Source: Strong Genetic Influence on a UK Nationwide Test of Educational Achievement at the End of Compulsory Education at Age 16 (2013), PLoS ONE.

Examining the effects of assessment

Researchers at the RAND Corporation have conducted a series of literature reviews that focus on topics such as high-stakes testing, performance assessment, and formative evaluation.

Their findings, published in a new report, suggest that there are a wide variety of effects that testing might have on teachers’ activities in the classroom, including changes in curriculum content and emphasis (eg, changes in the sequence of topics, reallocation of emphasis across and within topics); changes in how teachers allocate time and resources across different pedagogical activities (eg, focusing on test preparation); and changes in how teachers interact with individual pupils (eg, using test results to personalise teaching). The report, “New Assessments, Better Instruction? Designing Assessment Systems to Promote Instructional Improvement”, also identifies a number of factors (eg, pupil characteristics and regional policies) that mediate the relationship between assessment and teaching practices.

The authors suggest that the role of tests would be enhanced by policies that ensure tests mirror high-quality teaching, are part of a larger, systemic change effort, and are accompanied by specific supports for teachers.

Source: New Assessments, Better Instruction? Designing Assessment Systems to Promote Instructional Improvement (2013), RAND Corporation.

What are the effects of a technology-based algebra curriculum?

This working paper from the RAND Corporation examines the effectiveness of Cognitive Tutor Algebra I (CTAI), a technology-based algebra course designed for pupils at a variety of ability and year levels. The curriculum includes traditional textbook and workbook materials along with automated tutoring software that provides self-paced individualised tuition and attempts to bring pupils to mastery of a topic before they progress further.

Schools participating in the study were matched into similar pairs and randomly assigned to either continue with their current algebra curriculum for two years or to adopt CTAI. The sample included 73 high schools and 74 middle schools in seven US states.

Analysis of post-test outcomes on an algebra proficiency exam found no effects in the first year of implementation, but strong evidence in support of a positive effect in the second year. The estimated effect is statistically significant for high schools but not for middle schools. The authors report that in both cases, the magnitude is sufficient to improve the average pupil’s performance by approximately eight percentile points.

Source: Effectiveness of Cognitive Tutor Algebra I at Scale (2013), RAND Education.