A new research brief by John F Pane and colleagues at the RAND Corporation asks the question: “Does personalised learning improve pupil learning more than other educational approaches?” As part of their report, the authors present findings from an evaluation of personalised learning (PL) schools conducted by RAND Corporation researchers for the Bill and Melinda Gates Foundation.
The research team analysed maths and reading scores for approximately 5,500 pupils in 32 US schools that received funding from the Next Generation Learning Challenges (NGLC) initiative to support highly personalised approaches to learning. These schools took the Northwest Evaluation Association (NWEA) Measures of Academic Progress (MAP) mathematics and reading assessments for one academic year: autumn 2014 to spring 2015. The research team compared the achievement of pupils in PL schools with matched peers attending non-PL schools and national norms.
Key findings from the research brief include:
- Early evidence suggests that PL can improve achievement for pupils, regardless of their starting level of achievement.
- Benefits of PL may take some time to emerge. Analyses suggest that effects may be more positive after schools have experience implementing PL.
- To date, the field lacks evidence on which practices are most effective or what policies must be in place to maximise the benefits.
The authors note that additional research is needed using more rigorous experimental studies.
Source: How Does Personalized Learning Affect Student Achievement? RAND Corporation, 2017.
Out-of-school-time (OST) programmes typically provide children with additional academic lessons outside of school hours and/or recreational and enrichment activities. To examine the evidence base on OST programmes, Jennifer McCombs and colleagues from the RAND Corporation reviewed meta-analyses and large-scale, rigorous experimental and quasi-experimental evaluations of after-school and summer programmes. Their review included specialty programmes (eg, sports or arts programmes); multipurpose programmes (eg, Boys and Girls clubs); and academic programmes (eg, summer learning programmes).
After reviewing the research, the authors compiled the following conclusions:
- OST programmes provide measurable benefits to children and families on outcomes directly related to programme content.
- Academic OST programmes with sufficient “dosage” (measured by the hours of content provided) can demonstrably improve pupil achievement.
- Programme quality and intentionality influence outcomes.
- Children need to attend regularly to measurably benefit from programming.
The authors provide a complete list of studies reviewed and their key findings.
A previous issue of Best Evidence in Brief included a study by the Nuffield Foundation, which examines the effect of OST study programmes on GCSE performance in England.
Source: The value of out-of-school-time programmes (2017), PE-267-WF, RAND Corporation
A new research brief by Jennifer L Steele and colleagues, published by the RAND Corporation, presents new research on dual-language immersion (DLI) programmes. These programmes provide both native English speakers and children learning English as an additional language (EAL) with general academic teaching in two languages from kindergarten (Year 1) onwards.
In partnership with the American Councils on International Education and the Portland Public Schools in Oregon (PPS), the authors conducted a random-assignment study of DLI education. The goal was to estimate the causal effects of the district’s DLI programmes on pupil performance over time in reading, mathematics and science, and on EAL pupils’ reclassification as English proficient.
PBS allocates immersion slots using a random-assignment lottery process for those who apply to the programmes. The study focused on 1,625 DLI lottery applicants in the kindergarten cohorts from 2004–2005 to 2010–2011. Pupil achievement was tracked until 2013–2014.
Key findings of the study were as follows:
- PPS pupils randomly assigned to dual-language immersion programmes outperformed their peers on state reading tests by 13% of a standard deviation in grade 5 (Year 6) and by 22% of a standard deviation in grade 8 (Year 9).
- Immersion-assigned pupils did not show statistically significant benefits or deficits in terms of mathematics or science performance.
- There were no clear differences in the effects of dual-language immersion according to pupils’ native language.
- EAL pupils assigned to dual-language immersion were more likely than their peers to be classified as English proficient by grade 6 (Year 7). This effect was mostly attributed to EAL pupils whose native language was the same as one of the two languages taught.
Source: Dual-language immersion programs raise student achievement in English (2017), RAND Corporation Research Brief, RB-9903
A new report from the RAND Corporation presents findings from a review of research on preschool programmes in the US. Specifically, the report examines whether high-quality preschool produces favourable effects for participating children and their families, the magnitude of the impacts, and how long the benefits last. The preschool programmes ran for one or two years (equivalent to the Reception year, or the Nursery and Reception years, in the UK).
The researchers analysed evaluation findings from 15 full-scale, publicly funded preschool programmes implemented at the national, state, and local levels. The researchers examined evidence for the specific programmes, as well as results from syntheses across multiple preschool programme evaluations. In cases where children had been followed beyond the preschool years, they also considered research regarding longer-term effects.
Key findings of their review include:
- Favourable impacts have been demonstrated for part- and full-day preschool programmes, as well as one- and two-year programmes, but the research is not definitive about the comparative effectiveness of these options.
- High quality is a common element among the preschool programmes with the largest effects on school readiness and with sustained effects at older ages. These effective programmes include such features as well-trained classroom teachers who are provided with ongoing professional development support through coaching and other mechanisms, a learning environment that supports teachers and children, a well-defined curriculum that is implemented with fidelity in the classroom and aligned with the early school years, and ongoing monitoring of programme quality and other metrics that support continuous quality improvement.
- High-quality preschool programmes show sustained benefits for aspects of school performance other than achievement scores, such as lower rates of special education use, fewer students being held back a year, and higher rates of high school graduation.
- Children across the income spectrum may benefit from high-quality preschool, but the impacts tend to be larger for more disadvantaged children. The researchers also examined evidence on the investment value of preschool, reporting that estimates of the economic return for full-scale high-quality preschool range from about $2 to $4 for every $1 invested.
Source: Informing Investments in Preschool Quality and Access in Cincinnati (2015), RAND Corporation.
The US National Education Policy Center’s Think Twice Think Tank Review Project recently reviewed a RAND study on personalised learning. The RAND study examined the effects of three school-wide personalised learning initiatives on pupil achievement to try to find evidence linking specific learning strategies to achievement outcomes.
RAND defined “personalised learning” (PL) as incorporating five specific characteristics including data-supported pupil goals accessible to teachers and pupils, and personalised learning of each pupil’s choice with in-school support and learning outside school.
The RAND study compared the MAP reading and maths scores of 11,000 pupils in 62 schools who had been using a personalised learning approach for two years to the scores of pupils matched at baseline to serve as a comparison. Researchers found higher achievement scores for the PL group, especially at primary age. In addition, the study showed that personalised learners’ scores increased at a greater rate than the nation’s scores. Overall, researchers deemed PL promising practice.
However, the Think Twice Think Tank Review Project disagrees. In a review of the study researchers felt that the study’s limitations prevented it from demonstrating true evidence of promising practice. First, reviewers noted that only pupil involvement in analysing their own data and goal setting was associated with consistent gains. They pointed out that two of the attributes ascribed to the success of PL – flexible learning environments and student grouping – were also used in schools not using PL. They noted that the largest departure from usual classroom practice (competency-based progression) was not used in the majority of the experimental schools, casting doubt on its pertinence. In addition, reviewers were dubious about the generalisability of the findings because 90% of the study schools were charter schools.
Think Tank reviewers concluded that the study suggests there may indeed be personalised practices associated with test score gains, but that the practices in the three experimental models weren’t drastically different than practices in untreated schools. The study’s limitations cast doubt on the models’ generalisation.
Source: Continued Progress Promising Evidence on Personalized Learning (2015), RAND Corporation, and Review of Continued Progress: Promising Evidence on Personalized Learning (2016), NEPC.
A US programme intended to boost pupil achievement by providing teachers with two years of professional development, including formal training sessions and meetings with a leadership coach, is showing signs of potential, according to a new RAND Corporation report.
The Leading Educators (LE) Fellowship programme, selects mid-career teachers through a competitive application process. To examine the impact of LE, researchers are comparing pupil achievement gains for teachers who participated in the programme as fellows or mentees with the pupil achievement gains of other teachers.According to RAND, early findings of the programme are mixed, but suggest that it shows promise in improving pupil achievement. Specifically, they report:
- Among fellows, there are both some statistically significant positive and negative programme effects on pupil achievement, with results that vary across states, subject areas, and model specifications.
- Among mentee teachers, for whom sample sizes are larger, there is some suggestive evidence of impacts on pupil achievement — in particular, marginally significant and significant positive programme effects among mentees who teach maths and social studies, respectively, in Louisiana.
- The impact of the programme on teacher retention is unclear, with no consistent pattern of retention impacts across cohorts or states.
The authors note that the current results are based on few years of data and on a small sample of teachers, and results may change when there are more fellows and mentored teachers included in future studies.
Source: Examining the Early Impacts of the Leading Educators Fellowship on Student Achievement and Teacher Retention (2015), RAND Corporation.