Examining the research on charter schools

The What Works Clearinghouse (WWC) has released a new topic area that focuses on the impact of charter schools on pupil achievement and other outcomes. As part of the launch, the WWC released three intervention reports, which review available research on an intervention or programme to determine if there is strong evidence that it has a positive impact on pupil outcomes.

The intervention reports examine the following three programmes:

  • Knowledge is Power Program (KIPP), a non-profit network of more than 200 US public charter schools educating early childhood, elementary, middle and high school pupils. According to the WWC intervention report, research shows that KIPP had positive effects on mathematics achievement and English language achievement, and potentially positive effects on science achievement and social studies achievement for middle and high school pupils (Years 7 to 13), and no discernible effects on pupil progression (eg, high school graduation within four years of grade 9 (Year 10) entry) for high school pupils.
  • Green Dot Public Schools, a non-profit organisation that operates more than 20 public charter middle and high schools in California, Tennessee and Washington. The WWC reports that Green Dot Public Schools had potentially positive effects on mathematics achievement, pupil progression, school attendance and English language achievement for high school pupils (Years 10 to 13).
  • Harlem Children’s Zone (HCZ) Promise Academy Charter Schools, a non-profit organisation designed to serve low-income children and families living in Harlem in New York City. According to the intervention report, the WWC is unable to draw any conclusions based on available research about the effectiveness or ineffectiveness of the HCZ Promise Academy Charter Schools on elementary, middle and high school pupils. Research that meets WWC design standards is needed to determine the effectiveness or ineffectiveness of this intervention.

Source:  Charter Schools, What Works Clearinghouse

Interventions can reduce school exclusion but the effect is temporary

This Campbell Systematic Review examined the impact of interventions to reduce exclusion from school. School exclusion, also known as suspension, involves the removal of pupils from regular teaching for a period during which they are not allowed to be present at school. In some extreme cases, the pupil is not allowed to come back to the same school (expulsion).

The review summarised 37 studies, reporting 38 interventions’ effect sizes. Most studies were from the US (n=33) and the UK (n=3). All of them were randomised controlled trials.

The evidence suggested that school-based interventions are effective at reducing school exclusion during the first six months after the intervention (effect size =+0.30), but that this effect is not sustained. Some specific types of interventions showed more promising results than others. Of the nine different types of school-based interventions included in the review, four types (enhancement of academic skills, counselling, mentoring/monitoring and skills training for teachers) showed positive results in reducing exclusion. However, based on the number of studies involved, the researchers suggest that results must be treated with caution.

Source: School-based interventions for reducing disciplinary school exclusion: a systematic review (January 2018), A Campbell Systematic Review 2018:1, Campbell Collaboration Crime and Justice Coordinating Group

Programme components and disadvantaged pupils

Research shows that pupils from low socioeconomic backgrounds are less likely to attend pre-school or to have a home environment incorporating literacy and language activities than their less disadvantaged peers. As a result, children from low socioeconomic backgrounds are less likely to enter school with the social and academic skills needed to set them up for success. Jans Deitrichson and colleagues at the Danish National Centre for Social Research recently performed a meta-analysis aimed at determining what components within academic interventions are the most effective at improving the achievement of primary school students from low-socioeconomic backgrounds.

A total of 101 studies performed between 2000–2014 were included in the meta-analysis. Seventy-six percent were randomised controlled trials and the rest were quasi-experimental studies. Studies had to target pupils from low socioeconomic backgrounds, utilise standardised test results in reading and maths as the outcome measures, and take place in OECD or EU countries, although most were in the US. They also had to contain information that allowed the researchers to calculate effect sizes.

The authors sorted each study’s academic intervention into “component categories” (the methods used). Examples include coaching/ mentoring of pupils, cooperative learning, incentives, small-group tutoring, or a combination of these or other methods. Analysis demonstrated that tutoring, feedback and progress monitoring, and cooperative learning were the components with the largest effect sizes. The authors stated that although the average effect sizes for these components were not large enough to close the achievement gap between high- and low-socioeconomic pupils, they certainly reduced it. They suggest that cost-effectiveness studies should be performed on these programmes to give policymakers and educators a fuller picture of programme benefits.

Source: Academic interventions for elementary and middle school students with low socioeconomic status: A systematic review and meta-analysis (January 2017), Review of Educational Research, Vol 87, Issue 2

Scale-up evaluation of Switch-on disappoints

An independent evaluation for the Education Endowment Foundation (EEF) of the Switch-on intervention has found no evidence that it improves the reading outcomes of pupils struggling with literacy at Key Stage 1 (ages 5–7 years) compared to schools’ usual practices.

Switch-on is an intensive, targeted literacy intervention that aims to improve the reading skills of pupils who are struggling with literacy. There are two versions of the intervention: Switch-on Reading and Switch-on Reading and Writing. Both involve specially trained Teaching Assistants (TAs) delivering a tailored programme of literacy support in daily 20-minute sessions over a ten-week period.

Schools selected pupils in Year 3 who were working below age-related expectations at the end of Key Stage 1 and who did not have a high level of special needs. Each of the 184 participating schools was then randomly assigned to receive either Switch-on Reading, Switch-on Reading and Writing, or to continue their usual practices of supporting pupils with reading difficulties. In total, 999 pupils were involved in the trial.

Estimated effect sizes were zero and not statistically significant. The intervention also showed no effect on pupils eligible for free school meals. These findings contradict a previous, smaller EEF-funded evaluation of Switch-on which had shown signs of promise in raising reading outcomes for Year 7 pupils.

Source: Switch-on – effectiveness trial (May 2017), Education Endowment Foundation

Academic interventions for low-SES pupils

A systematic review and meta-analysis published in Review of Education Research looks at effective academic interventions for pupils with low socio-economic status (SES).

Jens Dietrichson and colleagues included studies that used a treatment–control group design, were performed in Organisation for Economic Cooperation and Development (OECD) and EU countries and measured achievement with standardised tests in maths or reading. The analysis included 101 studies performed between 2000 and 2014, 76% of which were randomised controlled trials.

Positive effect sizes (ES) were reported for many of the interventions. Comparatively large and robust average effect sizes were found for interventions that involved tutoring (ES = +0.36), feedback and progress monitoring (ES = +0.32) and co-operative learning (ES = +0.22). The report points out that, although these effect sizes are not large enough to close the gap between high- and low-SES pupils, they represent a substantial reduction of that gap if targeted towards low-SES students.

Source: Academic interventions for elementary and middle school students with low socioeconomic status: a systematic review and meta-analysis (January 2017), Review of Educational Research

Does the KiVa anti-bullying programme work?

A study in Prevention Science evaluates the effectiveness of the KiVa anti-bullying programme in Italy through a randomised controlled trial of students in grades 4 and 6 (equivalent to Years 5 and 7). The sample involved 2,042 students across 13 schools that were randomly assigned to intervention (KiVa) or control (usual school provision) conditions. The Italian school system is divided into primary school (grades 1–5), middle school (grades 6–8), and secondary school (grades 9–14), so only schools which had both primary and middle schools were included.

KiVa is a research-based anti-bullying programme developed by the University of Turku, Finland. It is a schoolwide intervention that is focused on the bystanders’ reactions to a bullying situation, which assist and reinforce the bully, and aims to change their attitudes and behaviours.

Researchers Annalaura Nocentini and Ersilia Menesini considered different outcomes (bullying, victimisation, pro-bullying attitudes, pro-victim attitudes, empathy toward victims), analyses, and estimates of effectiveness in order to compare the Italian results with those from other countries. Multilevel models showed significant results for KiVa for all outcomes and analyses in grade 4. In grade 6, KiVa also reduced bullying, victimisation, and pro-bullying attitudes, but the effects were smaller as compared to grade 4, although still significant. The results also showed that the odds of being a victim were 1.93 times higher for a control student than for a KiVa student in grade 4. Overall, their findings provide evidence of the effectiveness of the programme in Italy.

Source: KiVa Anti-Bullying Program in Italy: Evidence of Effectiveness in a Randomized Control Trial (2016), Prevention Science, 17(8)