Suspending suspensions

Out-of-school suspensions have typically been used as punishment for pupils who are truant (absent from school without parental consent) or chronically absent (missing 10% or more of school days). Given that the goal is to keep pupils in school and academically engaged, a few US states have banned this practice. A recent JESPAR article examined the effects of this ban on absence rates in Arkansas, which established a law in 2013 banning out-of-school suspensions. The state offered no training to schools, and each was left to make its own way with the policy change. Although out-of-school suspensions were banned, other punishments were allowed to continue, including in-school suspension, which takes a pupil out of the regular classroom for a time but allows them to continue their work elsewhere.

Using data from all Arkansas state schools, researchers compared the attendance of truant and non-truant pupils between 2012–13 (pre-policy) and 2013–14 (post-policy) to see if there were any dramatic changes in attendance for truant pupils that did not occur with non-truant pupils. Subjects were limited to grades 7–12 (Years 8–13), in which 96% of truancy occurs.

Researchers found that compliance with the law was low, particularly in disadvantaged schools, with only a third of all schools complying. Among schools that did comply, there was no evidence of change in student behaviour after the policy went into effect. Three key findings were:

  • Policy alone is not enough to change behaviour—implementation of a policy must be overseen and reinforced.
  • When policies change, schools must be evaluated regarding whether their resources are sufficient to enforce this change, or whether they need support or training in order to be able to comply.
  • High-level policy changes need to be followed by quantitative and qualitative evaluation to assess key outcomes and compliance.

In addition, researchers reflected that, perhaps because there was still other punishment, truancy continued. They stated that punishment does not address the root causes as to why pupils are truant, and that pupil outcomes might not change if schools simply replace out-of-school suspensions with other types of punishment.

Source: Discipline reform: The impact of a statewide ban on suspensions for truancy (January 2019), Journal of Education for Students Placed at Risk (JESPAR), Volume 24, Issue 1

Is a government-mandated reduced-class size policy likely to improve pupils’ achievement in France? The data says oui

French pupils from disadvantaged areas demonstrate lower achievement than their more affluent peers. In an effort to close this achievement gap, the French government issued a policy in 2017 reducing Year 2 class size in high-priority educational areas to no more than 12 pupils, extending to Year 3 classes and priority educational areas in 2018. In order to provide evidence regarding the feasibility of such a policy, researchers used data from a 2003 first-grade-class-size-reduction policy in France to examine its carry-over effects into the second grade.

The 2003 study involved assigning classrooms to either small (12 pupils/class n=100 classes) or large (20–25 pupils/class, n=100 classes) class sizes. At the start of the 2002–03 school year, children were pre-tested on pre-reading skills and matched. In post-tests at the end of the school year, results favoured the small-class-size group on word reading (ES = +0.14) and word spelling (ES = +0.22). These effects are very small in light of the costs of halving class size.

The new study examined these pupils’ reading achievement at the end of Year 3, where the pupils formerly placed in smaller classrooms had been placed in full-sized classes again. Subjects were 1,264 pupils (663 in the intervention group and 601 in the control group) who had received both the initial testing in Year 2 and had test scores at the end of Year 3. Results showed that while both groups were equivalent at the start of Year 2, and by the end of the year the small-class-size group showed greater academic achievement than the control group, this gain diminished over the summer break and had completely disappeared by the end of Year 3. That is, there was no long-term impact of one year of reduced class size.

For more information, the original 2003 study was reported in Best Evidence in Brief in July.

Source: Reducing the number of pupils in French first-grade classes: Is there evidence of contemporaneous and carryover effects? (November 2018), International Journal of Educational Research, Volume 96,

Small class size vs. evidence-based interventions

The Ministry of Education in France introduced a policy in 2002 that reduced class size to no more than 12 pupils in areas determined to have social difficulties and high proportions of at-risk pupils, called Zones d’Education Prioritaire (ZEP). In order to evaluate the effectiveness and usefulness of this policy, researcher Jean Ecalle and colleagues in France examined the results of the policy-mandated class size reduction on the reading achievement of first grade (Year 2) pupils (Study 1), and compared them to the effects of an evidence-based literacy intervention on the reading achievement of at-risk children in normal-sized classes (20 pupils) (Study 2).

Study 1, reducing class size, involved assigning classrooms to either small (12 pupils/class n=100 classes) or large (20–25 pupils/class, n=100 classes) class sizes (with the support of the Ministry). At the start of the 2002–03 school year, 1,095 children were pre-tested on pre-reading skills and matched at pre-test. At the end of the school year, children were post-tested, with results favouring the small-class-size group on word reading (effect size=+0.14) and word spelling (effect size=+0.22).

In Study 2, researchers separated 2,803 first grade (Year 2) pupils in ZEP areas into an experimental group who received an evidence-based reading intervention, and a control group who did not. The intervention was a protocol developed by the Association Agir pour l’Ecole (Act for School), who developed a hierarchy of teaching reading based on evidence-based methods of learning to read, progressing from training phonological skills, to learning letter sounds, decoding, and fluency. Act for School monitored compliance with the protocol weekly. Class size for both groups was 20 pupils. Experimental teachers received one day of training, and provided 30 minutes of teaching a day to average or high readers in groups of 10 to 12, and one hour a day for lower readers in groups of four to six. Again, children were pre-tested on reading skills and matched between groups. All areas post-tested favoured the experimental group, with significant effects in word reading (effect size=+0.13) and word spelling (effect size=+0.12).

Researchers stated that based on the results of both studies, the optimal recommendation to improve literacy skills for at-risk pupils would be a double intervention, combining evidence-based practices within small classes.

Source: Effects of policy and educational interventions intended to reduce difficulties in literacy skills in grade 1 (June 2019), Studies in Educational Evaluation, Volume 61

Evidence and policy

In a review of important 2017 releases, MDRC recently referenced a memo to policymakers with recommendations for increasing research use and applying evidence to all policy decisions, both educational and otherwise.

Recommendations included:

  • Programmes and policies should be independently evaluated. To ensure high-quality evaluations, they should be directly relevant to policy, free of political or other influences and credible to subjects and consumers.
  • The government should provide incentives for programmes to apply evidence results to improve their performance.
  • Utilise a tiered evidence strategy, such as is used in the Every Student Succeeds Act, to set clear guidelines for standards.
  • Existing funding sources should be applied to generate evidence. A 1% set-aside was recommended.
  • Federal and state agencies should be allowed to access and share their data for evaluation purposes.

Source: Putting evidence at the heart of making policy (February 2017), MDRC

Best practices for informing policy-makers about research

A new report from Child Trends reviews the literature on conditions under which US policy-makers are most likely to use research, including the presentation formats that best facilitate their use. The authors, Elizabeth Jordan and P Mae Cooper, offer several insights based on their review of the evidence, including:

  • Policy-makers prefer a personal connection or conversation to a written report. One reason the authors cite is that reports are undigested information, meaning they require some expertise to pull out the information that is most relevant to the situation at hand.
  • While personal connections are usually best, no legislator can build and maintain relationships with experts in every field. The authors say that usually it is legislative staffers who fill this gap. Reports that summarise findings from a body of research are particularly useful to staffers, as they cover a variety of topics at one time.
  • For research to be useful to policy-makers and their staff, it must be relevant. The authors note that the information must relate to current policy debates, show an impact on “real people”, present information that is useful across states or localities, and be easy to read.
  • There are some formatting decisions that can help improve a written report’s accessibility. The authors suggest bulleted lists, highlighted text, charts, and graphs to help a policy-maker or staffer quickly absorb the main points of the research.

The report also provides several real-life examples of how research has informed public policy. For instance, the authors describe how rigorous evidence of the short- and long-term positive outcomes for children and families who participated in early childhood home visiting led the Obama Administration to create a new federal home visiting programme.

Source: Building bridges: How to share research about children and youth with policymakers (2016), Child Trends

Implement, educate, evaluate

An OECD report, Education Policy Outlook 2015: Making Reforms Happen, calls for a coherent framework of assessment and analysis of the effectiveness of education reforms. The report looks at the implementation of educational policy reforms through national and international comparisons.

Pasi Sahlberg, visiting Professor of Practice at Harvard University, critiques the report in a recent article. He highlights one of the report’s conclusions that “once new policies are adopted, there is little follow-up” and that only one in ten of the policies considered in the OECD report has been evaluated for its impact. Professor Sahlberg also praises increases in research and policy analysis in some countries, naming the UK and USA as examples.

Source: Education Policy Outlook 2015: Making Reforms Happen (2015), OECD.