Evidence and policy

In a review of important 2017 releases, MDRC recently referenced a memo to policymakers with recommendations for increasing research use and applying evidence to all policy decisions, both educational and otherwise.

Recommendations included:

  • Programmes and policies should be independently evaluated. To ensure high-quality evaluations, they should be directly relevant to policy, free of political or other influences and credible to subjects and consumers.
  • The government should provide incentives for programmes to apply evidence results to improve their performance.
  • Utilise a tiered evidence strategy, such as is used in the Every Student Succeeds Act, to set clear guidelines for standards.
  • Existing funding sources should be applied to generate evidence. A 1% set-aside was recommended.
  • Federal and state agencies should be allowed to access and share their data for evaluation purposes.

Source: Putting evidence at the heart of making policy (February 2017), MDRC

Best practices for informing policy-makers about research

A new report from Child Trends reviews the literature on conditions under which US policy-makers are most likely to use research, including the presentation formats that best facilitate their use. The authors, Elizabeth Jordan and P Mae Cooper, offer several insights based on their review of the evidence, including:

  • Policy-makers prefer a personal connection or conversation to a written report. One reason the authors cite is that reports are undigested information, meaning they require some expertise to pull out the information that is most relevant to the situation at hand.
  • While personal connections are usually best, no legislator can build and maintain relationships with experts in every field. The authors say that usually it is legislative staffers who fill this gap. Reports that summarise findings from a body of research are particularly useful to staffers, as they cover a variety of topics at one time.
  • For research to be useful to policy-makers and their staff, it must be relevant. The authors note that the information must relate to current policy debates, show an impact on “real people”, present information that is useful across states or localities, and be easy to read.
  • There are some formatting decisions that can help improve a written report’s accessibility. The authors suggest bulleted lists, highlighted text, charts, and graphs to help a policy-maker or staffer quickly absorb the main points of the research.

The report also provides several real-life examples of how research has informed public policy. For instance, the authors describe how rigorous evidence of the short- and long-term positive outcomes for children and families who participated in early childhood home visiting led the Obama Administration to create a new federal home visiting programme.

Source: Building bridges: How to share research about children and youth with policymakers (2016), Child Trends

Keep up the good work

In his Huffington Post blog, Robert Slavin, director of the Center for Research and Reform in Education, discusses a study that evaluated a behaviour management programme, First Step to Success, for students with behaviour problems. The programme has been evaluated successfully many times. In this latest study, 200 children in grades 1 to 3 (Years 2 to 4) with serious behaviour problems were randomly assigned to experimental or control groups. On behaviour and achievement measures, students in the experimental group scored much higher, with effect sizes of +0.44 to +0.87.

The researchers came back a year later to see if the outcomes had been maintained. Despite the substantial impacts seen previously, none of three prosocial/adaptive behaviour measures, only one of three problem/maladaptive behaviours, and none of four academic achievement measures now showed positive outcomes. However, the students had passed from teachers who had been trained in the First Step method to teachers who had not.

Dr Slavin says, “Imagine that all teachers in the school learned the program and all continued to implement it for many years. In this circumstance, it would be highly likely that the first-year positive impacts would be sustained and most likely improved over time.” He discusses the implications of the research, and the importance of continuing with successful interventions.

Source: Keep Up the Good Work (To Keep Up the Good Outcomes) (2016), Huffington Post

How do educational leaders use research?

The US National Center for Research in Policy and Practice has released a study examining how school and district leaders use research to inform their decisions.

Respondents were advised that the term research should be considered as action taken to answer a specific question (for example, how standardised test results in primary school relate to GCSE results) as opposed to solely looking at data (for example, to determine which students need extra help in a subject).

The survey was distributed to a nationally-representative sample of 14,276 leaders involved in making decisions about teaching in elementary or middle schools, where there is the most research available on effective programmes and great variety in teaching materials.

Instrumental research, used to guide a specific decision, was most often used in activities regarding the use of professional development programmes and directing resources to programmes. 70-88% of respondents said they use this type of research either frequently or all the time.

Conceptual uses of research, used to solve problems in schools and districts, were also widely reported, mostly when leaders needed clarification of an issue (61% frequently or all the time), or needed information to guide reform efforts.

Symbolic research, used for political purposes, was used most often to convince other people to join their point of view (68% frequently or all the time) or to rally support for a programme (67% frequently or all the time).

Respondents identified the most useful types of research resources as books, research and policy reports, and journal articles that were peer reviewed. When leaders were questioned about the relevancy, value, and credibility of research in education, they stated they felt research was relevant to practice, useful to the field, and valuable to educators, but its usefulness decreased with the time lag between the research itself and its publication.

Leaders’ ability to interpret research results varied. Although most understood how to interpret effect sizes and the reason for purposeful sampling in qualitative research, few could draw accurate conclusions from a case study, and more than half could not name any advantages of random assignment.

Source: Findings from a National Study on Research Use Among School and District Leaders (2016), National Center for Research in Policy and Practice

Using research to improve teaching practice

The Education Endowment Foundation has reported on two studies that looked at using education research to improve teaching practice.

Research into Practice was a pilot intervention aimed at supporting teachers to use evidence-based teaching and learning strategies to improve student progress. The project ran for a year in ten primary schools in Rochdale (north-west England). It involved professional development sessions and direct consultant support to help teachers:

  • Have more positive views about the use of research for improving teaching and learning;
  • Apply education research findings in the classroom and at a strategic level; and
  • Establish a stronger culture of evidence-based inquiry and practice.

There were some positive changes in teachers’ attitudes toward research. However, there was no evidence that teachers were more likely to use research evidence to inform their teaching practice.
The Research Champions project used a senior teacher based at one of five schools to work with research leads, other teachers, and senior leaders to promote engagement with research evidence. There were “audits” of school research needs, research symposia for teachers, periodic research and development forums, and personalised support. However, there was no evidence that teachers’ attitudes toward research, or their use of research, changed during the intervention.

Source: Research into Practice – Evidence-informed CPD in Rochdale and Research Champions (2016), Education Endowment Foundation.