No impact for inquiry-based learning intervention

The Education Endowment Foundation has published an evaluation of an inquiry-based learning intervention – CREST Silver Award.

Delivered by the British Science Association, the CREST programme aims to help pupils engage with science, technology, engineering and maths (STEM) subjects by allowing them to develop their own project ideas. Eighty secondary schools in London and the south east took part in the trial, involving 2,810 Year 9 pupils (ages 13–14). While CREST can normally be delivered by any STEM department in the school, for the trial, CREST was delivered by the science department in each school. Schools had the flexibility to decide how they would deliver CREST (for example, as a whole class activity or as a STEM club) and when they would run the programme (during school, after school, lunch break, or during class time). Pupils were expected to complete 30 hours of project work in total.

The independent evaluation by NatCen found that pupils who took part in the programme made no additional progress in science achievement (as measured by the Progress Test in Science) compared to similar pupils who were not offered the programme (effect size = -0.01). Nor was there any evidence that the CREST Silver Award improved self-efficacy in science or increased the percentage of pupils aspiring to a STEM career; however, small positive impacts were found for pupil confidence and attitudes toward school.

Source: CREST Silver: Evaluation report (December 2019), Education Endowment Foundation

Suspending suspensions

Out-of-school suspensions have typically been used as punishment for pupils who are truant (absent from school without parental consent) or chronically absent (missing 10% or more of school days). Given that the goal is to keep pupils in school and academically engaged, a few US states have banned this practice. A recent JESPAR article examined the effects of this ban on absence rates in Arkansas, which established a law in 2013 banning out-of-school suspensions. The state offered no training to schools, and each was left to make its own way with the policy change. Although out-of-school suspensions were banned, other punishments were allowed to continue, including in-school suspension, which takes a pupil out of the regular classroom for a time but allows them to continue their work elsewhere.

Using data from all Arkansas state schools, researchers compared the attendance of truant and non-truant pupils between 2012–13 (pre-policy) and 2013–14 (post-policy) to see if there were any dramatic changes in attendance for truant pupils that did not occur with non-truant pupils. Subjects were limited to grades 7–12 (Years 8–13), in which 96% of truancy occurs.

Researchers found that compliance with the law was low, particularly in disadvantaged schools, with only a third of all schools complying. Among schools that did comply, there was no evidence of change in student behaviour after the policy went into effect. Three key findings were:

  • Policy alone is not enough to change behaviour—implementation of a policy must be overseen and reinforced.
  • When policies change, schools must be evaluated regarding whether their resources are sufficient to enforce this change, or whether they need support or training in order to be able to comply.
  • High-level policy changes need to be followed by quantitative and qualitative evaluation to assess key outcomes and compliance.

In addition, researchers reflected that, perhaps because there was still other punishment, truancy continued. They stated that punishment does not address the root causes as to why pupils are truant, and that pupil outcomes might not change if schools simply replace out-of-school suspensions with other types of punishment.

Source: Discipline reform: The impact of a statewide ban on suspensions for truancy (January 2019), Journal of Education for Students Placed at Risk (JESPAR), Volume 24, Issue 1

Future planning and achievement among pupils

Several studies have related the benefits of future planning to academic achievement, but not many have examined whether academic achievement also influences how pupils plan their future. Zhao and colleagues from Shandong Normal University conducted a longitudinal study to examine the relationship between Chinese middle school pupils’ academic achievement and future planning in educational and occupational domains.

The study included three assessments six months apart from spring 2014 to spring 2015 in Shandong Province in eastern China. A total of 775 pupils from sixth to eighth grade (Years 7–9) participated in the first assessment wave. The questionnaire measured pupils’ future explorations, commitments, and affects concerning future education and occupations. Data on their academic achievement were collected from school records of their scores in Chinese, English and maths. The relationships were analysed with data collected at different times.

The analysis showed that:

  • There were reciprocal relationships between academic achievement and pupils’ future educational planning.
  • Reciprocal relationships were not seen between academic achievement and future planning in the occupational domain.
  • Commitment’s relationship to achievement was more robust than that of exploration to achievement.
  • The relationships were the same for both boys and girls.

The authors suggest that understanding the importance of educational performance led middle school pupils to invest more effort into improving achievement. The social status brought by high academic achievement in Chinese society might also trigger positive affects concerning future planning.

Source: Longitudinal relations between future planning and adolescents’ academic achievement in China (August 2019), Journal of Adolescence, Volume 75

More evidence for growth mindset

The findings of an MDRC evaluation of a growth mindset intervention have suggested a positive impact on pupils’ academic performance.

To test whether a growth mindset intervention could improve pupils’ academic performance, the National Study of Learning Mindsets implemented a randomised controlled trial of a low-cost growth mindset intervention specifically designed for ninth grade (Year 10) pupils. A total of 11,888 pupils from 63 high schools across the US took part in the intervention, which included two 25-minute self-administered online training modules on the topic of brain development. Pupils in the intervention group were given modules about growth mindset and were asked to answer reflective questions in a survey. Instead of learning about the brain’s malleability, pupils in the control group learned about basic brain functions, and they were also asked to answer survey questions.

The results of the evaluation found a positive impact on pupils’ average grade point average (GPA) (effect size = +0.04), as well as their maths GPA (effect size = +0.05). Other results from the evaluation suggest that:

  • The intervention changed pupils’ self-reported mindset beliefs, their attitudes towards efforts and failure and their views on academic challenges.
  • Immediately after the intervention, students were more likely to attempt more challenging academic tasks.
  • Pupils who were lower performing at pre-test benefited more than their higher-performing peers.

Source: Using a growth mindset intervention to help ninth-graders: An independent evaluation of the National Study of Leaning Mindsets (November 2019), MDRC

Effects of Word Generation on academic language, vocabulary and reading comprehension outcomes

A study published in the Journal of Research on Educational Effectiveness reports on the impact of Word Generation on academic language, vocabulary and reading comprehension outcomes for pupils in grades 4 to 7 (Years 5 to 8).

Word Generation (WG) is a vocabulary programme designed to teach academic vocabulary words through English, maths, science and social studies classroom activities. For this study, 7,725 fourth to seventh grade pupils from 25 schools in the northeast US were randomised within pairs to either treatment or business-as-usual control conditions. In treatment schools, the programme was implemented throughout the school year. In grades 4 and 5 (Years 5 and 6), this involved 12 ten-day long units of 45-50 minutes per day. For grades 6 and 7 (Years 7 and 8), the programme was implemented in six-week long units designed to take 45 minutes each day in science and social studies classes.

At the end of the first year, pupils in grades 4 and 5 also made improvements on their academic language skills (ES = +0.06), and in their reading comprehension at the end of the second year (ES = +0.15). Reading comprehension also improved at the end of the second year for pupils in grades 6 and 7 (ES = +0.10).

The study also showed gains on tests of the specific words emphasised in the programme, but these effects are considered potentially inflated.

Source: Experimental effects of Word Generation on vocabulary, academic language, perspective taking and reading comprehension in high-poverty schools (August 2019), Journal of Research on Educational Effectiveness, 12:3

Little evidence of the effectiveness of CPD

This Campbell systematic review looks at the effect of continuing professional development (CPD) approaches for education professionals on educational and social outcomes for children, and also any effects on professional practice.

The review summarises evidence from 51 studies, including 48 randomised controlled trials, however, only 26 studies were included in the meta-analysis. The 51 studies were grouped into three CPD areas: social and emotional development interventions, language and literacy development interventions, and stress reduction interventions.

The main findings of the review were:

  • No effect of CPD on social and emotional development interventions on pupil academic outcomes. The weighted average effect size = +0.05.
  • No effect of CPD language and literacy development interventions on pupil academic outcomes. The weighted average effect size = +0.04.
  • It was not possible to draw any conclusions concerning the effectiveness or ineffectiveness of CPD on social and emotional development or language and literacy development interventions on teacher outcomes.
  • As there was only one study in the CPD category of stress reduction interventions, it was not possible to draw any conclusions.

The researchers conclude that there is insufficient evidence for conclusions to be drawn, with the exception of language and literacy development interventions. For this type of CPD, there seems to be no positive impact on pupil academic outcomes.

Source: Effectiveness of continuing professional development training of welfare professionals on outcomes for children and young people: A systematic review (November 2019), Campbell Systematic Reviews