AmeriCorps volunteers and pupil maths achievement

AmeriCorps is a US organisation that trains volunteers to serve the community in various civically-minded ways. A recent evaluation examined the effects on pupils’ maths achievement of training AmeriCorps volunteers to teach maths strategies to struggling maths pupils in grades 4–8 (Years 5–9). The volunteers used scripted protocols to teach three maths strategies to struggling pupils. Each strategy was studied in prior research and shown to have positive effects on achievement: concrete-representational-abstract, which uses concrete objects to teach concepts; cover-copy-compare, which teaches steps for computation and provides practice; and cognitive-strategy instruction, which teaches pupils to use procedures and reasoning to solve word problems.

AmeriCorps volunteers had to agree to a year-long, full-time commitment and received four days of training before starting the intervention, with additional training one and two months after. Each school received at least one volunteer from AmeriCorps, who was mentored by one school-staff member who was fully trained in the programme.

Subjects were 489 pupils in 150 Minnesota schools who were randomly assigned to either receive the intervention at the start of the school year (n=310), or to a control group who would receive the intervention a few months later (n=179). All pupils had scored below proficient in the prior year’s state maths assessment. During the intervention, pupil pairs with similar maths scores were to receive maths support for 90 minutes a week for a term. Post-tests using STAR Math were analysed two ways: the intent-to-treat analysis included all pupils who received the intervention, and showed significant positive effects as compared to the control group (effect size = +0.17); and the optimal dosage analysis that included pupils who received the targeted 12 weeks of intervention for at least an hour a week. Effect sizes for the experimental group increased to +0.24 when pupils were given the optimal dosage.

Source: Evaluation of a math intervention program implemented with community support (May 2019), Journal of Research on Educational Effectiveness, DOI: 10.1080/19345747.2019.1571653

Open Court Reading receives judgement in multi-year trial

A multi-year scale-up study has examined the effectiveness of Open Court Reading (OCR), a phonics-based curriculum for grades K-6 (Reception to Year 7).

The study, by Michael Vaden-Kiernar and colleagues, and published in the Journal of Research on Educational Effectiveness, was a school-level, cluster randomised trial, involving around 4,500 pupils and 1,000 teachers in 49 elementary schools across the US.

The OCR curriculum includes pupil materials, teacher manuals, diagnostic and assessment tools and test preparation practice guides. In all grades (K-5), the instructional format is a three-part lesson with specific instruction in phonemic awareness and phonics, vocabulary and comprehension skills and writing skills. The programme includes one- or two-day summer workshops at the start of each school year to train teachers on programme implementation, and ongoing support by OCR reading consultants.

An implementation study showed adequate to high levels of fidelity to the programme. However, there were no statistically significant effects on reading performance in Year 1, and a small negative effect (effect size = -0.09) in Year 2. Relative to the “business-as-usual” controls, no positive overall impacts of OCR and mixed impacts for pupil subgroups were found.

Source: Findings from a multiyear scale-Up effectiveness trial of Open Court Reading (June 2017), Journal of Research on Educational Effectiveness

Comprehension Circuit Training delivered via tablet shows reading benefits

Comprehension Circuit Training (CCT) is a programme for teenagers designed to improve reading comprehension through a set of circuit-like exercises in pre-reading, reading, and after-reading to improve foundational reading skills and text-processing abilities. A recent randomised study investigated the effects of CCT delivered electronically via tablet on the reading comprehension of struggling teenage readers.

Three schools in Texas in the US, involving 3 teachers and 228 struggling sixth- to eighth-graders (Years 7–9), participated in this study. Using a within-teacher design, middle school teachers’ reading intervention classes were randomly assigned to electronic CCT (n=9 classes, 112 pupils) or business as usual (n=7 classes, 116 pupils). All pupils had failed to score at the “proficient” level on the prior year’s state reading assessment, and no significant pre-test differences were found between the two groups. CCT pupils received 39 e-CCT lessons in word reading, vocabulary, and reading comprehension that were organised into ten levels delivered in a standard sequence. Each 50-minute lesson contained four video-instruction components – an Opening Comprehension Circuit, WarmUp Station, Reading Core Station, and Knowledge Flex Station – delivered three days a week via Apple iPad. After video instruction, pupils partnered to practice lesson content, with teacher-led assessment occurring in the Knowledge Flex Station.

Results showed statistically significant effects in favour of the experimental group on post-test measures of reading comprehension, vocabulary, and silent reading efficiency. Pupils who entered with lower-level reading comprehension showed the greatest gains.

Source: Impact of a technology-mediated reading intervention on adolescents’ reading comprehension. Journal of Research on Educational Effectiveness, Vol. 10, 2.

Science professional development and pupil achievement: A cluster-randomised trial

Joseph Taylor of Abt Associates and colleagues conducted a rigorous study of the Science Teachers Learning Through Lesson Analysis (STeLLA) professional development (PD) programme.

STeLLA is designed to increase elementary (primary) teachers’ science knowledge. Instead of the standard practice of teaching pupils to memorise science concepts and then perform activities that prove these concepts, STeLLA teachers lead pupils to discover science concepts through experience and experimentation. One of STeLLA’s main tenets is to have pupils think through science problems aloud so that teachers can respond to pupils’ ideas and guide them to scientific conclusions and specific learning goals. Its other distinguishing feature is that during the course of a year, groups of 5–10 teachers led by a PD coach watch and critique videos of experienced science teachers’ lessons, later moving on to their own and their colleagues’ lessons, to analyse them regarding science content, teaching and learning. In addition, STeLLA teachers are taught by university-level science teachers the summer prior to implementation to provide them with greater science content knowledge, a process called “content deepening”.

In the current study, researchers used a cluster-randomised design to compare STeLLA to The Content Deepening Program, a PD programme that deepens teachers’ science knowledge through university faculty-led science teaching, like STeLLA does, but without STeLLA’s analysis-of-practice component. Seventy-seven schools, with 144 teachers and 2,823 fourth and fifth grade pupils (Years 5 and 6) in Colorado, were randomly assigned either to STeLLA (n=42 schools) or to The Content Deepening Program (n=35 schools) in two cohorts, the first in 2011–12 and the second in 2012–13. Teachers in both conditions experienced 88 hours of PD and had the same learning goals for their pupils. Pupils were pre- and post-tested on a science measure based on established assessments. Although the control group demonstrated a slight achievement advantage at baseline, results showed that pupils in STeLLA classes scored higher (effect size = +0.55) at post-test than pupils in classes whose teachers had been through The Content Deepening Program.

Source: The effect of an analysis-of-practice, videocase-based, teacher professional development program on elementary students’ science achievement (2017),  Journal of Research on Educational Effectiveness, Volume 10:  Issue 2

Effective vocabulary teaching

A recent study in the Journal of Research on Educational Effectiveness looks at the effects of Rich Vocabulary (RVOC) teaching on primary pupils’ vocabulary and reading comprehension in a multi-cohort randomised trial.

Rich Vocabulary is an approach where children are exposed to frequent and varied encounters with specific vocabulary words incorporated into classroom teaching. The hope is that this carries over into increased reading comprehension.

The study took place over three years in fourth and fifth grade (Years 5 and 6) classrooms in the northwest US and Canada. A total of 1,232 pupils were assigned to treatment (n=627) or control (n=605) classes from 61 classrooms at 24 schools. This study used at least 12 exposures per word per week. Pupils were given vocabulary instruction 30 minutes per day for four days, with a 10-minute quiz on the fifth day using two novels, A Long Way from Chicago and Maniac Magee, over the course of 14 weeks each year. The pupils had seven weekly reading assignments per book, and teachers were provided with all necessary materials to teach related lessons (worksheets, overheads, etc).

Pupils were pretested in the autumn, and post-tested in the spring each year. Preliminary analysis showed that RVOC treatment pupils scored significantly lower at pretest than control students. Authors suggest this was due to the randomisation assignments, and that they controlled for this in their analyses. Results showed that on norm-referenced tests, the RVOC group scored better than controls in tests of targeted vocabulary, but did not improve general vocabulary knowledge or general reading comprehension.

Source: Efficacy of Rich Vocabulary Instruction in Fourth- and Fifth-Grade Classrooms (2015), Journal of Research on Educational Effectiveness, 8(3).