Digital vs. paper reading

A study published in the American Educational Research Journal compares reading processes and outcomes for pupils when reading a text from paper with the same text delivered on a touchscreen laptop.

Amanda P Goodwin and colleagues conducted the study with 371 pupils in grades 5–8 (Years 6–9) from three schools in an urban district in the southeastern US. Pupils were randomly assigned to two conditions: Condition A read the first section of a text on paper, and the second half digitally, whereas pupils in Condition B read the first part digitally and the second part on paper. The content in both conditions was identical. When reading on paper, pupils had access to highlighters, pens and sticky notes; when reading digitally, they had access to digital highlighters, annotating and dictionaries.

Results suggest that pupils highlight and annotate more when reading on paper vs. digital text. Also, reading on paper vs. digitally was slightly supportive of reading comprehension for the longer sections of text, although effect sizes were very small (odds ratio of 1.077).

Source: Digital versus paper reading processes and links to comprehension for middle school students (December 209), American Educational Research Journal

Targeting EALs with science

A recent large-scale randomised controlled trial, published in the American Educational Research Journal, has examined the impact of a science curriculum with a focus on pupils with English as an Additional Language (EALs).

The study was implemented in 66 schools (33 treatment and 33 control) across three school districts in one south-eastern US state. During the 2012–2013 school year, the project involved 258 teachers (123 treatment and 135 control) and a total of 6,673 students. The trial evaluated P-SELL, a science curricular and professional development intervention for fifth-grade students with a focus on EALs.

The P-SELL curriculum’s approach aligns with state science standards and high-stakes science assessments administered at fifth grade. It is based on an inquiry-oriented approach and addresses the learning needs of EALs by providing guidance and scaffolding for English language development. Teachers are supported with a teacher’s guide and professional development workshops. The workshops incorporated critical features of effective professional development: content focus, active learning, coherence, sufficient duration, and collective participation.

The study used both the high-stakes state science assessment as an outcome measure and a researcher-developed science assessment that was administered at the beginning and end of the year and allowed for a pre-measure of science achievement. The study examined the effect of the intervention on science achievement for all students and for students of varying levels of English proficiency (EAL, recently reclassified EAL, former EAL, and non-EAL).

The results found significant and meaningfully sized average intervention effects on the researcher-developed science assessment scores (effect size = +0.25) and the state science assessment scale scores (+0.15). The P-SELL intervention had significant and meaningfully sized effects for EALs (+0.35) on the researcher-developed assessment. The intervention effects were positive but not statistically significant for EALs (+0.12) on the state science assessment, although other subcategories (non-EALs and former EALs) were positive and significant. This is the first year of a three year study, and future years will provide information on the long-term impact of the teachers’ professional development.

Source: Impact of a Large-Scale Science Intervention Focused on English Language Learners (2016), American Educational Research Journal.

The effects of data-based decision making

The use of data to inform educational decisions is becoming increasingly popular worldwide. An article in the most recent American Educational Research Journal describes the effect of a two-year schoolwide data-based decision-making intervention, called Focus, on student achievement.

Focus trains schoolwide teams of teachers and administrators to use data to guide their teaching using a protocol developed at the University of Twente in the Netherlands. Staff receive seven training meetings in year 1 and four training meetings in year 2, and are provided with documents and planning aids to help them track student data and progress.

Fifty-three primary schools (1,193 staff members) in the Netherlands used Focus to apply student achievement data to guide instruction during a two-year study. All schools (n=53) were trained to use data-based decision-making in mathematics during years 1 and 2, and had the option to also use it in spelling lessons in year 2 (n=38). Student achievement data from standardised maths tests given twice a year were collected for children aged 6-12 for two years before implementing Focus and then for two years during the intervention. Results showed benefits of the intervention equal to an extra month of schooling and were most statistically significant for students from low socio-economic backgrounds.

Source: Assessing the Effects of a School-Wide Data-Based Decision-Making Intervention on Student Achievement Growth in Primary Schools (2016), American Educational Research Journal.

Second success launches Number Rockets

A recent article in the American Educational Research Journal describes a scaled-up replication study of a randomised controlled trial evaluating the effects of Number Rockets, a small-group intervention for children aged 6/7 at risk of mathematical difficulties.

A replication study repeats the procedures of an earlier study to see if the effects of the intervention are consistent. If similarly positive results are reproduced, this lends credence to that programme’s efficacy. Replication rates for education research are often low.

In this case, the original study found a positive impact for pupils using Number Rockets in a single US state district under ideal conditions. It was conducted in ten schools and involved 139 pupils (70 in the experimental group, 69 in control) identified as at-risk because of their performance in the lowest fifth on a screening assessment. The current, scaled-up study involved 76 schools across four districts in four US states, with 994 pupils (615 experimental, 379 control) identified by a screening score as being in the sample’s lowest third.

In an effort to more closely replicate real-world implementation, tutors were provided with less support and monitoring than in the original study. The experimental group received the Number Rockets curriculum in groups of 2-3 pupils three or more times a week for 17 weeks in addition to their regular maths curriculum, while the control pupils continued with their current curriculum. The Number Rockets curriculum is scripted, and focuses on concepts and operations with whole numbers, such as addition/subtraction, equality, comparing quantities, number placement on a number line, and also includes 10 minutes of fact practice.

The Number Rockets group outperformed the control group on the TEMA-3 standardised maths test, replicating the findings of the original study. Coincidentally, effect sizes for the experimental group were +0.34 for both the original and current studies.

Source: Intervention for First Graders With Limited Number Knowledge: Large-Scale Replication of a Randomized Controlled Trial (2015), American Educational Research Journal, 52(3).

Teaching academic vocabulary knowledge

Children whose first language is not English have to cope with learning English and academic content in English at the same time. A new study has shown the benefit of specifically teaching academic vocabulary.

The randomised controlled trial (RCT) was carried out in 14 middle schools in California, where 50 teachers were assigned to treatment or control conditions. A total of 2,082 sixth-grade (Year 7) students participated, 71% of whom spoke another language (mostly Spanish) at home. They followed Academic Language Instructions for All Students (ALIAS), a 20-week programme teaching academic vocabulary – words that are not subject-specific but often appear in sixth-grade textbooks (such as expanse, integrated, generate, according to). The programme was supported with teaching materials and professional development.

Students improved their vocabulary knowledge, morphological awareness skills, and comprehension of expository texts that used the academic language that was taught. They also improved their performance on a standardised measure of written language skills (effect size=+0.19). The effects were generally larger for children whose home language was not English and for those who started the intervention with underdeveloped vocabulary knowledge.

Source: Effects of Academic Vocabulary Instruction for Linguistically Diverse Adolescents (2014), American Educational Research Journal 51(6)

Disappointing findings for Responsive Classroom

A new article in the American Educational Research Journal describes a randomised controlled trial (RCT) to examine the efficacy of the Responsive Classroom (RC) approach on pupil achievement. The authors found that pupils taught using RC did not outperform those at schools assigned to the control condition in maths or reading.

RC is a widely used professional development intervention comprising practical teaching strategies designed to support children’s social, academic, and self-regulatory skills. More than 120,000 teachers have been trained in the approach. This trial involved 2,904 children from 24 US schools. They were randomised into intervention and control conditions, and studied from the end of second grade (Year 3) to fifth grade (Year 6).

Results showed that random assignment to RC did not have an impact on achievement outcomes. The authors say that other RCT results linking social-emotional learning (SEL) interventions to SEL outcomes are similarly lacklustre, and that there are several plausible explanations including the trial involving too few schools to detect a small effect. They also note that some outcomes (eg, motivation and engagement) may not adequately translate into outcomes measured by state standardised achievement tests, and that adopting interventions such as the RC approach involves a long process of teacher change ranging from three to five years. Data for this study was gathered during teachers’ first and second years of RC implementation, early in the process of adoption.

Source: Efficacy of the Responsive Classroom Approach: Results From a 3-Year, Longitudinal Randomized Controlled Trial (2014), American Educational Research Journal, 51(3).