Examining the impact of content-intensive teacher professional development

A new evaluation report by Michael S Garet and colleagues, published by the US Institute of Education Sciences, examines the impact of providing elementary (primary) school teachers with content-focused maths continuing professional development (CPD) on their knowledge, teaching, and students’ achievement.

The study’s CPD had three components totalling 93 hours. The core of the CPD was Intel Math, an 80-hour workshop delivered in the summer of 2013 that focused on deepening teachers’ knowledge of grades K–8 mathematics (Years 1 to 9). Two additional CPD components totalling 13 hours were delivered during the 2013–14 school year.

The study’s sample included grade 4 (Year 5) teachers from 94 schools in six US school districts and five states that were randomly assigned within schools to either a treatment group that received the study CPD or a control group that did not receive the study CPD.
Key findings were as follows:

  • The CPD had a positive impact on teacher knowledge. On average, treatment teachers’ maths knowledge scores on a study-administered maths assessment were 21 percentile points higher than control teachers’ scores in spring 2014, after the CPD was completed.
  • The CPD had a positive impact on some aspects of teaching practice, particularly Richness of Mathematics, which emphasises the conceptual aspects of maths, such as the use and quality of mathematical explanations.
  • Despite the CPD’s generally positive impact on teacher outcomes, the CPD did not have a positive impact on student achievement. On average, treatment teachers’ students scored 2 percentile points lower than control teachers’ students in spring 2014 on both a study-administered maths assessment aligned with the content of the CPD and the state maths assessment. This difference was statistically significant for the state maths assessment but not for the study-administered assessment.

Source: Focusing on Mathematical Knowledge: The Impact of Content-Intensive Teacher Professional Development (2016), Institute of Education Sciences

Using research to improve teaching practice

The Education Endowment Foundation has reported on two studies that looked at using education research to improve teaching practice.

Research into Practice was a pilot intervention aimed at supporting teachers to use evidence-based teaching and learning strategies to improve student progress. The project ran for a year in ten primary schools in Rochdale (north-west England). It involved professional development sessions and direct consultant support to help teachers:

  • Have more positive views about the use of research for improving teaching and learning;
  • Apply education research findings in the classroom and at a strategic level; and
  • Establish a stronger culture of evidence-based inquiry and practice.

There were some positive changes in teachers’ attitudes toward research. However, there was no evidence that teachers were more likely to use research evidence to inform their teaching practice.
The Research Champions project used a senior teacher based at one of five schools to work with research leads, other teachers, and senior leaders to promote engagement with research evidence. There were “audits” of school research needs, research symposia for teachers, periodic research and development forums, and personalised support. However, there was no evidence that teachers’ attitudes toward research, or their use of research, changed during the intervention.

Source: Research into Practice – Evidence-informed CPD in Rochdale and Research Champions (2016), Education Endowment Foundation.

Targeting EALs with science

A recent large-scale randomised controlled trial, published in the American Educational Research Journal, has examined the impact of a science curriculum with a focus on pupils with English as an Additional Language (EALs).

The study was implemented in 66 schools (33 treatment and 33 control) across three school districts in one south-eastern US state. During the 2012–2013 school year, the project involved 258 teachers (123 treatment and 135 control) and a total of 6,673 students. The trial evaluated P-SELL, a science curricular and professional development intervention for fifth-grade students with a focus on EALs.

The P-SELL curriculum’s approach aligns with state science standards and high-stakes science assessments administered at fifth grade. It is based on an inquiry-oriented approach and addresses the learning needs of EALs by providing guidance and scaffolding for English language development. Teachers are supported with a teacher’s guide and professional development workshops. The workshops incorporated critical features of effective professional development: content focus, active learning, coherence, sufficient duration, and collective participation.

The study used both the high-stakes state science assessment as an outcome measure and a researcher-developed science assessment that was administered at the beginning and end of the year and allowed for a pre-measure of science achievement. The study examined the effect of the intervention on science achievement for all students and for students of varying levels of English proficiency (EAL, recently reclassified EAL, former EAL, and non-EAL).

The results found significant and meaningfully sized average intervention effects on the researcher-developed science assessment scores (effect size = +0.25) and the state science assessment scale scores (+0.15). The P-SELL intervention had significant and meaningfully sized effects for EALs (+0.35) on the researcher-developed assessment. The intervention effects were positive but not statistically significant for EALs (+0.12) on the state science assessment, although other subcategories (non-EALs and former EALs) were positive and significant. This is the first year of a three year study, and future years will provide information on the long-term impact of the teachers’ professional development.

Source: Impact of a Large-Scale Science Intervention Focused on English Language Learners (2016), American Educational Research Journal.

Is professional development better than being dismissed?

The last issue of Best Evidence in Brief reported on a study in which low-performing teachers were dismissed. A new working paper from the National Bureau of Economic Research reports on an experiment where low-performing teachers were provided with coaching from higher-performing peers.

The experiment took place in Tennessee in 14 elementary and middle schools. Tennessee teachers are observed in the classroom many times each year, and scored on 19 specific skills (eg, questioning, lesson structure and pacing, and managing student behaviour). Schools were randomly assigned to a treatment condition or business-as-usual control group. In the treatment schools, low-performing “target” teachers were matched with high-performing teachers, based on the outcomes of their classroom observations. The high-performing teachers were chosen based on their high scores in skills for which the low-performing teachers had received a low score. The pairs were encouraged to work together on these skills, as well as more generally on observing each other’s teaching, discussing strategies for improvement, and following up on each other’s commitments throughout the year.

After a year, students in treatment schools (whether taught by target or non-target teachers) showed a small improvement (effect size +0.06) on maths and English tests, when compared with students in control schools. Gains by students taught by target teachers were higher (+0.12). These improvements persisted and grew. In the following year, the effect for target teachers was a marginally significant +0.25.

Source: Learning Job Skills from Colleagues at Work: Evidence from a Field Experiment Using Teacher Performance Data (2016), The National Bureau of Economic Research.

Video playback pays back dividends

Researchers at Harvard University are conducting a study called The Best Foot Forward Project to determine the accuracy and usefulness of teacher observation using video rather than in person. A new report describes the first-year results of a randomised controlled trial of the project.
As part of the study, 162 teachers were randomly assigned to an experimental group instructed to video their classroom performance for a year. They were then asked to send five clips of their choice for feedback to 51 randomly assigned administrators, who had received training in video observation. The administrators were each assigned three teachers to evaluate. The results were then compared to 50 administrators and 185 teachers assigned to a control group who underwent in-person observation, as they had done in the past. Teachers were matched on years of experience, race/ethnicity, gender, and their schools’ test scores.
Results of the first year of implementation included:
  • Videoed teachers were more likely than controls to report that the post-observation feedback from administrators was fair.
  • Videoed teachers were more likely than controls to change classroom practice as a result of post-observation feedback.
  • Videoed teachers rated their performance as lower than control teachers. They commented that they noticed behaviours when watching themselves on video that they wouldn’t have noticed otherwise.
A smaller study was completed to determine if using videos chosen by a teacher could mask their true performance. A group of external observers used the Classroom Assessment Scoring System to compare videos that teachers did not submit to videos that they did. Results showed that teachers’ strengths and weaknesses were consistent among submitted and not-submitted videos.
Researchers concluded that video observations offer several advantages over in-person observation: it reduces teacher anxiety and increases their perceptions of fairness; it promotes more congenial post-observation meetings between administrators and teachers than in-person observation; videoed teachers are more likely to make behavioural changes; and it allows administrators to perform evaluations on their schedule. The study is continuing and will examine the effects on pupil achievement of teacher behavioural changes following video observations and feedback.
Source: The Best Foot Forward Project – Substituting Teacher-Collected Video for In-person Classroom Observations: First Year Implementation Report (2015), Harvard University.

Early results in for teacher leadership programme

A US programme intended to boost pupil achievement by providing teachers with two years of professional development, including formal training sessions and meetings with a leadership coach, is showing signs of potential, according to a new RAND Corporation report.

The Leading Educators (LE) Fellowship programme, selects mid-career teachers through a competitive application process. To examine the impact of LE, researchers are comparing pupil achievement gains for teachers who participated in the programme as fellows or mentees with the pupil achievement gains of other teachers.According to RAND, early findings of the programme are mixed, but suggest that it shows promise in improving pupil achievement. Specifically, they report:

  • Among fellows, there are both some statistically significant positive and negative programme effects on pupil achievement, with results that vary across states, subject areas, and model specifications.
  • Among mentee teachers, for whom sample sizes are larger, there is some suggestive evidence of impacts on pupil achievement — in particular, marginally significant and significant positive programme effects among mentees who teach maths and social studies, respectively, in Louisiana.
  • The impact of the programme on teacher retention is unclear, with no consistent pattern of retention impacts across cohorts or states.

The authors note that the current results are based on few years of data and on a small sample of teachers, and results may change when there are more fellows and mentored teachers included in future studies.

Source: Examining the Early Impacts of the Leading Educators Fellowship on Student Achievement and Teacher Retention (2015), RAND Corporation.