Video playback pays back dividends

Researchers at Harvard University are conducting a study called The Best Foot Forward Project to determine the accuracy and usefulness of teacher observation using video rather than in person. A new report describes the first-year results of a randomised controlled trial of the project.
As part of the study, 162 teachers were randomly assigned to an experimental group instructed to video their classroom performance for a year. They were then asked to send five clips of their choice for feedback to 51 randomly assigned administrators, who had received training in video observation. The administrators were each assigned three teachers to evaluate. The results were then compared to 50 administrators and 185 teachers assigned to a control group who underwent in-person observation, as they had done in the past. Teachers were matched on years of experience, race/ethnicity, gender, and their schools’ test scores.
Results of the first year of implementation included:
  • Videoed teachers were more likely than controls to report that the post-observation feedback from administrators was fair.
  • Videoed teachers were more likely than controls to change classroom practice as a result of post-observation feedback.
  • Videoed teachers rated their performance as lower than control teachers. They commented that they noticed behaviours when watching themselves on video that they wouldn’t have noticed otherwise.
A smaller study was completed to determine if using videos chosen by a teacher could mask their true performance. A group of external observers used the Classroom Assessment Scoring System to compare videos that teachers did not submit to videos that they did. Results showed that teachers’ strengths and weaknesses were consistent among submitted and not-submitted videos.
Researchers concluded that video observations offer several advantages over in-person observation: it reduces teacher anxiety and increases their perceptions of fairness; it promotes more congenial post-observation meetings between administrators and teachers than in-person observation; videoed teachers are more likely to make behavioural changes; and it allows administrators to perform evaluations on their schedule. The study is continuing and will examine the effects on pupil achievement of teacher behavioural changes following video observations and feedback.
Source: The Best Foot Forward Project – Substituting Teacher-Collected Video for In-person Classroom Observations: First Year Implementation Report (2015), Harvard University.

Early results in for teacher leadership programme

A US programme intended to boost pupil achievement by providing teachers with two years of professional development, including formal training sessions and meetings with a leadership coach, is showing signs of potential, according to a new RAND Corporation report.

The Leading Educators (LE) Fellowship programme, selects mid-career teachers through a competitive application process. To examine the impact of LE, researchers are comparing pupil achievement gains for teachers who participated in the programme as fellows or mentees with the pupil achievement gains of other teachers.According to RAND, early findings of the programme are mixed, but suggest that it shows promise in improving pupil achievement. Specifically, they report:

  • Among fellows, there are both some statistically significant positive and negative programme effects on pupil achievement, with results that vary across states, subject areas, and model specifications.
  • Among mentee teachers, for whom sample sizes are larger, there is some suggestive evidence of impacts on pupil achievement — in particular, marginally significant and significant positive programme effects among mentees who teach maths and social studies, respectively, in Louisiana.
  • The impact of the programme on teacher retention is unclear, with no consistent pattern of retention impacts across cohorts or states.

The authors note that the current results are based on few years of data and on a small sample of teachers, and results may change when there are more fellows and mentored teachers included in future studies.

Source: Examining the Early Impacts of the Leading Educators Fellowship on Student Achievement and Teacher Retention (2015), RAND Corporation.

Effective CPD should be evidence-based

A new review published by the Teacher Development Trust has analysed existing research into effective professional development for teachers. The authors say their findings emphasise the importance of the use of evidence; both evidence from pupils’ responses to teachers’ developing understanding and practices, and the strength of the evidence and rationale underpinning the CPD activity or programme.

The review also found that those CPD opportunities that are carefully designed and have a strong focus on pupil outcomes have a significant impact on pupil achievement. Successful programmes should be underpinned by:

  • Both subject knowledge and subject-specific pedagogy;
  • Clarity around learner progression, starting points and next steps; and
  • Content and activities dedicated to helping teachers understand how pupils learn, both generally and in specific subject areas.

The authors identified several key design features that increased the likelihood of CPD having a lasting impact on teacher practice and pupils’ outcomes. These were:

  • Appropriate duration. CPD has to be prolonged in order to produce profound and lasting change. The most effective CPD lasted at least two terms, more usually a year or longer.
  • Rhythm. It was important that programmes created a rhythm of follow-up, consolidation, and support activities.
  • Designing for participants’ needs. CPD should create an overt sense of relevance for participants, including recognising the differences between different teachers and their starting points.
  • Creating a shared sense of purpose through a positive professional learning environment, sufficient time, and consistency with participants’ wider context.
  • Alignment across various activities. There should be a logical thread through components of the programme, and consistency between the teacher learning and the principles of pupil learning being promoted.

Source: Developing Great Teaching: Lessons from the International Reviews into Effective Professional Development (2015), Teacher Development Trust.

More time needed for multiplicative reasoning trial?

The Department for Education has published a new report describing a randomised controlled trial of a pilot of the Multiplicative Reasoning Project (MRP). MRP focuses on developing teachers’ understanding and capacity to teach topics that involve multiplicative reasoning to Key Stage 3 (KS3) pupils.

The pilot was delivered by the National Centre for Excellence in the Teaching of Mathematics (NCETM). Approximately 60 teachers in 30 schools engaged in three regional professional development networks led by professional development leaders and supported by university researchers. At school level, the programme was implemented by two core teachers in each school. Approximately 36-54 hours’ worth of high quality research-informed curriculum materials were also created.

The trial ran between October 2013 and June 2014 and involved 8,777 pupils clustered into 418 KS3 mathematics classes. Approximately half the schools, teachers, and pupils participated in the intervention and half formed a control group.

The authors found that the programme did not have any statistically significant impacts on general mathematical attainment or on items on the tests specifically associated with multiplicative reasoning, but suggest that the lack of impact could be due to the timescale of the project. There were a number of positives: The programme was considered effective as a curriculum design project and pilot project for teacher professional development, it had a positive impact on pupils’ relationship with mathematics as reported by teachers in surveys and interviews, and it led to a number of positive changes in teacher beliefs and practices.

Source: Multiplicative Reasoning Professional Development Programme: Evaluation (2015), Department for Education.

Preschool maths and science professional development fails to improve learning

A new article published in the Journal of Educational Psychology describes a study into the impact of professional development on maths and science learning in early childhood education.

For the study, 65 staff from 34 varied early childhood settings in Ohio were randomly assigned to experience 10.5 days (64 hours) of training on maths and science or an alternative topic (art and creativity). The maths and science training was adapted from the Core Knowledge Preschool Sequence, which provides a developmental progression based on early childhood research and theory.

The study looked at both maths and science learning opportunities, and the maths and science learning gains of the children (n=385). In terms of opportunities, the authors found that the professional development significantly impacted on the provision of science learning opportunities, but not maths. However, in terms of learning gains, none were observed.

The authors suggest a number of factors that may have contributed to the outcome. These include the fact that although educators were provided with hands-on opportunities to try new maths and science activities during training, there were no systematic means of ensuring they had regular opportunities to apply these in their classrooms. Also, that changes in practice may be difficult to achieve as the emphasis on these subject areas is relatively new in early childhood education.

Source: Professional Development for Early Childhood Educators: Efforts to Improve Math and Science Learning Opportunities in Early Childhood Classrooms (2015), Journal of Educational Psychology, 107(2).

Effective professional development for teachers of early years

Previous research has shown that the quality of classroom practices, including teacher-child interaction, is a factor in pupil readiness for starting school. Child Trends has released a report describing the results of a three-year randomised study of two professional development models for teacher–child interactions in pre-kindergarten (Reception).

Between 2011 and 2013, 486 pre-kindergarten teachers in the US state of Georgia were randomly selected to participate either in Making the Most of Classroom Interactions (MMCI, n=175), MyTeachingPartner (MPT, n=151), or a control group (n=160). MMCI teachers met in a group with instructors for a series of 10 workshops (one a month), between which they had homework. MTP teachers received one-to-one online coaching, where they sent videos of themselves working in a classroom to a coach who sent them feedback in two-week cycles for seven months. Both the experimental groups and the control group had access to online demonstrations of best practices. Teachers were evaluated in the spring using the Classroom Assessment Scoring System which looks at three teacher–child interaction domains: emotional support, instructional support, and classroom organisation.

Results showed that when compared to the control group, MMCI improved emotional and instructional support, and MTP improved emotional support. When compared to each other, there were no statistically significant differences between MMCI and MTP teachers’ improvements. In terms of practicality, MMCI required less time and staff to implement fully than MTP. The authors note that more research needs to be done to determine the optimal conditions in which to implement each programme.

Source: Georgia’s Pre-K Professional Development Evaluation: Executive Summary (2015), Child Trends