The impact of web-based coaching on mathematics instruction

A new working paper, published by Brown University, reports on an online coaching programme that aimed to support maths teachers.

Mathematical Quality of Instruction (MQI) is an observational instrument that helps structure teachers’ and coaches’ reflections about maths teaching. There are four “dimensions” of the instrument – richness of the mathematics, Common Core-aligned pupil practices, working with pupils and mathematics, and teacher errors. In several studies, teachers’ scores on MQI have predicted pupils’ academic achievement gains.

In MQI coaching, teachers selected an element of their practice to work on and filmed one of their lessons. A coach then selected a couple of extracts and chose a comparison stock film clip. The teacher watched the clips and then the two discussed them and developed a plan for improvement.

In the current trial, 142 upper elementary and middle school teachers (Years 4-9) from 51 schools in a mid-western state in the US were assigned to MQI coaching or a control condition. Teachers assigned to MQI coaching took part in a two-day summer school, followed by a bi-weekly coaching cycle for the following academic year.

At the end of that year, the coached teachers showed substantial improvements in their scores on the MQI instrument. They also had improved scores in pupil perceptions of classroom practices. However, there was no measurable impact on state achievement tests (the Smarter Balance Assessment Consortium (SBAC)). In the follow-up year, there were still large impacts on the four MQI dimensions, but none were found on pupils’ assessment of teachers or pupil achievement.

Source: Developing ambitious mathematics instruction through web-based coaching: A randomized field trial. (November 2018). Brown University Working Paper

Reform needed for early years initiative

As part of their Straight Talk on Evidence initiative, the Laura and John Arnold Foundation (LJAF) has released a report that discusses new findings from a randomised controlled trial (RCT) of Tennessee’s voluntary pre-kindergarten programme for low-income children.

The Voluntary Pre-K (VPK) initiative provides Tennessee’s four-year-old children—with an emphasis on four year olds who are at-risk—an opportunity to develop school readiness skills (pre-academic and social skills). The study randomly assigned 3,131 eligible children who applied for admission at one of 79 oversubscribed VPK programmes across the state to either a programme group that was offered admission or a control group that was not (but could access other available child and family services in the community). Pupil achievement and other outcomes were measured in third grade (Year 4) using state educational records.

According to the LJAF report, the study found positive short-term effects on achievement (at the end of the pre-k year), but these effects dissipated as children entered elementary (primary) school and turned modestly negative by third grade (Year 4). At the third-grade follow-up, the control group scored significantly higher in maths and science achievement than the pre-k group.

The report offers possible reasons for the adverse effects, and suggests that the programme be reformed by incorporating evidence-based funding criteria aimed at improving its effectiveness over time.

Source: Large randomized trial finds state pre-k program has adverse effects on academic achievement. Reform is needed to increase effectiveness. Straight Talk on Evidence, The Laura and John Arnold Foundation

Urban violence and pupil performance

A new study published in Sociology of Education finds that children who attend school with many pupils from violent neighbourhoods can earn significantly lower test scores than peers with classmates from safer areas. The lead author of the study was sociologist Julia Burdick-Will from Johns Hopkins University.

Burdick-Will studied pupils who attended Chicago Public Schools from 2002 to 2010, analysing administrative data from the school system, crime statistics from the Chicago Police Department and school surveys from the University of Chicago Consortium on Chicago School Research. She looked at five cohorts of pupils who were freshmen (Year 10) between the autumn of 2002 and 2006, and followed each pupil for up to four years. Results indicated that in schools where more pupils have a high exposure to violence, their classmates score as much as 10 percent lower on annual standardised maths and reading tests.

According to the report, the study shows that when pupils experience higher levels of neighbourhood violence, the whole school reports feeling less safe, having more disciplinary problems and feeling less trust in their teachers.

Source: Neighborhood violence, peer effects, and academic achievement in Chicago (June 2018), Sociology of Education DOI: 10.1177/0038040718779063

 

Maths anxiety, working memory and self-concept

A study conducted by researchers at the University of Jaén, Spain, and published in the British Journal of Educational Psychology looks at the relationship between maths anxiety and maths performance in primary school children, and also the possible mediating role of working memory and maths self-concept.

A total of 167 pupils in grades 3 and 5 (age 8–12 years) took part in the study. Each pupil completed a set of questionnaires to assess maths anxiety and self-concept as well as their mathematical performance. Working memory was assessed using two backward span tasks. Teachers were also asked to rate each pupils’ maths achievement.

As expected, results showed that pupils who demonstrated higher levels of anxiety about maths tended to have lower scores on maths outcomes such as ability, problem‐solving and teacher‐rated maths achievement. However, this relationship was lessened once the effects of working memory and self-concept were considered. The researchers suggest, therefore, that it is worth taking into consideration working memory and self-concept when designing interventions aimed at helping pupils with maths anxiety.

Source: Math anxiety and math performance in children: The mediating roles of working memory and math self‐concept (May 2017), British Journal of Educational Psychology, Volume 87, Issue 4

Later school start time may produce benefits but more evidence is needed

Many studies have been conducted to examine the impact for pupils of later school start times, some of which can be found in previous issues of Best Evidence in Brief. This Campbell systematic review summarises the findings from 17 studies to examine the evidence on the impact of later school start times on pupils’ mental health and academic performance.

The studies included in the review were randomised controlled trials, controlled before-and-after studies and interrupted time series studies with data for pupils aged 13 to 19 and that compared different school start times. The studies reported on 11 interventions in six countries, with a total of almost 300,000 pupils.

The main results of the review suggest that later school starts may be associated with positive academic benefits and psychosocial outcomes. Later school start times also appear to be associated with an increase in the amount of sleep children get. Effect sizes ranged from +0.38 to +2.39, equivalent to an extra 30 minutes to 2 hours of sleep each night. However, the researchers point out that, overall, the quality of the body of evidence is very low, and so the effects of later school start times cannot be determined with any confidence.

Source: Later school start times for supporting the education, health, and well-being of high school students: a systematic review (December 2017), A Campbell Systematic Review 2017:15

Examining the research on charter schools

The What Works Clearinghouse (WWC) has released a new topic area that focuses on the impact of charter schools on pupil achievement and other outcomes. As part of the launch, the WWC released three intervention reports, which review available research on an intervention or programme to determine if there is strong evidence that it has a positive impact on pupil outcomes.

The intervention reports examine the following three programmes:

  • Knowledge is Power Program (KIPP), a non-profit network of more than 200 US public charter schools educating early childhood, elementary, middle and high school pupils. According to the WWC intervention report, research shows that KIPP had positive effects on mathematics achievement and English language achievement, and potentially positive effects on science achievement and social studies achievement for middle and high school pupils (Years 7 to 13), and no discernible effects on pupil progression (eg, high school graduation within four years of grade 9 (Year 10) entry) for high school pupils.
  • Green Dot Public Schools, a non-profit organisation that operates more than 20 public charter middle and high schools in California, Tennessee and Washington. The WWC reports that Green Dot Public Schools had potentially positive effects on mathematics achievement, pupil progression, school attendance and English language achievement for high school pupils (Years 10 to 13).
  • Harlem Children’s Zone (HCZ) Promise Academy Charter Schools, a non-profit organisation designed to serve low-income children and families living in Harlem in New York City. According to the intervention report, the WWC is unable to draw any conclusions based on available research about the effectiveness or ineffectiveness of the HCZ Promise Academy Charter Schools on elementary, middle and high school pupils. Research that meets WWC design standards is needed to determine the effectiveness or ineffectiveness of this intervention.

Source:  Charter Schools, What Works Clearinghouse