Study shows benefits of Healthy Harlem programme

Mathematica Policy Research posted a new research brief that summarises findings from a study of Healthy Harlem, an after-school programme aimed at promoting healthy lifestyles. The study, by James Mabli, Martha Bleeker and Mary Kay Fox, showed that participation in the Healthy Harlem programme led to increased physical activity and improved weight status for overweight and obese pupils.

Key components of Healthy Harlem include physical activity, healthy snacks, nutrition education lessons and parent workshops. To assess Healthy Harlem’s effectiveness, the authors monitored pupils at 21 after-school sites during an initial baseline year and then measured programme impacts after two and three years of participation. They collected data through a pupil survey, a fitness test and direct measurements of height and weight. Key findings were as follows:

  • A 5.5 percent decrease in mean BMI z-scores after two years of participation and a 9.0 percent decrease after three years of participation. According to the report, a BMI z-score reflects the number of standard deviations a pupil’s BMI is from the mean BMI for a reference population.
  • A decrease of 12.2 percentage points in the percentage of pupils who were overweight or obese after two years, and a decrease of 18.4 percentage points after three years.
  • An increase in the percentage of pupils considered to be within the Harlem Fitness Zone, a measure of fitness based on a pupil’s ability to complete a minimum number of laps, defined for age-and-gender subgroups.

Source: The impact of Healthy Harlem on the body mass index and weight status of adolescents after two and three Years (March 2018), Mathematica Policy Research

Does consistency in programming lead to academic gains in KIPP?

While numerous studies show positive immediate effects of pre-k (Reception), studies also show that these effects usually fade as soon as kindergarten or first grade (Year 1 and Year 2). To discover if consistency in programming from pre-school to elementary (primary) school can extend these positive effects, Virginia Knechtel and colleagues at Mathematica Policy Research (MPR) recently performed the first randomised study of the effectiveness of The Knowledge is Power Program (KIPP) pre-school on second graders (Year 3) who had continued with the KIPP programme into elementary school.

KIPP is a network of 200 elementary and secondary charter schools serving 80,000 pupils in the US, most of whom are low-income and African American or Latino. Admission to KIPP is via lottery. KIPP schools emphasise academics and character development in safe environments that foster pupils’ progression to further education. As part of an i3 scale-up grant, MPR performed a randomised study on the effects of KIPP on elementary to high school pupils, and found positive, statistically significant effects for KIPP pupils. For the pre-school study, the researchers drew their population sample from pupils in the 2015 study who had started KIPP in pre-K (n=97), comparing them to pupils who did not win the KIPP lottery and attended other schools (n=147). At the end of second grade (Year 3), when most pupils had attended KIPP for five years, both reading and maths scores were higher on subtests of the Woodcock-Johnson for KIPP pupils than for control pupils (ES=+0.43 on Letter Word ID, +0.21 for Passage Comprehension, +0.34 for Applied Problems and +0.31 for Calculation). This is not a lasting effect of pre-k, but the cumulative impact of everything KIPP schools did in grades pre-K-2 (Reception to Year 3).

Authors interviewed school staff and identified six key factors that differed between KIPP and non-KIPP programmes. These included that the schools’ structures allow for continuity among year groups, KIPP pre-K is academically focused, and there is a conscious effort to build relationships between school staff and pupils’ families.

Source: Pre-kindergarten impacts over time: An analysis of KIPP charter schools (August 2017), Mathematica Policy Research

Playworks at the playground

Research has shown that children who engage in at least moderate physical activity during breaks demonstrate both health and academic benefits. Studies also note that boys engage in higher physical activities during breaks than girls.

A study by Mathematica Policy Research (MPR) examined the effects of introducing Playworks, a structured-play programme, on girls’ activity levels at breaks. The Playworks programme uses a coach to introduce games and equipment and organise activities during breaks and encourage girls to be less sedentary.

Twenty-nine schools in six US cities were randomly assigned to receive Playworks (n=17) or no intervention (n=12) during the 2010-11 or 2011-2012 school years. Schools were randomly assigned within blocks matched by size, grade levels, ethnicity, and free-lunch count. A total of 1,573 fourth and fifth grade (Years 5 and 6) students participated, 823 girls and 750 boys. After one year of each treatment group’s exposure to Playworks, all experimental and control students wore accelerometers to measure physical activity levels for 10 minutes each break during a one-week period in each school.

Results yielded significant increases for the Playworks groups in girls’ activity levels as compared to the control group, but no significant increases for boys. The authors discuss that modifying the programme to increase boys’ activity would be beneficial.

An earlier randomised study of Playworks by MPR showed positive effects of the programme in reducing bullying, increasing social climate and feelings of school safety, and students’ readiness to learn.

Source: The impact of Playworks on boys’ and girls’ physical activity during recess (2015), Mathematica Policy Research

Teach For America

Can inexperienced teachers in the non-profit Teach For America (TFA) programme perform as well as teachers who qualified using regular routes? A new report from Mathematica Policy Research says they can.

TFA recruits and trains teachers to work in low-income schools in the US and in 2010 launched an expansion. By the second year of the scale-up TFA had increased its placements of first- and second-year TFA teachers by 25%. Mathematica Policy Research evaluated this expansion in a study of 2,000 students, 156 teachers, and 36 schools across 10 states. The TFA teachers had an average of 1.7 years of experience compared with 13.6 years for the comparison teachers.

The key findings were:

  • Despite their lack of experience, TFA teachers in elementary school were as effective as other teachers in teaching maths and reading at the same high-poverty elementary schools.
  • TFA teachers in lower elementary were more effective at teaching reading than their non-TFA colleagues. The statistically significant positive effect on student reading was equivalent to an additional 1.3 months of school.
  • There were no statistically significant effects for other groups of elementary school TFA teachers.

Previous studies of TFA teachers found them to be equally effective as other teachers at teaching reading in elementary school and more effective at teaching maths at all grade levels. The new evaluation is consistent with the previous studies on teaching reading at elementary school but did not find that TFA teachers were more effective at teaching maths than other teachers.

Source: Assessing the Effectiveness of the Teach For America i3 Scale-Up (2015), Mathematica Policy Research.

The impact of alternative teacher training programmes on pupil achievement

New research by Mathematica Policy Research has assessed the mathematics achievement of pupils taught by teachers from two highly selective recruitment and training schemes that run in the US – Teach for America (TFA) and Teaching Fellows. TFA works with graduates from some of the best universities and places them for two years. Teaching Fellows recruits both graduates and professionals looking to change careers, and expects participants to make a long-term commitment to teaching. Both schemes place their teachers in hard-to-staff schools in deprived areas.

The study took place in 2009/10 and 2010/11 in schools identified as having two or more classes that would be teaching the same maths course. At the beginning of the year, pupils in each school (n=8,689) were randomly allocated either to a class taught by a TFA or Teaching Fellow teacher, or to a class taught by a comparison teacher (who entered teaching through traditional or other, less selective programmes). Exams taken at the end of the year showed that TFA teachers produced gains significantly greater than teachers who came through traditional teaching programmes or other alternative but less selective certification schemes, but the effect size (ES=+0.07) was very small. Still, this was estimated to be the equivalent of an additional 2.6 months of school for the average pupil nationwide. In contrast, there were no differences in maths outcomes between Teaching Fellows and their controls.

In the UK, Rebecca Allen from the Institute of Education (IOE) presented findings of new research into Teach First at the BERA Conference. Like TFA, Teach First places graduates with good degrees in challenging classrooms. The IOE study found that the introduction of Teach First teachers produced no school-wide gain in the first year, but in years two and three there were gains equivalent to a boost of one grade in one of a pupil’s eight best subjects.

Source: The Effectiveness of Secondary Math Teachers from Teach For America and the Teaching Fellows Programs (2013), Institute of Education Sciences.

New report is positive on KIPP middle schools

The Knowledge Is Power Program (KIPP) is a chain of US charter schools that aims to improve education for low-income children. A new report by Mathematica Policy Research says that the impact on student achievement in KIPP middle schools (for ages 11-14) is positive, statistically significant and educationally substantial. The report compared the progress of children in 43 KIPP middle schools with that of their former peers from elementary school.

The two groups of children were similar, although KIPP students were less likely to have received special education services or have limited proficiency in English. The estimated gains made by KIPP students over the three years were substantial, with an effect size of +0.36 for maths (equivalent to an additional 11 months learning) and +0.21 for reading (equivalent to an extra 7 months). There were few differences in student behaviour and attitude, but KIPP students did complete around 50% more homework than their peers. KIPP schools also have an extended school day (on average more than nine hours), but the study found that KIPP schools with the longest days produced less impact on achievement, perhaps because the extra time was spent on non-core academic activities.

Source: KIPP Middle Schools: Impacts on Achievement and Other Outcomes Final Report (2013), Mathematica Policy Research.