Positive effects of a healthy debate

Johns Hopkins University’s Daniel Shackelford has conducted the first quantitative study examining the effects of participation in an extracurricular debate club during pre-adolescence on pupils’ later academic and engagement outcomes, including entry to selective-entrance high schools.

Dr Shackelford examined a 10-year sample of 2,263 4th to 8th grade pupils (Year 5 to Year 9) participating in Baltimore City’s Baltimore Urban Debate League (BUDL) between the 2004 to 2013 school years, comparing their standardised maths and reading scores, attendance, and entry to selective-entrance high schools to 81,906 peers who did not participate in BUDL. Ninety-one percent of both groups were African American, and 96% of both groups received free and reduced-price lunch.

It is of note that these two groups were not matched at baseline: pupils who became debaters differed from controls prior to their participation in BUDL, with higher standardised test scores and attendance, so no true causal conclusion can be drawn from comparing groups. Yet among the debate pupils themselves, results showed that pre-adolescent debate participation yielded more than a 6% increase in reading scores and a 4% increase in maths scores on standardised testing. While debate inherently involves reading and might be accountable for increased reading achievement, Dr Shackelford observes that debaters were 10% more likely not to be chronically absent than non-debaters, and this increased engagement in school may have yielded the improvements in maths scores. BUDL pupils were also more likely to attend a selective high school (+0.12) or selective career tech high school (+0.01) than to attend a traditional high school.

Source: The BUDL effect: Examining academic achievement and engagement outcomes of preadolescent Baltimore Urban Debate League participants (February 2019), Educational Researcher. 

Effects of initial skill and group size in the ROOTS Tier-2 maths intervention

Many factors influence a child’s responsiveness to an academic programme. The University of Oregon’s Ben Clark and colleagues recently evaluated the effects of baseline maths skills and their interaction with group size on the maths achievement of at-risk kindergartners (Year 1) in the ROOTs programme.

The ROOTS programme is a 50-lesson Tier-2 maths programme that addresses whole-number concepts and skills as a supplement to maths teaching. In this study, the researchers examined data from a randomised evaluation (Clark et al., 2017) studying kindergartners from 69 classrooms during two separate school years. Subjects were tested using five measures of whole-number sense each autumn, and those whose scores fell below a determined threshold were assigned to either a 2:1 ROOTS group (n=120), a 5:1 ROOTS group (n=295), or to the no-intervention control group (n=177). ROOTS pupils received 20-minute small-group sessions five times a week during ten weeks spanning late fall to early spring. Post-tests in the spring of kindergarten (Year 1) and then six months into first grade (Year 2) found that the pupils with lower initial maths skills demonstrated greater gains than others on two of the six outcome measures of the TEMA-3, although there was no correlation with intervention group size.

Source: Exploring the relationship between initial mathematics skill and a kindergarten mathematics intervention (January 2019) Exceptional Children, 85(2)

An evaluation of PACE Center for Girls

Megan Millenky and colleagues from MDRC have released a new reporton an evaluation of PACE Center for Girls. PACE, a Florida-based organisation, provides academic and social services to at-risk middle and high school girls. According to the report, PACE operates daily, year-round; on a typical day, girls attend academic classes and receive additional support such as individual counselling, academic advice, and referrals to other services.

The research team used a random assignment design to evaluate the impact of PACE. From August 2013 to November 2015, a sample of 1,125 girls were enrolled in the study (673 in the programme group, and 452 in the control group). Data sources included administrative records, a survey, and interviews.

Key findings from the study were as follows:

  • The programme group received more academic and social services — and received them more often from a professional source — than the control group.
  • Over a one-year period, PACE increased school enrolment and attendance for the girls it served, compared with the control group. Girls in the programme group were also more likely to be “on track” academically than those in the control group.
  • Girls in both the programme and control groups appeared goal-orientated and hopeful about their futures and reported relatively low levels of risky behaviour one year after study enrolment.
  • The cost of PACE’s holistic package of services is, on average, $10,400 per pupil more than the cost of the services received by control group members through academic and social services provided in the community. The additional cost is largely driven by PACE’s extensive social services; the cost of academic services is similar to those of Florida public schools.

The authors note that further follow-up research would be necessary to see whether PACE affects longer-term academic and delinquency outcomes and to complete a full benefit-cost analysis.

Source: Focusing on girls’ futures: Results from the evaluation of PACE Center for Girls (January 2019), MDRC

Evidence of disciplinary bias against sexual minority females

LGBTQ (lesbian, gay, bisexual, transgender, questioning) pupils are coming out at earlier ages and becoming more visible in schools, creating a need for research on their educational experiences and outcomes. Exclusionary bias studies, which look at the proportions of pupils suspended or expelled, have historically focused on the bias against pupils of colour, yet sexual minority pupils face similar risks.

Joel Mittleman of Princeton University introduced a new data source for research on sexual minority pupils: The Fragile Families and Childhood Wellbeing Study. It is comprised of data on 4,898 children born in 20 US cities between 1998 and 2000, and at baseline was representative of all births at this time in cities with more than 200,000 people. The recent Year 15 follow up includes information on sexual orientation. Dr Mittleman used this data to relate sexual orientation to educational experiences and outcomes. He found that compared to teenagers solely attracted to the opposite gender:

  • Same-sex attracted teenagers are 29% more likely to experience exclusionary discipline.
  • This risk is stratified by gender, increasing to 95% higher odds of discipline among females. Yet based on parent report, Mittleman attributes only 38% of these disciplinary actions to behavioural problems.

This unexplained gap in discipline raises a red flag indicating that homophobia in schools is not gender-neutral, and warrants further research into the treatment of sexual minority status females versus males.

Source: Sexual orientation and school discipline: New evidence from a population-based sample (January 2018), Educational Researcher, Volume: 47 issue: 3

Do EAL pupils need an extra year to learn English?

Since 2002, all third grade (Year 4) pupils in Florida are required to obtain specific state-wide reading test scores in order to progress to the fourth grade (Year 5). A new NBER working paper considers whether this third grade retention policy, which includes additional teaching and support in reading, might be particularly beneficial for pupils with English as an additional language (EAL).

David N Figlio and Umut Özek used longitudinal data for all pupils between grades three and ten (Years 4 to 11) from 12 US school districts in Florida in order to examine the short-, medium- and long-term effects of repeating the third grade on EAL pupils’ English skills, as measured by their reading test scores, the length of time needed for them to reach required levels of English proficiency, and their course choice in middle and high school.

The results find that repeating the third grade (Year 4) can help to improve the English skills of EAL pupils, and that the benefits are even greater for EAL pupils born outside of the US, pupils whose first language is Spanish, and pupils in lower-poverty elementary schools.

Specifically, they suggest that EAL pupils who repeat the third grade:

  • do better on reading test scores in elementary and middle school
  • reach the required levels of English proficiency in half the time
  • are less likely to take a remedial English course in middle school
  • are more likely to take an advanced course in maths and science in middle school
  • are more likely to take college credit-bearing courses in high school.

Source: An extra year to learn English? Early grade retention and the human capital development of English learners (January 2019), NBER Working Paper No. 25472, National Bureau of Economics Research

Is free and reduced-price lunch a valid measure of educational disadvantage?

Almost 60% of American pupils receive lunch through the National School Lunch Program (NSLP), which provides free- and reduced-price lunches (FRPL) to pupils in households who demonstrate economic disadvantage. Income information is based on parent report of household income in the month preceding application to the programme. Because NSLP enrolment has been correlated with lower pupil achievement and is an indicator of how disadvantaged a school’s population is, this classification plays an important role in how funds are allocated to schools and how schools are classified in educational research.

Researchers at the University of North Carolina at Chapel Hill, the US Census Bureau, the University of California at Irvine, and NORC at the University of Chicago recently examined how accurately FRPL enrolment measures actual income and educational disadvantage by comparing IRS income records with pupils’ lunch enrolment and achievement records. Specifically, researchers examined the records of all eighth grade (Year 9) pupils in a California public school district from 2008-2014 (n=14,000) and in Oregon public schools from 2004-2014 (n=363,000), examining the relationship between FRPL enrolment, IRS income records, and eighth grade English language achievement scores on the California Achievement Test and the Oregon Assessment of Knowledge and Skills.

Results showed that school lunch programme enrolment explains the relationship between economic disadvantage and pupil achievement better than IRS-reported annual income. Compared to their non-NSLP peers, California free-lunch pupils scored 0.40 standard deviations (SD) lower on the eighth grade English language test, and the reduced-lunch pupils scored 0.20 lower than those not enrolled. In Oregon, the FRPL pupils scored 0.36 SD lower than those not enrolled. In comparison, when using IRS-reported household income to explain English language achievement, economically disadvantaged pupils scored approximately 0.15 SD lower than pupils appearing to be ineligible for FRPL in one California district and 0.26 SD lower across Oregon public schools. In other words, FRPL enrolment accounted for 16% of the variance in English language scores, whereas IRS data only accounted for 13% of the variance. This indicates that FRLP appears to capture aspects of disadvantage that IRS data do not.

Source: Is free and reduced-price lunch a valid measure of educational disadvantage? (December 2018), Educational Researcher, Volume: 47 issue: 9