This working paper from the RAND Corporation examines the effectiveness of Cognitive Tutor Algebra I (CTAI), a technology-based algebra course designed for pupils at a variety of ability and year levels. The curriculum includes traditional textbook and workbook materials along with automated tutoring software that provides self-paced individualised tuition and attempts to bring pupils to mastery of a topic before they progress further.
Schools participating in the study were matched into similar pairs and randomly assigned to either continue with their current algebra curriculum for two years or to adopt CTAI. The sample included 73 high schools and 74 middle schools in seven US states.
Analysis of post-test outcomes on an algebra proficiency exam found no effects in the first year of implementation, but strong evidence in support of a positive effect in the second year. The estimated effect is statistically significant for high schools but not for middle schools. The authors report that in both cases, the magnitude is sufficient to improve the average pupil’s performance by approximately eight percentile points.
Source: Effectiveness of Cognitive Tutor Algebra I at Scale (2013), RAND Education.
A new report from the RAND Corporation in the US describes recent RAND work related to K–12 education (primary to sixth form), including teacher pay for performance, measuring teacher effectiveness, school leadership, school systems and reform, and out-of-school time. Headlines include:
- No evidence that incentive pay for teacher teams improves pupil outcomes
- Incorporating pupil performance measures into teacher evaluation systems (Recommendations include: (1) promote consistency in the pupil performance measures that teachers are allowed to choose, and (2) use multiple years of pupil achievement data in value-added estimation, and, where possible, use average teachers’ value-added estimates across multiple years.)
- First-year principals in urban school districts: how actions and working conditions relate to outcomes (A key finding of this study was that teacher capacity and cohesiveness were the school and district conditions most strongly related to pupil outcomes.)
When viewing the report online, each headline links to the corresponding RAND report on the topic.
Source: Focus on K-12 education (2012), RAND
This RAND study assesses “Operation Purple”, a free one-week summer camp programme in the US for children with a parent who is on military deployment. The study used a quasi-experimental approach to determine whether there were differences between attendees and non-attendees in four Operation Purple theme areas: comfort and skill in communicating about feelings, understanding and appreciation of military life, sense of service/stewardship, and outdoor education. Data included children and parent survey data (from both camp attendees and a control group of non-attendees), camp after-action reports, and visitor observation logs. Key findings of the study were as follows:
- The most significant difference between children who attended an Operation Purple camp and those who did not was in parent reports that children had a greater ability to communicate feelings of anxiety and stress surrounding parental deployment and a greater connection to the military and their peers. Parents also reported that camp participants had a greater interest in camping in the follow-up surveys.
- The study found no significant differences between children who attended camp and those who did not in the area of sense of service/stewardship
Source: Assessing Operation Purple: A program evaluation of a summer camp for military youth (2012), RAND
In the last issue of Best Evidence in Brief, we included a PISA in Focus review on performance-based pay for teachers. This US study from the RAND Corporation also looks at performance pay, but specifically at the effects of rewarding teams of teachers. The study, which used a randomised design, included 159 teams of teachers teaching pupils in grades 6 to 8 (KS3) in nine schools. Teachers on selected teams had the opportunity to earn a bonus based on their pupil’s growth in achievement in mathematics, English language arts, science, and social studies.
The study showed that the intervention had no effect on pupil achievement, teacher practices, or teacher attitudes. Pupils taught by teacher teams who were offered incentives scored slightly better on some standardised tests, but the differences were small and not statistically significant.
Source: No evidence that incentive pay for teacher teams improves student outcomes (2012), RAND Corporation
This research brief from the RAND Corporation presents a summary of research on first-year principals’ (head teachers in the UK) experiences, actions, working conditions, and outcomes. It was created to inform efforts to promote school improvement and principal retention. To complete their study, RAND researchers looked at the experiences of first-year principals in six districts across the US. Findings included:
- More than a fifth of first-year principals left their school within two years;
- Schools that lost a principal after one year underperformed the following year;
- The quality of principals’ actions was more relevant to outcomes than the amount of time devoted to the actions;
- Greater teacher capacity and cohesiveness were related to better pupil outcomes; and
- Principals’ personnel management skills are important.
Source: Challenges and opportunities facing principals in the first year at a school (2012), RAND Corporation
A study from the RAND Corporation examines what makes for good reading coaches and coaching. The study included 113 schools from 8 districts in Florida. All used reading coaches to work with school staff to improve their reading teaching and leadership skills. The data showed no relationship between teacher and principal perceptions of coach quality and students’ reading achievement.
The researchers suggest that being an effective literacy coach may require more than content-area expertise and experience teaching children. They identify “understanding how to support adult learners” as a key area of expertise that was sometimes lacking with the coaches in the study.
Source: Reading Coach Quality: Findings from Florida Middle Schools (2012), Literacy Research and Instruction, 51(1).