The effect of a World Cup on pupils’ effort and achievement

A study published in the Journal of Public Economics examines how leisure time can impact pupils’ effort and educational achievement by looking at the overlap of major football tournaments (the FIFA World Cup and the UEFA European Championship) with GCSE exams in England.

Using seven years of subject data on pupils in England, taken from the National Pupil Database, Robert Metcalfe and colleagues estimated the overall effect of a tournament by comparing within-pupil variation in performance during the exam period between tournament and non-tournament years.

Overall, they found a negative average effect of the tournament on exam performance, as measured by whether pupils achieved a grade C or higher in at least 5 subjects at GCSE. In tournament years, the odds of achieving the benchmark of a grade C or higher in at least 5 subjects fell by 12%. For pupils who are likely to be very interested in football (defined as likely to be white, male, disadvantaged pupils), the impact is greater, with the odds of achieving the benchmark reduced by 28%. This result is important as this group is already the lowest performing, with only 21.3% achieving a grade C or higher in at least 5 subjects at GCSE in non-tournament years.

An earlier study reported in a previous issue of Best Evidence in Brief also found that some pupils perform less well in their GCSEs in years when there is a major international football tournament taking place.

Source: Students’ effort and educational achievement: Using the timing of the World Cup to vary the value of leisure (January 2019), Journal of Public Economics, Volume 172

Prerequisites for Assessment for Learning

A new systematic review in the Educational Research Review has analysed the evidence on prerequisites for implementing Assessment for Learning (AfL) in classroom practice. The aim was not to provide a “recipe for success,” but to generate a better understanding of what needs to be considered.

A total of 25 studies met the inclusion criteria. Of these, nine were conducted in the context of primary education, ten in secondary, and six covered both. The results included data from eleven different countries, although nine of the studies were conducted in the US.

The authors found that the knowledge, skills, and attitudes of individual teachers influenced the establishment of an AfL-based learning environment. Pedagogical knowledge and content knowledge had an impact on a teacher’s ability to provide pupils with accurate and complete feedback. They also needed to have the ability to foster the participation of pupils in discussions about their answers, and construct questions that drew out evidence about their learning. The authors found that the quality of AfL was influenced by teachers’ commitment to its underlying ideals, the extent to which they felt responsible for pupils’ attainment of goals, and their willingness to change their assessment practices to implement AfL.

To a lesser extent, pupils’ knowledge, skills, and attitudes also had an impact. A positive attitude and taking an active role in their own learning process was found to foster autonomy and responsibility.

The wider school context also determined how successfully the implementation of AfL was facilitated. School leaders played an important role in establishing a school-wide AfL culture with a vision and expectations for AfL use, as well as providing time to prepare for and carry out professional development.

The review considered the nature of the assessment itself. It concluded that feedback should be substantial, constructive, and focused, and provide the pupils with cues for how to proceed. AfL should be aligned with the curriculum and standards, as well as being integrated into classroom teaching (rather than being an add-on activity).

Source: A Systematic Review of Prerequisites for Implementing Assessment for Learning in Classroom Practice (2016), Educational Research Review, 17.

Putting testing to the test

In response to the lack of evidence surrounding debate in the US over whether pupils are being over tested, the Council of the Great City Schools has conducted a detailed study on testing. They examined test practices at primary and secondary level in 66 of the largest urban school districts during the 2014-15 school year.

The authors found that the average pupil took eight standardised tests a year. Grade 8 (Year 9) pupils were tested the most, spending an average of 4.22 days per year being tested. Yet there was no correlation between the amount of test time and maths and reading achievement.

The study also revealed a number of problems with testing. States reported having to wait 2-4 months for school-level test results meaning the data could not usefully guide teaching, test results were used in ways they weren’t intended to be (eg, to judge an individual staff member’s performance), the tests themselves were not an accurate measure of content knowledge, and pupils were tested in the same subject more than once for different reasons.

A survey of the parents revealed that they support testing that accurately reflects their child’s performance in school, and that they do not support more difficult tests.

Source: Student testing in America’s great city schools: An inventory and preliminary analysis (2015), Council of the Great City Schools.

Buoyant students rise above exams

A study of GCSE exam performance among 705 secondary school students in state-funded schools in England shows that it is OK for students to be tense, but not good for them to be anxious, about high-stakes exams.

The authors used self-reported data from the students to investigate the relationship between academic buoyancy (withstanding routine setbacks, challenges, and pressures), test anxiety (feeling threatened by exams), and high-stakes exam performance. Buoyancy was defined as distinct from resilience (withstanding more severe adversity).

The students used the Revised Test Anxiety Scale to report worry and tension components of test anxiety and the Academic Buoyancy Scale to report academic buoyancy. Academic achievement was measured using average scores from English, maths, and science GCSE exams.

Student data revealed that academic buoyancy was high where the worry component of test anxiety was low (and vice versa). This was reflected in exam results where low worry and high buoyancy were associated with better average GCSE scores. The tension element of test anxiety was unrelated to exam results.

The authors suggested that future studies could take past academic achievement into account and investigate other aspects of test anxiety, such as test-irrelevant thinking and off-task behaviours. They also suggested that insights from the study may inform interventions that aim to reduce test anxiety, improve academic buoyancy, and boost exam performance.

Source: Academically buoyant students are less anxious about and perform better in high-stakes examinations (2015), British Journal of Educational Psychology

Student test scores a reliable measure of principal performance?

The US states of Tennessee, Florida, and Louisiana have linked hiring, promotion, and dismissal of principals (head teachers) to student test scores. A recent paper in Educational Evaluation and Policy Analysis modelled three approaches to assessing principal performance through student test measures and compared these with each other and against non-test methods.

Approach one: school effectiveness

This method takes a simple measure of school effectiveness and attributes this performance to the principal. Drawbacks of this approach include that it does not account for factors such as neighbourhood effects, student backgrounds, and the legacy of previous leaders.

Approach two: relative within-school effectiveness

This method compares school effectiveness under different principals. It has the advantage of accounting for neighbourhood effects, but it does not reflect changes in challenges over time and can only be used where schools have data for more than one principal.

Approach three: school improvement

This method measures school improvement between years. Unlike methods one and two, this does not assume that principal performance is reflected immediately in student scores. A main difficulty of this method is that it requires a principal to serve enough time to enable multiple-year comparisons.

The researchers analysed data on 523 principals in Florida public schools from 2003 to 2011. The three methods provided different results (particularly method three, which rarely correlated with the other two). Compared against non-test school outcome measures, method one showed the best correlation and method three was negatively correlated.

Source: Using student test scores to measure principal performance (2015), Educational Evaluation and Policy Analysis

Test results don’t show how effective teachers are

A new study has looked at the link between instructional alignment (how teaching is aligned with standards and assessments), value-added measures of teacher effectiveness, and composite measures of teacher effectiveness using multiple measures.

The issue is important as, in the US and around the world, there is more emphasis on measuring teacher effectiveness and rewarding effective teachers. The study looked at 324 teachers of fourth and eighth grade (Year 5 and Year 9) mathematics and English language arts in five US states. They completed a Survey of Enacted Curriculum to measure their instructional alignment. This was then compared with value-added measures (taken from state assessments and two supplementary assessments) and teacher effectiveness (using Framework for Teaching scores, widely used by states).

The results showed modest evidence of a relationship between instructional alignment and value-added measures, although this disappeared when controlling for pedagogical quality. The one significant relationship they found was that the association between instructional alignment and value-added measures is more positive when pedagogy is high quality. There was no association between instructional alignment and measures of teacher effectiveness.

These results suggest that the tests used for calculating value-added measures are not able to detect differences in the content or quality of classroom teaching.

Source: Instructional Alignment as a Measure of Teaching Quality (2014), Education Evaluation and Policy Analysis, online first, May 2014.