A thirty-year look at studies on computer-assisted maths

During the past 30 years, thousands of articles have been written about technology’s effects on pupil achievement. In order to quantify technology’s effects on maths achievement, Jamaal Young at the University of Texas conducted a meta-analysis of all of the meta-analyses on the topic during the last three decades. His second-order meta-analysis was comprised of 19 meta-analyses representing 663 primary studies, more than 141,000 pupils and 1,263 effect sizes. Each meta-analysis that was included had to address the use of technology as a supplement to instruction, use pupil maths achievement as an outcome measure, report an effect size or enough data to calculate one, have been published after 1985 and be accessible to the public.

The author found that all technology enhancements positively affected pupil achievement, regardless of the technology’s purpose. However, technology that helped pupils perform computational functions had the greatest effects on pupil achievement, while combinations of enhancements demonstrated the least effects on pupil achievement. The author found that study quality and the type of technology used in the classroom were the main influencers on effect sizes. The highest-quality studies had the lowest effect sizes, which he attributes to their more rigorous analysis procedures. The high-quality reviews gave an overall effect size for the use of technology of +0.16 (compared with +0.38 for low- and +0.46 for medium-quality reviews).

Source: Technology-enhanced mathematics instruction: A second-order meta-analysis of 30 years of research (November 2017), Educational Research Review, Volume 22

New research on technology integration in the classroom

Dr Jennifer Morrison and colleagues from the Center for Research and Reform in Education (CRRE) are evaluating a technology-integration initiative for Baltimore County Public Schools in Maryland, US, called Students and Teachers Accessing Tomorrow (S.T.A.T.). The initiative began in the 2014-15 school year with 10 pilot elementary (primary) schools and has since expanded to all elementary schools, middle schools and selected high schools. CRRE is conducting a mixed-methods evaluation of the initiative, including classroom observations, interviews, focus groups, surveys and an examination of pupil achievement data. The evaluation is taking place over five years.

CRRE recently completed the third year of the evaluation, and results revealed continuing changes in teacher practice, perceptions of increased pupil engagement and positive trends in achievement for pupils in grades 3 to 5 (Years 4 to 6).

In a previous edition of Best Evidence in Brief, we reported on a review of the research literature on the infusion of technology into the school curriculum, also completed by CRRE. Multiple studies in that review reported higher engagement of pupils with their coursework when involved in one-to-one laptop programmes, which produced two key benefits: a development of a deeper level of understanding and an increase in pupil achievement. However, our researchers pointed out that, “Just as buying a professional-looking mixer will not make you a better cook, technology alone will not make pupil learning better. If the teacher, though, introduces new methods of teaching requiring different uses of a computer rather than to simply present information, then we are likely to see an improvement in learning”.

Source: Students and teachers accessing tomorrow – year three evaluation report (September 2017), Center for Research and Reform in Education

What does the evidence say about technology use?

New educational technology programmes are being released faster than researchers can evaluate them. The National Bureau of Economic Research in the US has written a working paper, Education Technology: An Evidence-Based Review, which discusses the evidence to date on the use of technology in the classroom, with the goal of finding decision-relevant patterns.

Maya Escueta and colleagues compiled publicly available quantitative research that used either randomised controlled trials or regression discontinuity designs (where pupils qualify for inclusion in a programme based on a cut-off score at pre-test). All studies had to examine the effects of an ed-tech intervention on any education-related outcome. Therefore, the paper included not only the areas of technology access, computer-assisted learning and online courses, but also the less-often-studied technology-based behavioural interventions.

Authors found that:

  • Access to technology may or may not improve academic achievement at the K-12 level (Years 1–13), but does have a positive impact on the academic achievement of higher education students (ES=+0.14).
  • Computer-assisted learning, when equipped with personalisation features, was an effective strategy, especially in maths.
  • Behavioural intervention software, such as text-message reminders or e-messages instructing parents how to practise reading with their children, showed positive effects at all levels of education, and was also a cost-effective approach. Four main uses for behavioural intervention software emerged: encouraging parental involvement in early learning activities, communication between the school and parents, successfully transitioning into and through higher education, and creating mindset interventions. Research is recommended to determine the areas where behavioural intervention software is most impactful.
  • Online learning courses had the least amount of research to examine and showed the least promise of the four areas. However, when online courses were accompanied by in-person teaching, the effect sizes increased to scores comparable to fully in-person courses.

 Source: Education technology: an evidence-based review (August 2017), National Bureau of Economic Research (NBER) Working Paper No. 23744

Comprehension Circuit Training delivered via tablet shows reading benefits

Comprehension Circuit Training (CCT) is a programme for teenagers designed to improve reading comprehension through a set of circuit-like exercises in pre-reading, reading, and after-reading to improve foundational reading skills and text-processing abilities. A recent randomised study investigated the effects of CCT delivered electronically via tablet on the reading comprehension of struggling teenage readers.

Three schools in Texas in the US, involving 3 teachers and 228 struggling sixth- to eighth-graders (Years 7–9), participated in this study. Using a within-teacher design, middle school teachers’ reading intervention classes were randomly assigned to electronic CCT (n=9 classes, 112 pupils) or business as usual (n=7 classes, 116 pupils). All pupils had failed to score at the “proficient” level on the prior year’s state reading assessment, and no significant pre-test differences were found between the two groups. CCT pupils received 39 e-CCT lessons in word reading, vocabulary, and reading comprehension that were organised into ten levels delivered in a standard sequence. Each 50-minute lesson contained four video-instruction components – an Opening Comprehension Circuit, WarmUp Station, Reading Core Station, and Knowledge Flex Station – delivered three days a week via Apple iPad. After video instruction, pupils partnered to practice lesson content, with teacher-led assessment occurring in the Knowledge Flex Station.

Results showed statistically significant effects in favour of the experimental group on post-test measures of reading comprehension, vocabulary, and silent reading efficiency. Pupils who entered with lower-level reading comprehension showed the greatest gains.

Source: Impact of a technology-mediated reading intervention on adolescents’ reading comprehension. Journal of Research on Educational Effectiveness, Vol. 10, 2.

Predicting success in STEM

The Regional Educational Laboratory Southwest has conducted a literature review to determine what predictors from primary school of postsecondary STEM success have been identified in peer-reviewed studies, with a focus on predictors for Hispanic students.

The review defined postsecondary STEM success as enrollment in, persistence in, and completion of postsecondary STEM majors or degrees. Twenty-three relevant studies were identified, but only four examined factors predictive of success specifically for Hispanic students.
Key findings from the review included:

  • The number of high school maths and science courses taken and the level of those courses predict postsecondary STEM success for all student subgroups, but racial/ethnic minority students were less likely than White students to take the highest level maths and science courses.
  • Interest or confidence in STEM showed statistically significant predictive relationships with students’ postsecondary STEM success, and the relationships were evident as early as middle school. Racial/ethnic minority and White students had similar interest and confidence in STEM.
  • Statistically significant high school predictors of postsecondary STEM success included schools’ academic rigour, percentage of students enrolled in college preparatory programmes, students’ satisfaction with their teachers, and levels of parent participation.

Source: A Review of the Literature to Identify Leading Indicators Related to Hispanic STEM Postsecondary Educational Outcomes (2016), Regional Educational Laboratory Southwest.

Trends in our schools

The Parliamentary Office of Science and Technology has produced a new briefing on trends in compulsory education across the whole of the UK, as well as the factors shaping these trends.

Using a broad raft of data, combined with previous research, they identified five trends.

  1. Changes in the attainment gap between different pupil groups. Although comparing data from different nations within the UK is problematic, the authors conclude that the largest attainment gap is between pupils from different socio-economic backgrounds – bigger than both the largest ethnic minority gap and gender gap.
  2. Rising pupil numbers. Pupil numbers have increased across the UK as a whole in recent years, although they are falling in secondary schools in Scotland and Northern Ireland.
  3. Decreasing numbers of people entering initial teacher training. The number of people entering training has been declining at secondary level in England, Scotland, and Wales since 2005/06 (data could not be identified for Northern Ireland). However, with the exception of Wales, the ratio of pupils to teachers has remained constant across the UK at all levels.
  4. The growing use of technology. Although technology is increasingly being adopted in classroom, research has shown that this does not in itself improve learning. The authors suggest one of the most promising ways in which technology could benefit teaching and learning is through changes to assessment.
  5. Changing levels of school autonomy and diversity. Schools in England have more autonomy from local authorities than in other UK nations, and there is a greater diversity of types of school.

The report also identified three key factors shaping these trends: population growth, inequality, and the changing labour market (particularly in terms of concerns about whether schools are providing children with the right knowledge and skills for the labour market of the future).

Source: Trends in Compulsory Education (2015), Parliamentary Office of Science and Technology.