Communicating comparative findings from meta-analysis in educational research: some examples and suggestions
Authors:
- 23 March 2016
- journal article
- research article
- Published by Taylor & Francis in International Journal of Research & Method in Education
- Vol. 39 (3) , 237-254
Abstract
This article reviews some of the strengths and limitations of the comparative use of meta-analysis findings, using examples from the Sutton Trust-Education Endowment Foundation Teaching and Learning ‘Toolkit’ which summarizes a range of educational approaches to improve pupil attainment in schools. This comparative use of quantitative findings has similar characteristics to umbrella reviews which provide a succinct but applicable summary of the current state of evidence to inform practice or policy. Meta-analysis helps to identify which approaches have, on average, made the most difference to tested learning outcomes, in terms of effect size. We suggest that any comparative inferences made between meta-analyses should be treated cautiously, but taken seriously. Additionally, we present alternative ways of interpreting effect sizes, security ratings and cost-estimates to make research findings accessible, whilst retaining appropriate accuracy which is discussed using the ‘Toolkit’ as an example. We conclude by arguing that we should consider the available information not as ‘what works’, but ‘what has worked’ to understand its value and limits in terms of supporting the development of research-based practice.Keywords
Funding Information
- Education Endowment Foundation (Development of the Teaching and Learning Toolkit)
- Sutton Trust (Strategies For Improving Learning)
This publication has 47 references indexed in Scilit:
- The importance of studying the implementation of interventions in school settingsOxford Review of Education, 2012
- Consort 2010 statement: extension to cluster randomised trialsBMJ, 2012
- Clarifying differences between review designs and methodsSystematic Reviews, 2012
- Is there a “school effect” on pupil outcomes? A review of multilevel studiesJournal of Epidemiology and Community Health, 2006
- Statistical analysis and optimal design for cluster randomized trials.Psychological Methods, 1997
- Individualized Systems of Instruction in Secondary SchoolsReview of Educational Research, 1983
- A meta‐analysis of instructional systems applied in science teachingJournal of Research in Science Teaching, 1983
- Evaluative Applications of Meta- and Mega-analysisEvaluation News, 1982
- SuperanalysisEvaluation News, 1982
- Primary, Secondary, and Meta-Analysis of ResearchEducational Researcher, 1976