Which Data to Meta-Analyze, and How?
Authors:
- 1 January 2019
- journal article
- review article
- Published by Hogrefe Publishing Group in Zeitschrift für Psychologie
- Vol. 227 (1) , 64-82
Abstract
Which data to analyze, and how, are fundamental questions of all empirical research. As there are always numerous flexibilities in data-analytic decisions (a “garden of forking paths”), this poses perennial problems to all empirical research. Specification-curve analysis and multiverse analysis have recently been proposed as solutions to these issues. Building on the structural analogies between primary data analysis and meta-analysis, we transform and adapt these approaches to the meta-analytic level, in tandem with combinatorial meta-analysis. We explain the rationale of this idea, suggest descriptive and inferential statistical procedures, as well as graphical displays, provide code for meta-analytic practitioners to generate and use these, and present a fully worked real example from digit ratio (2D:4D) research, totaling 1,592 meta-analytic specifications. Specification-curve and multiverse meta-analysis holds promise to resolve conflicting meta-analyses, contested evidence, controversial empirical literatures, and polarized research, and to mitigate the associated detrimental effects of these phenomena on research progress.Keywords
This publication has 85 references indexed in Scilit:
- No Evidence that 2D:4D is Related to the Number of CAG Repeats in the Androgen Receptor GeneFrontiers in Endocrinology, 2013
- Analysis of decisions made in meta-analyses of depression screening and the risk of confirmation bias: A case studyBMC Medical Research Methodology, 2012
- Is CAG sequence length in the androgen receptor gene correlated with finger-length ratio?Personality and Individual Differences, 2012
- Digit ratio (2D:4D) and sex-role orientation: Further evidence and meta-analysisPersonality and Individual Differences, 2011
- Special issue preamble: Digit ratio (2D:4D) and individual differences researchPersonality and Individual Differences, 2011
- 2D:4D ratios in the first 2years of life: Stability and relation to testosterone exposure and sensitivityHormones and Behavior, 2011
- Sex-related variation in human behavior and the brainTrends in Cognitive Sciences, 2010
- Minireview: Organizational Hypothesis: Instances of the FingerpostEndocrinology, 2010
- Digit ratio (2D:4D) and sensation seeking: New data and meta-analysisPersonality and Individual Differences, 2010
- Violent video game effects on aggression, empathy, and prosocial behavior in Eastern and Western countries: A meta-analytic review.Psychological Bulletin, 2010