Statistical Perspectives on Reproducibility: Definitions and Challenges
Open Access
- 27 May 2025
- journal article
- research article
- Published by Springer Nature in Journal of Statistical Theory and Practice
- Vol. 19 (3) , 1-40
Abstract
Reproducibility is a widely discussed topic, yet many experimental results cannot be confirmed due to factors such as publication bias, poor documentation, and inappropriate statistical methods. A lack of standard definitions for reproducibility and related terms further complicates the matter. This paper reviews the literature on reproducibility, clarifies key terminology by defining five types of reproducibility, and addresses variations in published definitions by considering changes in datasets, labs, and experimental conditions. We explore the causes of low reproducibility in scientific studies and discuss statistical perspectives on quantifying and improving reproducibility. In particular, we propose framing statistical reproducibility as a predictive problem, providing a framework to evaluate and address reproducibility challenges.Keywords
This publication has 122 references indexed in Scilit:
- The Importance of Predefined Rules and Prespecified Statistical AnalysesJAMA, 2019
- Guidelines for the Content of Statistical Analysis Plans in Clinical TrialsJAMA, 2017
- Reproducibility in Chemical ResearchAngewandte Chemie International Edition, 2016
- Reproducibility from the Perspective of Meta-Analysis2016
- Reproducibility: Principles, Problems, Practices, and Prospects2016
- Multiplicity and replicability: two sides of the same coinPharmaceutical Statistics, 2014
- Introduction to Imprecise Probabilities2014
- Reproducibility of clinical data II: categorical outcomesPharmaceutical Statistics, 2004
- Reproducibility of clinical data I: continuous outcomesPharmaceutical Statistics, 2004
- A comment on replication, p‐values and evidence S.N.Goodman, Statistics in Medicine 1992; 11:875‐879Statistics in Medicine, 2002