A Systematic Review on the Evolution of Power Analysis Practices in Psychological Research
Open Access
- 9 January 2025
- journal article
- review article
- Published by Ubiquity Press, Ltd. in Psychologica Belgica
- Vol. 65 (1) , 17-37
Abstract
Performing hypothesis tests with adequate statistical power is indispensable for psychological research. In response to several large-scale replication projects following the replication crisis, concerns about the root causes of this crisis – such as questionable research practices (QRPs) – have grown. While initial efforts primarily addressed the inflation of the type I error rate of research due to QRPs, recent attention has shifted to the adverse consequences of low statistical power. In this paper we first argue how underpowered studies, in combination with publication bias, contribute to a literature rife with false positive results and overestimated effect sizes. We then examine whether the prevalence of power analyses in psychological research has effectively increased over time in response to the increased awareness regarding these phenomena. To address this, we conducted a systematic review of 903 published empirical articles across four APA-disciplines, comparing 453 papers published in 2015–2016, with 450 papers from 2020–2021. Although the prevalence of power analysis across different domains in psychology has increased over time (from 9.5% to 30%), it remains insufficient overall. We conclude by discussing the implications of these findings and elaborating on some alternative methods to a priori power analysis that can help ensure sufficient statistical power.Keywords
This publication has 88 references indexed in Scilit:
- Why psychologists must change the way they analyze their data: The case of psi: Comment on Bem (2011).Journal of Personality and Social Psychology, 2011
- Standardized or simple effect size: What should be reported?British Journal of Psychology, 2009
- Why Most Discovered True Associations Are InflatedEpidemiology, 2008
- G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciencesBehavior Research Methods, 2007
- Information-based monitoring of clinical trialsStatistics in Medicine, 2006
- The Persistence of Underpowered Studies in Psychological Research: Causes, Consequences, and Remedies.Psychological Methods, 2004
- Internal pilot studies for estimating sample sizeStatistics in Medicine, 1994
- A power primer.Psychological Bulletin, 1992
- The role of internal pilot studies in increasing the efficiency of clinical trialsStatistics in Medicine, 1990
- The statistical power of abnormal-social psychological research: A review.The Journal of Abnormal and Social Psychology, 1962