Statistical Significance Versus Clinical Importance of Observed Effect Sizes: What Do P Values and Confidence Intervals Really Represent?
Open Access
- 1 March 2018
- journal article
- research article
- Published by Wolters Kluwer Health in Anesthesia & Analgesia
- Vol. 126 (3) , 1068-1072
Abstract
Effect size measures are used to quantify treatment effects or associations between variables. Such measures, of which >70 have been described in the literature, include unstandardized and standardized differences in means, risk differences, risk ratios, odds ratios, or correlations. While null hypothesis significance testing is the predominant approach to statistical inference on effect sizes, results of such tests are often misinterpreted, provide no information on the magnitude of the estimate, and tell us nothing about the clinically importance of an effect. Hence, researchers should not merely focus on statistical significance but should also report the observed effect size. However, all samples are to some degree affected by randomness, such that there is a certain uncertainty on how well the observed effect size represents the actual magnitude and direction of the effect in the population. Therefore, point estimates of effect sizes should be accompanied by the entire range of plausible values to quantify this uncertainty. This facilitates assessment of how large or small the observed effect could actually be in the population of interest, and hence how clinically important it could be. This tutorial reviews different effect size measures and describes how confidence intervals can be used to address not only the statistical significance but also the clinical significance of the observed effect or association. Moreover, we discuss what P values actually represent, and how they provide supplemental information about the significant versus nonsignificant dichotomy. This tutorial intentionally focuses on an intuitive explanation of concepts and interpretation of results, rather than on the underlying mathematical theory or concepts.Keywords
This publication has 26 references indexed in Scilit:
- The fallacy of placing confidence in confidence intervalsPsychonomic Bulletin & Review, 2015
- Some Common Misperceptions About P ValuesStroke, 2014
- Local Insufflation of Warm Humidified CO2 Increases Open Wound and Core Temperature During Open Colon SurgeryAnesthesia & Analgesia, 2012
- Guide for calculating and interpreting effect sizes and confidence intervals in intellectual and developmental disability research studiesJournal of Intellectual & Developmental Disability, 2012
- On effect size.Psychological Methods, 2012
- The proof and measurement of association between two thingsInternational Journal of Epidemiology, 2010
- A Dirty Dozen: Twelve P-Value MisconceptionsSeminars in Hematology, 2008
- Effect size, confidence interval and statistical significance: a practical guide for biologistsBiological Reviews, 2007
- Reporting the size of effects in research studies to facilitate assessment of practical or clinical significancePsychoneuroendocrinology, 1992
- Outline of a Theory of Statistical Estimation Based on the Classical Theory of ProbabilityPhilosophical Transactions of the Royal Society of London. Series A, Mathematical and Physical Sciences, 1937