Inappropriate Fiddling with Statistical Analyses to Obtain a Desirable P-value: Tests to Detect its Presence in Published Literature
Open Access
- 8 October 2012
- journal article
- research article
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 7 (10) , e46363
Abstract
Much has been written regarding p-values below certain thresholds (most notably 0.05) denoting statistical significance and the tendency of such p-values to be more readily publishable in peer-reviewed journals. Intuition suggests that there may be a tendency to manipulate statistical analyses to push a “near significant p-value” to a level that is considered significant. This article presents a method for detecting the presence of such manipulation (herein called “fiddling”) in a distribution of p-values from independent studies. Simulations are used to illustrate the properties of the method. The results suggest that the method has low type I error and that power approaches acceptable levels as the number of p-values being studied approaches 1000.Keywords
This publication has 6 references indexed in Scilit:
- Testing for the Presence of Positive-Outcome Bias in Peer ReviewArchives of internal medicine (1960), 2010
- Illustrations on Using the Distribution of a P-value in High Dimensional Data Analyses2010
- An unexpected influence of widely used significance thresholds on the distribution of reported P‐valuesJournal of Evolutionary Biology, 2007
- Interval estimation in a finite mixture model: Modeling -values in multiple testing applicationsComputational Statistics & Data Analysis, 2006
- A mixture model approach for the analysis of microarray gene expression dataComputational Statistics & Data Analysis, 2002
- Publication bias: evidence of delayed publication in a cohort study of clinical research projectsBMJ, 1997