Publication Bias in Empirical Sociological Research
Authors:
- 1 August 2008
- journal article
- Published by SAGE Publications in Sociological Methods & Research
- Vol. 37 (1) , 3-30
Abstract
Despite great attention to the quality of research methods in individual studies, if publication decisions of journals are a function of the statistical significance of research findings, the published literature as a whole may not produce accurate measures of true effects. This article examines the two most prominent sociology journals (the American Sociological Review and the American Journal of Sociology) and another important though less influential journal (The Sociological Quarterly) for evidence of publication bias. The effect of the .05 significance level on the pattern of published findings is examined using a ``caliper'' test, and the hypothesis of no publication bias can be rejected at approximately the 1 in 10 million level. Findings suggest that some of the results reported in leading sociology journals may be misleading and inaccurate due to publication bias. Some reasons for publication bias and proposed reforms to reduce its impact on research are also discussed.Keywords
This publication has 39 references indexed in Scilit:
- Alphas and Asterisks: The Development of Statistical Significance Testing Standards in SociologySocial Forces, 2005
- Panel Models in Sociological Research: Theory into PracticeAnnual Review of Sociology, 2004
- Estimating Causal Effects With Matching Methods in the Presence and Absence of Bias CancellationSociological Methods & Research, 2004
- An Application of Gibbs Sampling to Estimation in Meta-Analysis: Accounting for Publication BiasJournal of Educational and Behavioral Statistics, 1997
- An Assessment of Publication Bias Using a Sample of Published Clinical TrialsJournal of the American Statistical Association, 1989
- Publication Bias: A Problem in Interpreting Medical DataJournal of the Royal Statistical Society Series A: Statistics in Society, 1988
- Estimation of Effect Size under Nonrandom Sampling: The Effects of Censoring Studies Yielding Statistically Insignificant Mean DifferencesJournal of Educational Statistics, 1984
- The file drawer problem and tolerance for null results.Psychological Bulletin, 1979
- Consequences of prejudice against the null hypothesis.Psychological Bulletin, 1975
- The Replication Problem in Sociology: A Report and a Suggestion*Sociological Inquiry, 1973