Model Uncertainty in Sociological Research: An Application to Religion and Economic Growth
- 1 June 2009
- journal article
- research article
- Published by SAGE Publications in American Sociological Review
- Vol. 74 (3) , 380-397
Abstract
Model uncertainty is pervasive in quantitative research. Classical statistical theory assumes that only one (true) model is applied to a sample of data. In practice, however, researchers do not know which exact model specification is best. Modern computing power allows researchers to estimate a huge number of plausible models, yet only a few of these estimates are published. The result is a severe asymmetry of information between analyst and reader. The applied modeling process produces a much wider range of estimates than is suggested by the usual standard errors or confidence intervals. I demonstrate this using the work of Barro and McCleary on religion and economic growth. Small, sensible changes in their model specification produce large changes in the results: the results are inconsistent across time, and the instrumental variables strategy suffers from a weak instrument set. Also, the observed relationship between religiosity and economic growth does not hold in the West; it is largely a feature of Asian and African countries and of countries whose data is poor quality. In short, empirical findings should be evaluated not just by their significance but also by their robustness to model specification. I conclude with suggestions for incorporating model uncertainty into practice and improving the transparency of social science research.Keywords
This publication has 31 references indexed in Scilit:
- Replication Standards for Quantitative Social ScienceSociological Methods & Research, 2007
- Salvation, Damnation, and Economic IncentivesJournal of Contemporary Religion, 2007
- The poor availability of psychological research data for reanalysis.American Psychologist, 2006
- Testing for Weak Instruments in Linear IV Regression2005
- Why the Middle East is Economically Underdeveloped: Historical Mechanisms of Institutional StagnationJournal of Economic Perspectives, 2004
- A Survey of Weak Instruments and Weak Identification in Generalized Method of MomentsJournal of Business & Economic Statistics, 2002
- Instrumental Variables Regression with Weak InstrumentsEconometrica, 1997
- Vague Theory and Model Uncertainty in MacrosociologySociological Methodology, 1996
- Problems with Instrumental Variables Estimation when the Correlation between the Instruments and the Endogenous Explanatory Variable is WeakJournal of the American Statistical Association, 1995
- Poverty, Income Inequality, and Violent Crime: A Meta-Analysis of Recent Aggregate Data StudiesCriminal Justice Review, 1993