Sensitivity Analysis for Survey Weights
Open Access
- 28 April 2023
- journal article
- research article
- Published by Cambridge University Press (CUP) in Political Analysis
- Vol. 32 (1) , 1-16
Abstract
Survey weighting allows researchers to account for bias in survey samples, due to unit nonresponse or convenience sampling, using measured demographic covariates. Unfortunately, in practice, it is impossible to know whether the estimated survey weights are sufficient to alleviate concerns about bias due to unobserved confounders or incorrect functional forms used in weighting. In the following paper, we propose two sensitivity analyses for the exclusion of important covariates: (1) a sensitivity analysis for partially observed confounders (i.e., variables measured across the survey sample, but not the target population) and (2) a sensitivity analysis for fully unobserved confounders (i.e., variables not measured in either the survey or the target population). We provide graphical and numerical summaries of the potential bias that arises from such confounders, and introduce a benchmarking approach that allows researchers to quantitatively reason about the sensitivity of their results. We demonstrate our proposed sensitivity analyses using state-level 2020 U.S. Presidential Election polls.Keywords
All Related Versions
Funding Information
- National Science Foundation (2146752)
This publication has 24 references indexed in Scilit:
- Statistical paradises and paradoxes in big data (I): Law of large populations, big data paradox, and the 2016 US presidential electionThe Annals of Applied Statistics, 2018
- An Evaluation of the 2016 Election Polls in the United StatesPublic Opinion Quarterly, 2018
- Construction of Weights in Surveys: A ReviewStatistical Science, 2017
- Model-Assisted Survey Estimation with Modern Prediction TechniquesStatistical Science, 2017
- Sensitivity analysis for an unobserved moderator in RCT-to-target-population generalization of treatment effectsThe Annals of Applied Statistics, 2017
- Assessing Methods for Generalizing Experimental Impact Estimates to Target PopulationsJournal of Research on Educational Effectiveness, 2016
- Sensitivity analysis for causal inference using inverse probability weightingBiometrical Journal, 2011
- Correcting for Survey Nonresponse Using Variable Response PropensityJournal of the American Statistical Association, 2010
- A Distributional Approach for Causal Inference Using Propensity ScoresJournal of the American Statistical Association, 2006
- Statistical Analysis with Missing Data2002