A Deeper Dive into the Digital Divide: Reducing Coverage Bias in Internet Surveys
Authors:
- 20 May 2022
- journal article
- research article
- Published by SAGE Publications in Social Science Computer Review
- Vol. 41 (5) , 1902-1920
Abstract
Survey research is increasingly turning to online research and as a consequence, sampling only, or disproportionately from, households with internet access. While the percentage of non-internet households has declined, it persists at about one in ten households. This raises the question of coverage error and bias, and whether there is an approach to reduce possible biases in internet-only samples. We processed nearly 5000 variables from over a dozen major public and private datasets to assess the extent of the bias. Finding substantive differences, we then applied a series of data-reducing techniques to arrive at 38 variables that independently skew across internet and non-internet populations. We then developed and fielded a survey of these metrics to assess which dozen or less could be used to construct an efficient propensity model to reduce bias in internet-only samples. Our analyses revealed that many variables noted in prior research are important predictors of non-internet use, but also identified others. Our final propensity model of 10 variables was highly effective, reducing bias significantly. Many variables tested had bias reduced fourfold. Contributions: Prior research has not investigated the digital divide from a wide array of public datasets, nor done enough to consider the bias inherent in internet users-only samples. Our novel random-forest approach and subsequent survey of key candidate variables controlled for correlations among the variables in identifying the most important variables across multiple datasets. Our coding of 542 significant predictors of internet use contributes to the sociological understanding of internet access.Keywords
This publication has 16 references indexed in Scilit:
- Comparing the Accuracy of RDD Telephone Surveys and Internet Surveys Conducted with Probability and Non-Probability SamplesPublic Opinion Quarterly, 2011
- Estimating Propensity Adjustments for Volunteer Web SurveysSociological Methods & Research, 2011
- National Surveys Via Rdd Telephone Interviewing Versus the InternetPublic Opinion Quarterly, 2009
- The Foundations of Quality InitiativeJournal of Advertising Research, 2009
- Predictor correlation impacts machine learning algorithms: implications for genomic studiesBioinformatics, 2009
- Conditional variable importance for random forestsBMC Bioinformatics, 2008
- The Effect of Survey Mode and Sampling on Inferences about Political Attitudes and Behavior: Comparing the 2000 and 2004 ANES to Internet Surveys with Nonprobability SamplesPolitical Analysis, 2007
- Online versus offline research: implications for evaluating digital mediaAslib Proceedings, 2002
- Model-Based Direct AdjustmentJournal of the American Statistical Association, 1987
- The central role of the propensity score in observational studies for causal effectsBiometrika, 1983