Matthew DeBell
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Matthew DeBell has a total of 46 co-authors. Here are the top co-authors by number of shared publications:
Bo MacInnis
Institute for Research in the Social Sciences, Stanford University, Stanford, USA
3 shared publicationsCharles R. Ebersole
Department of Psychology, University of Virginia, Charlottesville, USA
3 shared publications- John Protzko
Department of Psychological Science, Central Connecticut State University, Connecticut, US
3 shared publications Jon Krosnick
Institute for Research in the Social Sciences, Stanford University, Stanford, USA
3 shared publications- Michael O'DOnnell
McDonough School of Business, Georgetown University, Washington, USA
3 shared publications Scott S. Roeder
Darla Moore School of Business, University of South Carolina, Columbia, USA
3 shared publications
Latest Publications
- Open Access
- 28 April 2026
- Published by Wiley in American Journal of Political Science
Why are surveys struggling to estimate vote shares?
Polling in the 2020 US presidential elections significantly underestimated Trump support, calling into question the accuracy of all political surveys. Although many have speculated that this bias is due to Trump supporters refusing to respond to surveys, we have previously lacked the data to directly evaluate this theory. This paper leverages the rich and unique set of data collected alongside the 2020 American National Election Studies (ANES) to identify and quantify sources of error using the Total Survey Error framework. We document evidence of nonignorable unit nonresponse; that is, Trump supporters were less likely to respond and less likely to complete the 2020 ANES if they did respond. We are also able to rule out other potential sources of error. Our analyses identify a fundamental threat to political surveys, and careful diagnosis and quantification of the bias is critical for informing corrections and future survey design.
- 28 April 2026
- Published by Wiley in American Journal of Political Science
Open AccessWhy are surveys struggling to estimate vote shares?
Polling in the 2020 US presidential elections significantly underestimated Trump support, calling into question the accuracy of all political surveys. Although many have speculated that this bias is due to Trump supporters refusing to respond to surveys, we have previously lacked the data to directly evaluate this theory. This paper leverages the rich and unique set of data collected alongside the 2020 American National Election Studies (ANES) to identify and quantify sources of error using the Total Survey Error framework. We document evidence of nonignorable unit nonresponse; that is, Trump supporters were less likely to respond and less likely to complete the 2020 ANES if they did respond. We are also able to rule out other potential sources of error. Our analyses identify a fundamental threat to political surveys, and careful diagnosis and quantification of the bias is critical for informing corrections and future survey design.
- RetractionOpen Access
- 24 September 2024
- Published by Springer Nature in Nature Human Behaviour
- Vol. 8 (10) , 2067
Retraction Note: High replicability of newly discovered social-behavioural findings is achievable
- 24 September 2024
- Published by Springer Nature in Nature Human Behaviour
- Vol. 8 (10) , 2067
RetractionOpen AccessRetraction Note: High replicability of newly discovered social-behavioural findings is achievable
- 17 April 2024
- Published by Oxford University Press (OUP) in Public Opinion Quarterly
- Vol. 88 (2) , 268-290
Validating the “Genuine Pipeline” to Limit Social Desirability Bias in Survey Estimates of Voter Turnout
It is well documented that survey overreporting of voter turnout due to social desirability bias threatens inference about political behavior. This paper reports four studies that contained question wording experiments to test questions designed to minimize that bias using a “pipeline” approach. The “pipeline” informs survey participants that researchers can perform vote validation to verify turnout self-reports. This approach reduced self-reported turnout by 5.7 points in the 2020 American National Election Study, which represents a majority of the estimated overreporting bias. It reduced reported turnout by 4 points in two nonprobability samples. No effect was found in a third nonprobability study with Amazon Mechanical Turk workers. Validated vote data also confirm that the pipeline approach reduced overreporting. We tested heterogeneous effects for sophistication and several other variables, but results were inconclusive. The pipeline approach reduces overreporting of voter turnout and produces more accurate estimates of voters’ characteristics.
- 17 April 2024
- Published by Oxford University Press (OUP) in Public Opinion Quarterly
- Vol. 88 (2) , 268-290
Validating the “Genuine Pipeline” to Limit Social Desirability Bias in Survey Estimates of Voter Turnout
It is well documented that survey overreporting of voter turnout due to social desirability bias threatens inference about political behavior. This paper reports four studies that contained question wording experiments to test questions designed to minimize that bias using a “pipeline” approach. The “pipeline” informs survey participants that researchers can perform vote validation to verify turnout self-reports. This approach reduced self-reported turnout by 5.7 points in the 2020 American National Election Study, which represents a majority of the estimated overreporting bias. It reduced reported turnout by 4 points in two nonprobability samples. No effect was found in a third nonprobability study with Amazon Mechanical Turk workers. Validated vote data also confirm that the pipeline approach reduced overreporting. We tested heterogeneous effects for sophistication and several other variables, but results were inconclusive. The pipeline approach reduces overreporting of voter turnout and produces more accurate estimates of voters’ characteristics.
- RetractedOpen Access
- 9 November 2023
- Published by Springer Nature in Nature Human Behaviour
- Vol. 8 (2) , 311-319
RETRACTED ARTICLE: High replicability of newly discovered social-behavioural findings is achievable
Failures to replicate evidence of new discoveries have forced scientists to ask whether this unreliability is due to suboptimal implementation of methods or whether presumptively optimal methods are not, in fact, optimal. This paper reports an investigation by four coordinated laboratories of the prospective replicability of 16 novel experimental findings using rigour-enhancing practices: confirmatory tests, large sample sizes, preregistration and methodological transparency. In contrast to past systematic replication efforts that reported replication rates averaging 50%, replication attempts here produced the expected effects with significance testing (P < 0.05) in 86% of attempts, slightly exceeding the maximum expected replicability based on observed effect sizes and sample sizes. When one lab attempted to replicate an effect discovered by another lab, the effect size in the replications was 97% that in the original study. This high replication rate justifies confidence in rigour-enhancing methods to increase the replicability of new discoveries.
- 9 November 2023
- Published by Springer Nature in Nature Human Behaviour
- Vol. 8 (2) , 311-319
RetractedOpen AccessRETRACTED ARTICLE: High replicability of newly discovered social-behavioural findings is achievable
Failures to replicate evidence of new discoveries have forced scientists to ask whether this unreliability is due to suboptimal implementation of methods or whether presumptively optimal methods are not, in fact, optimal. This paper reports an investigation by four coordinated laboratories of the prospective replicability of 16 novel experimental findings using rigour-enhancing practices: confirmatory tests, large sample sizes, preregistration and methodological transparency. In contrast to past systematic replication efforts that reported replication rates averaging 50%, replication attempts here produced the expected effects with significance testing (P < 0.05) in 86% of attempts, slightly exceeding the maximum expected replicability based on observed effect sizes and sample sizes. When one lab attempted to replicate an effect discovered by another lab, the effect size in the replications was 97% that in the original study. This high replication rate justifies confidence in rigour-enhancing methods to increase the replicability of new discoveries.
- 1 January 2023
- Published by Springer Nature
- p. 878-881
Children Living Without Their Fathers
- 1 January 2023
- Published by Springer Nature
- p. 177-179
American National Election Studies
- 15 November 2022
- Published by Oxford University Press (OUP) in Journal of Survey Statistics and Methodology
- Vol. 11 (5) , 991-1010
The Visible Cash Effect with Prepaid Incentives: Evidence for Data Quality, Response Rates, Generalizability, and Cost
A prior study found that mailing prepaid incentives with $5 cash visible from outside the envelope increased the response rate to a mail survey by 4 percentage points compared to cash that was not externally visible. This “visible cash effect” suggests opportunities to improve survey response at little or no cost, but many unknowns remain. Among them: Does the visible cash effect generalize to different survey modes, respondent burdens, and cash amounts? Does it differ between fresh samples and reinterview samples? Does it affect data quality or survey costs? This article examines these questions using two linked studies where incentive visibility was randomized in a large probability sample for the American National Election Studies. The first study used $10 incentives with invitations to a long web questionnaire (median 71 minutes, n = 17,849). Visible cash increased response rates in a fresh sample for both screener and extended interview response (by 6.7 and 4.8 percentage points, respectively). Visible cash did not increase the response rate in a reinterview sample where the baseline reinterview response rate was very high (72 percent). The second study used $5 incentives with invitations to a mail-back paper questionnaire (n = 8,000). Visible cash increased the response rate in a sample of prior nonrespondents by 4.0 percentage points (from 31.5 to 35.5), but it did not increase the response rate in a reinterview sample where the baseline reinterview rate was very high (84 percent). In the two studies, several aspects of data quality were investigated, including speeding, non-differentiation, item nonresponse, nonserious responses, noncredible responses, sample composition, and predictive validity; no adverse effects of visible cash were detected, and sample composition improved marginally. Effects on survey costs were either negligible or resulted in net savings. Accumulated evidence now shows that visible cash can increase incentives’ effectiveness in several circumstances.
- 15 November 2022
- Published by Oxford University Press (OUP) in Journal of Survey Statistics and Methodology
- Vol. 11 (5) , 991-1010
The Visible Cash Effect with Prepaid Incentives: Evidence for Data Quality, Response Rates, Generalizability, and Cost
A prior study found that mailing prepaid incentives with $5 cash visible from outside the envelope increased the response rate to a mail survey by 4 percentage points compared to cash that was not externally visible. This “visible cash effect” suggests opportunities to improve survey response at little or no cost, but many unknowns remain. Among them: Does the visible cash effect generalize to different survey modes, respondent burdens, and cash amounts? Does it differ between fresh samples and reinterview samples? Does it affect data quality or survey costs? This article examines these questions using two linked studies where incentive visibility was randomized in a large probability sample for the American National Election Studies. The first study used $10 incentives with invitations to a long web questionnaire (median 71 minutes, n = 17,849). Visible cash increased response rates in a fresh sample for both screener and extended interview response (by 6.7 and 4.8 percentage points, respectively). Visible cash did not increase the response rate in a reinterview sample where the baseline reinterview response rate was very high (72 percent). The second study used $5 incentives with invitations to a mail-back paper questionnaire (n = 8,000). Visible cash increased the response rate in a sample of prior nonrespondents by 4.0 percentage points (from 31.5 to 35.5), but it did not increase the response rate in a reinterview sample where the baseline reinterview rate was very high (84 percent). In the two studies, several aspects of data quality were investigated, including speeding, non-differentiation, item nonresponse, nonserious responses, noncredible responses, sample composition, and predictive validity; no adverse effects of visible cash were detected, and sample composition improved marginally. Effects on survey costs were either negligible or resulted in net savings. Accumulated evidence now shows that visible cash can increase incentives’ effectiveness in several circumstances.
- 1 March 2022
- Published by Hogrefe Publishing Group in European Journal of Psychological Assessment
- Vol. 38 (2) , 150-161
Improving the Measurement of “Big Five” Personality Traits in a Brief Survey Instrument
The Ten Item Personality Inventory (TIPI) is the leading brief instrument for the “Big Five” personality measurement. However, TIPI’s design has suboptimal features: agree-disagree response options, numeric instead of verbal response labels, and multiple items per page. This paper presents a version of TIPI that addresses these problems. Using two nationally representative sample surveys, we compare the original and revised TIPIs on several dimensions: completion time, item nonresponse, paired item reliability, and validity based on relations to other variables. Completion time is the same and item nonresponse rates are low, while reliability and criterion validity for the revised TIPI is better than the original. The results show how better personality data can be obtained at no additional cost by optimizing questionnaire design.
- 1 March 2022
- Published by Hogrefe Publishing Group in European Journal of Psychological Assessment
- Vol. 38 (2) , 150-161
Improving the Measurement of “Big Five” Personality Traits in a Brief Survey Instrument
The Ten Item Personality Inventory (TIPI) is the leading brief instrument for the “Big Five” personality measurement. However, TIPI’s design has suboptimal features: agree-disagree response options, numeric instead of verbal response labels, and multiple items per page. This paper presents a version of TIPI that addresses these problems. Using two nationally representative sample surveys, we compare the original and revised TIPIs on several dimensions: completion time, item nonresponse, paired item reliability, and validity based on relations to other variables. Completion time is the same and item nonresponse rates are low, while reliability and criterion validity for the revised TIPI is better than the original. The results show how better personality data can be obtained at no additional cost by optimizing questionnaire design.
- 16 February 2022
- Published by Oxford University Press (OUP) in International Journal of Public Opinion Research
- Vol. 34 (1)
Measuring Political Knowledge and Not Search Proficiency in Online Surveys
Most online survey questions testing political knowledge are susceptible to measurement error when participants look up the answers. This article reports five studies of methods to detect and prevent this common source of error. To detect lookups, “catch questions” are more reliable than self-reports, because many participants lie rather than admit looking up answers. Strongly worded instructions reduced lookups by about two-thirds, while the triple combination of instructions, requesting a promise not to look up answers, and adaptive feedback (asking participants who look up an answer to stop doing so) reduced the percentage of respondents looking up an answer by a further half, to 3%. For office recall knowledge items, photo-based open-ended questions eliminated lookups and had similar validity to traditional text-based versions, making them a good choice when a visual format is viable.
- 16 February 2022
- Published by Oxford University Press (OUP) in International Journal of Public Opinion Research
- Vol. 34 (1)
Measuring Political Knowledge and Not Search Proficiency in Online Surveys
Most online survey questions testing political knowledge are susceptible to measurement error when participants look up the answers. This article reports five studies of methods to detect and prevent this common source of error. To detect lookups, “catch questions” are more reliable than self-reports, because many participants lie rather than admit looking up answers. Strongly worded instructions reduced lookups by about two-thirds, while the triple combination of instructions, requesting a promise not to look up answers, and adaptive feedback (asking participants who look up an answer to stop doing so) reduced the percentage of respondents looking up an answer by a further half, to 3%. For office recall knowledge items, photo-based open-ended questions eliminated lookups and had similar validity to traditional text-based versions, making them a good choice when a visual format is viable.
- 1 January 2022
- Published by Springer Nature
- p. 1-3
American National Election Studies