MDMatthew DeBell

Matthew DeBell

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Matthew DeBell has a total of 46 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Why are surveys struggling to estimate vote shares?

    by Matthew Tyler, D. Sunshine Hillygus, Matthew DeBell, Ted Brader, Shanto Iyengar, Daron Shaw, Nicholas A. Valentino

    Polling in the 2020 US presidential elections significantly underestimated Trump support, calling into question the accuracy of all political surveys. Although many have speculated that this bias is due to Trump supporters refusing to respond to surveys, we have previously lacked the data to directly evaluate this theory. This paper leverages the rich and unique set of data collected alongside the 2020 American National Election Studies (ANES) to identify and quantify sources of error using the Total Survey Error framework. We document evidence of nonignorable unit nonresponse; that is, Trump supporters were less likely to respond and less likely to complete the 2020 ANES if they did respond. We are also able to rule out other potential sources of error. Our analyses identify a fundamental threat to political surveys, and careful diagnosis and quantification of the bias is critical for informing corrections and future survey design.


  • Retraction
    Open Access

    Retraction Note: High replicability of newly discovered social-behavioural findings is achievable

    by John Protzko, Jon Krosnick, Leif Nelson, Brian A. Nosek, Jordan Axt, Matt Berent, Nicholas Buttrick, Matthew DeBell, Charles R. Ebersole, Sebastian Lundmark, et al.


  • Validating the “Genuine Pipeline” to Limit Social Desirability Bias in Survey Estimates of Voter Turnout

    by Matthew DeBell, D Sunshine Hillygus, Daron R Shaw, Nicholas A Valentino

    It is well documented that survey overreporting of voter turnout due to social desirability bias threatens inference about political behavior. This paper reports four studies that contained question wording experiments to test questions designed to minimize that bias using a “pipeline” approach. The “pipeline” informs survey participants that researchers can perform vote validation to verify turnout self-reports. This approach reduced self-reported turnout by 5.7 points in the 2020 American National Election Study, which represents a majority of the estimated overreporting bias. It reduced reported turnout by 4 points in two nonprobability samples. No effect was found in a third nonprobability study with Amazon Mechanical Turk workers. Validated vote data also confirm that the pipeline approach reduced overreporting. We tested heterogeneous effects for sophistication and several other variables, but results were inconclusive. The pipeline approach reduces overreporting of voter turnout and produces more accurate estimates of voters’ characteristics.


  • Retracted
    Open Access

    RETRACTED ARTICLE: High replicability of newly discovered social-behavioural findings is achievable

    by John Protzko, Jon Krosnick, Leif Nelson, Brian A. Nosek, Jordan Axt, Matt Berent, Nicholas Buttrick, Matthew DeBell, Charles R. Ebersole, Sebastian Lundmark, et al.

    Failures to replicate evidence of new discoveries have forced scientists to ask whether this unreliability is due to suboptimal implementation of methods or whether presumptively optimal methods are not, in fact, optimal. This paper reports an investigation by four coordinated laboratories of the prospective replicability of 16 novel experimental findings using rigour-enhancing practices: confirmatory tests, large sample sizes, preregistration and methodological transparency. In contrast to past systematic replication efforts that reported replication rates averaging 50%, replication attempts here produced the expected effects with significance testing (P < 0.05) in 86% of attempts, slightly exceeding the maximum expected replicability based on observed effect sizes and sample sizes. When one lab attempted to replicate an effect discovered by another lab, the effect size in the replications was 97% that in the original study. This high replication rate justifies confidence in rigour-enhancing methods to increase the replicability of new discoveries.


  • The Visible Cash Effect with Prepaid Incentives: Evidence for Data Quality, Response Rates, Generalizability, and Cost

    by Matthew DeBell

    A prior study found that mailing prepaid incentives with $5 cash visible from outside the envelope increased the response rate to a mail survey by 4 percentage points compared to cash that was not externally visible. This “visible cash effect” suggests opportunities to improve survey response at little or no cost, but many unknowns remain. Among them: Does the visible cash effect generalize to different survey modes, respondent burdens, and cash amounts? Does it differ between fresh samples and reinterview samples? Does it affect data quality or survey costs? This article examines these questions using two linked studies where incentive visibility was randomized in a large probability sample for the American National Election Studies. The first study used $10 incentives with invitations to a long web questionnaire (median 71 minutes, n = 17,849). Visible cash increased response rates in a fresh sample for both screener and extended interview response (by 6.7 and 4.8 percentage points, respectively). Visible cash did not increase the response rate in a reinterview sample where the baseline reinterview response rate was very high (72 percent). The second study used $5 incentives with invitations to a mail-back paper questionnaire (n = 8,000). Visible cash increased the response rate in a sample of prior nonrespondents by 4.0 percentage points (from 31.5 to 35.5), but it did not increase the response rate in a reinterview sample where the baseline reinterview rate was very high (84 percent). In the two studies, several aspects of data quality were investigated, including speeding, non-differentiation, item nonresponse, nonserious responses, noncredible responses, sample composition, and predictive validity; no adverse effects of visible cash were detected, and sample composition improved marginally. Effects on survey costs were either negligible or resulted in net savings. Accumulated evidence now shows that visible cash can increase incentives’ effectiveness in several circumstances.


  • Improving the Measurement of “Big Five” Personality Traits in a Brief Survey Instrument

    by Matthew DeBell, Natalya Maisel, Ted Brader, Catherine Wilson, Simon Jackman

    The Ten Item Personality Inventory (TIPI) is the leading brief instrument for the “Big Five” personality measurement. However, TIPI’s design has suboptimal features: agree-disagree response options, numeric instead of verbal response labels, and multiple items per page. This paper presents a version of TIPI that addresses these problems. Using two nationally representative sample surveys, we compare the original and revised TIPIs on several dimensions: completion time, item nonresponse, paired item reliability, and validity based on relations to other variables. Completion time is the same and item nonresponse rates are low, while reliability and criterion validity for the revised TIPI is better than the original. The results show how better personality data can be obtained at no additional cost by optimizing questionnaire design.


  • Measuring Political Knowledge and Not Search Proficiency in Online Surveys

    by Matthew DeBell

    Most online survey questions testing political knowledge are susceptible to measurement error when participants look up the answers. This article reports five studies of methods to detect and prevent this common source of error. To detect lookups, “catch questions” are more reliable than self-reports, because many participants lie rather than admit looking up answers. Strongly worded instructions reduced lookups by about two-thirds, while the triple combination of instructions, requesting a promise not to look up answers, and adaptive feedback (asking participants who look up an answer to stop doing so) reduced the percentage of respondents looking up an answer by a further half, to 3%. For office recall knowledge items, photo-based open-ended questions eliminated lookups and had similar validity to traditional text-based versions, making them a good choice when a visual format is viable.