RTRoger Tourangeau

Roger Tourangeau

*This is an algorithmically generated author record based on publicly available information.

Roger Tourangeau has a total of 35 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Survey Reliability: Models, Methods, and Findings

    by Roger Tourangeau

    Although most survey researchers agree that reliability is a critical requirement for survey data, there have not been many efforts to assess the reliability of responses in national surveys. In addition, there are quite different approaches to studying the reliability of survey responses. In the first section of the Lecture, I contrast a psychological theory of over-time consistency with three statistical models that use reinterview data, multi-trait multi-method experiments, and three-wave panel data to estimate reliability. The more sophisticated statistical models reflect concerns about memory effects and the impact of method factors in reinterview studies. In the following section of the Lecture, I examine some of the major findings from the literature on reliability. Despite the differences across methods for exploring reliability, the findings mostly converge, identifying similar respondent and question characteristics as major determinants of reliability. The next section of the paper looks at the correlations among estimates of reliability derived from the different methods; it finds some support for the validity of the measures from traditional reinterview studies. The empirical claims motivating the more sophisticated methods for estimating reliability are not strongly supported in the literature. Reliability is, in my judgment, a neglected topic among survey researchers, and I hope the Lecture spurs further studies of the reliability of survey questions.


  • Comparing Methods for Assessing Reliability

    by Roger Tourangeau, Hanyu Sun, Ting Yan

    The usual method for assessing the reliability of survey data has been to conduct reinterviews a short interval (such as one to two weeks) after an initial interview and to use these data to estimate relatively simple statistics, such as gross difference rates (GDRs). More sophisticated approaches have also been used to estimate reliability. These include estimates from multi-trait, multi-method experiments, models applied to longitudinal data, and latent class analyses. To our knowledge, no prior study has systematically compared these different methods for assessing reliability. The Population Assessment of Tobacco and Health Reliability and Validity (PATH-RV) Study, done on a national probability sample, assessed the reliability of answers to the Wave 4 questionnaire from the PATH Study. Respondents in the PATH-RV were interviewed twice about two weeks apart. We examined whether the classic survey approach yielded different conclusions from the more sophisticated methods. We also examined two ex ante methods for assessing problems with survey questions and item nonresponse rates and response times to see how strongly these related to the different reliability estimates. We found that kappa was highly correlated with both GDRs and over-time correlations, but the latter two statistics were less highly correlated, particularly for adult respondents; estimates from longitudinal analyses of the same items in the main PATH study were also highly correlated with the traditional reliability estimates. The latent class analysis results, based on fewer items, also showed a high level of agreement with the traditional measures. The other methods and indicators had at best weak relationships with the reliability estimates derived from the reinterview data. Although the Question Understanding Aid seems to tap a different factor from the other measures, for adult respondents, it did predict item nonresponse and response latencies and thus may be a useful adjunct to the traditional measures.


  • Surveying Hard-to-Survey Populations Despite the Unfavorable Environment

    by Roger Tourangeau

    American Journal of Public Health (AJPH) from the American Public Health Association (APHA) Surveying Hard-to-Survey Populations Despite the Unfavorable Environment, an article from American Journal of Public Health, Vol 109 Issue 10


  • How Errors Cumulate: Two Examples

    by Roger Tourangeau

    This article examines the relationship among different types of nonobservation errors (all of which affect estimates from nonprobability internet samples) and between nonresponse and measurement errors. Both are examples of how different error sources can interact. Estimates from nonprobability samples seem to have more total error than estimates from probability samples, even ones with very low response rates. This finding suggests that the combination of coverage, selection, and nonresponse errors has greater cumulative effects than nonresponse error alone. The probabilities of having internet access, joining an internet panel, and responding to a particular survey request are probably correlated and, as a result, may lead to greater covariances with survey variables than response propensities alone; the biases accentuate one another. With nonresponse and measurement error, the two sources seem more or less uncorrelated, with one exception—those most prone to social desirability bias (those in the undesirable categories) are also less likely to respond. In addition, the propensity for unit nonresponse seems to be related to item nonresponse.


  • Web Surveys by Smartphone and Tablets

    by Roger Tourangeau, Aaron Maitland, Gonzalo Rivero, Hanyu Sun, Douglas Williams, Ting Yan

    With respondents increasingly completing web surveys on tablet computers and smartphones, several studies have examined the potential effects of the switch from PCs to mobile devices. The studies have looked at a range of outcomes, including completion rates, breakoffs, and item nonresponse. We carried out a field experiment that compared responses obtained by smartphones, tablets, and laptop computers, focusing on the potential effects of the different devices on measurement errors. We examined whether the differences across devices in screen size (and the related need to scroll to see the entire question or the full set of response options) might moderate the effects of response order, affect the strategy respondents used to decide which of two options was preferable, change the effect of question context, or influence the use of definitions. Our experiments were based on the principle of visual prominence—the idea that respondents are more likely to notice and consider information that is easy to see. The experiments were deliberately designed to maximize the impact of screen size on the results, since the screen size would affect the visual prominence of key information. However, like many of the prior studies examining mobile devices, although response order, context, and evaluation strategy affected the answers respondents gave, few device effects emerged.


  • Open Access

    Responsive Survey Designs for Reducing Nonresponse Bias

    by J. Michael Brick, Roger Tourangeau

    Survey researchers have been investigating alternative approaches to reduce data collection costs while mitigating the risk of nonresponse bias or to produce more accurate estimates within the same budget. Responsive or adaptive design has been suggested as one means for doing this. Falling survey response rates and the need to find effective ways of implementing responsive design has focused attention on the relationship between response rates and nonresponse bias. In our article, we re-examine the data compiled by Groves and Peytcheva (2008) in their influential article and show there is an important between-study component of variance in addition to the within-study variance highlighted in the original analysis. We also show that theory implies that raising response rates can help reduce the nonresponse bias on average across the estimates within a study. We then propose a typology of response propensity models that help explain the empirical findings, including the relative weak relationship between nonresponse rates and nonresponse bias. Using these results, we explore when responsive design tools such as switching modes, giving monetary incentives, and increasing the level of effort are likely to be effective. We conclude with some comments on the use of responsive design and weighting to control nonresponse bias.