A Non-parametric Bayesian Model for Detecting Differential Item Functioning: An Application to Political Representation in the US
Preprint
Abstract
A common approach when studying the quality of representation involves comparing the latent preferences of voters and legislators, commonly obtained by fitting an item-response theory (IRT) model to a common set of stimuli. Despite being exposed to the same stimuli, voters and legislators may not share a common understanding of how these stimuli map onto their latent preferences, leading to differential item-functioning (DIF) and incomparability of estimates. We explore the presence of DIF and incomparability of latent preferences obtained through IRT models by re-analyzing an influential survey data set, where survey respondents expressed their preferences on roll call votes that U.S. legislators had previously voted on. To do so, we propose defining a Dirichlet Process prior over item-response functions in standard IRT models. In contrast to typical multi-step approaches to detecting DIF, our strategy allows researchers to fit a single model, automatically identifying incomparable sub-groups with different mappings from latent traits onto observed responses. We find that although there is a group of voters whose estimated positions can be safely compared to those of legislators, a sizeable share of surveyed voters understand stimuli in fundamentally different ways. Ignoring these issues can lead to incorrect conclusions about the quality of representation.Keywords
All Related Versions
This publication has 23 references indexed in Scilit:
- Estimating Individuals’ Political Perceptions While Adjusting for Differential Item FunctioningPolitical Analysis, 2020
- Bayesian Repulsive Gaussian Mixture ModelJournal of the American Statistical Association, 2019
- Fast Estimation of Ideal Points with Massive DataAmerican Political Science Review, 2016
- Network Structure and Social Outcomes: Network Analysis for Social Science2016
- Ideology and Spatial Voting in American Elections2012
- Apples and Oranges? The Problem of Equivalence in Comparative ResearchPolitical Analysis, 2011
- Leapfrog Representation and Extremism: A Study of American Voters and Their Members in CongressAmerican Political Science Review, 2010
- Characterizing the variance improvement in linear Dirichlet random effects modelsStatistics & Probability Letters, 2009
- A Bayesian Semiparametric Item Response Model with Dirichlet Process PriorsPsychometrika, 2009
- Enhancing the Validity and Cross-Cultural Comparability of Measurement in Survey ResearchAmerican Political Science Review, 2004