Whose Opinions Do Language Models Reflect?
Preprint
Abstract
Language models (LMs) are increasingly being used in open-ended contexts, where the opinions reflected by LMs in response to subjective queries can have a profound impact, both on user satisfaction, as well as shaping the views of society at large. In this work, we put forth a quantitative framework to investigate the opinions reflected by LMs -- by leveraging high-quality public opinion polls and their associated human responses. Using this framework, we create OpinionsQA, a new dataset for evaluating the alignment of LM opinions with those of 60 US demographic groups over topics ranging from abortion to automation. Across topics, we find substantial misalignment between the views reflected by current LMs and those of US demographic groups: on par with the Democrat-Republican divide on climate change. Notably, this misalignment persists even after explicitly steering the LMs towards particular demographic groups. Our analysis not only confirms prior observations about the left-leaning tendencies of some human feedback-tuned LMs, but also surfaces groups whose opinions are poorly reflected by current LMs (e.g., 65+ and widowed individuals). Our code and data are available at https://github.com/tatsu-lab/opinions_qa.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Out of One, Many: Using Language Models to Simulate Human SamplesPolitical Analysis, 2023
- Using cognitive psychology to understand GPT-3Proceedings of the National Academy of Sciences, 2023
- Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity AnnotationProceedings of the ACM on Human-Computer Interaction, 2022
- Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective AnnotationsTransactions of the Association for Computational Linguistics, 2022
- Red Teaming Language Models with Language Models2022
- Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection2022
- Consequences of Asking Sensitive Questions in SurveysAnnual Review of Statistics and Its Application, 2021
- TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation2021
- StereoSet: Measuring stereotypical bias in pretrained language models2021
- Inherent Disagreements in Human Textual InferencesTransactions of the Association for Computational Linguistics, 2019