Exploring the Limits of LLMs in Simulating Partisan Polarization with Confirmation Bias Prompts
Open Access
- 20 August 2025
- conference paper
- Published by MDPI AG in Engineering Proceedings
- Vol. 107 (1) , 2
Abstract
In this study, we investigate the potential of large language models (LLMs) to simulate partisan political polarization through conversation experiments. While previous research has demonstrated that LLM agents fail to reproduce human-like partisan polarization due to their inherent biases, we hypothesized that incorporating confirmation bias prompts could help overcome these limitations. We conducted conversation simulations between LLM agents assigned Democratic and Republican ideologies, analyzing both intra-party and inter-party interactions. Results without confirmation bias prompts revealed that agents, particularly those with Republican ideologies, tended to shift toward Democratic positions, failing to replicate human partisan behavior. However, when confirmation bias prompts were introduced, agents maintained their initial political stances more consistently, especially in intra-party conversations. While some tendency toward moderation remained in cross-party discussions, the magnitude of position shifts was significantly reduced. These findings suggest that confirmation bias prompts can effectively mitigate LLMs’ inherent biases in partisan simulations, though additional refinements may be needed to fully replicate human polarization dynamics.Keywords
Funding Information
- JSPS KAKENHI (23H00504)
This publication has 22 references indexed in Scilit:
- The emergence of economic rationality of GPTProceedings of the National Academy of Sciences, 2023
- Role play with large language modelsNature, 2023
- More human than human: measuring ChatGPT political biasPublic Choice, 2023
- An Agent-Based Model of Opinion Polarization Driven by EmotionsComplexity, 2020
- Political psycholinguistics: A comprehensive analysis of the language habits of liberal and conservative social media users.Journal of Personality and Social Psychology, 2020
- Do discussions in like-minded groups necessarily lead to more extreme opinions? Deliberative democracy and group polarizationInternational Political Science Review, 2017
- Modeling confirmation bias and polarizationScientific Reports, 2017
- Political polarization on twitter: Implications for the use of social media in digital governmentsGovernment Information Quarterly, 2016
- The Polarization of Contemporary American PoliticsPolity, 2014
- On the Failure to Eliminate Hypotheses in a Conceptual TaskQuarterly Journal of Experimental Psychology, 1960