The Siren Song of LLMs: How Users Perceive and Respond to Dark Patterns in Large Language Models
Preprint
Abstract
Large language models can influence users through conversation, creating new forms of dark patterns that differ from traditional UX dark patterns. We define LLM dark patterns as manipulative or deceptive behaviors enacted in dialogue. Drawing on prior work and AI incident reports, we outline a diverse set of categories with real-world examples. Using them, we conducted a scenario-based study where participants (N=34) compared manipulative and neutral LLM responses. Our results reveal that recognition of LLM dark patterns often hinged on conversational cues such as exaggerated agreement, biased framing, or privacy intrusions, but these behaviors were also sometimes normalized as ordinary assistance. Users' perceptions of these dark patterns shaped how they respond to them. Responsibilities for these behaviors were also attributed in different ways, with participants assigning it to companies and developers, the model itself, or to users. We conclude with implications for design, advocacy, and governance to safeguard user autonomy.Keywords
All Related Versions
This publication has 23 references indexed in Scilit:
- The AI risk repository: A meta-review, database, and taxonomy of risks from artificial intelligencePatterns, 2026
- A novel evaluation benchmark for medical LLMs illuminating safety and effectiveness in clinical domainsnpj Digital Medicine, 2025
- To Rely or Not to Rely? Evaluating Interventions for Appropriate Reliance on Large Language Models2025
- Deception and manipulation in generative AIPhilosophical Studies, 2025
- Deception abilities emerged in large language modelsProceedings of the National Academy of Sciences, 2024
- From Awareness to Action: Exploring End-User Empowerment Interventions for Dark Patterns in UXProceedings of the ACM on Human-Computer Interaction, 2024
- The Dark Side of Language Models: Exploring the Potential of LLMs in Multimedia Disinformation Generation and DisseminationMachine Learning with Applications, 2024
- Folk theories of algorithmic operations during Internet use: A mixed methods studyThe Information Society, 2021
- One size fits all? What counts as quality practice in (reflexive) thematic analysis?Qualitative Research in Psychology, 2020
- The global landscape of AI ethics guidelinesNature Machine Intelligence, 2019