TALES: A Taxonomy and Analysis of Cultural Representations in LLM-generated Stories
Open Access
- 13 April 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-26
Abstract
Millions of users across the globe turn to AI chatbots for their creative needs, inviting widespread interest in understanding how they represent diverse cultures. However, evaluating cultural representations in open-ended tasks remains challenging and underexplored. In this work, we present TALES, an evaluation of cultural misrepresentations in LLM-generated stories for diverse Indian cultural identities. First, we develop TALES-Tax, a taxonomy of cultural misrepresentations by collating insights from participants with lived experiences in India through focus groups (N=9) and individual surveys (N=15). Using TALES-Tax, we evaluate 6 models through a large-scale annotation study spanning 2,925 annotations from 108 annotators with lived experience and native language proficiency from across 71 regions in India and 14 languages. Concerningly, we find that 88% of the generated stories contain misrepresentations, and such errors are more prevalent in mid- and low-resourced languages and stories based in peri-urban regions in India. We also transform the annotations into TALES-QA, a standalone question bank to evaluate the cultural knowledge of models.Keywords
All Related Versions
Funding Information
- Schmidt Sciences (G-24-66186)
This publication has 60 references indexed in Scilit:
- Echoes in AI: Quantifying lack of plot diversity in LLM outputsProceedings of the National Academy of Sciences, 2025
- Inspecting the Geographical Representativeness of Images from Text-to-Image Models2023
- More human than human: LLM-generated narratives outperform human-LLM interleaved narratives2023
- AI’s Regimes of Representation: A Community-centered Study of Text-to-Image Models in South Asia2023
- Taxonomy of Machine Learning Safety: A Survey and PrimerACM Computing Surveys, 2022
- How Different Groups Prioritize Ethical Values for Responsible AI2022
- Participatory Design of AI Systems: Opportunities and Challenges Across Diverse Users, Relationships, and Application Domains2022
- Thematic Analysis2022
- Re-imagining Algorithmic Fairness in India and Beyond2021
- Measurement and Fairness2021