Disaggregated Health Data in LLMs: Evaluating Data Equity in the Context of Asian American Representation
Open Access
- 15 October 2025
- journal article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society
- Vol. 8 (1) , 292-303
Abstract
Large language models (LLMs), such as ChatGPT and Claude, have emerged as essential tools for information retrieval, often serving as alternatives to traditional search engines. However, ensuring that these models provide accurate and equitable information tailored to diverse demographic groups remains an important challenge. This study investigates the capability of LLMs to retrieve disaggregated health-related information for sub-ethnic groups within the Asian American population, such as Korean and Chinese communities. Data disaggregation has been a critical practice in health research to address inequities, making it an ideal domain for evaluating representation equity in LLM outputs. We apply a suite of statistical and machine learning tools to assess whether LLMs deliver appropriately disaggregated and equitable information. By focusing on Asian American sub-ethnic groups—a highly diverse population often aggregated in traditional analyses—we highlight how LLMs handle complex disparities in health data. Our findings contribute to ongoing discussions about responsible AI, particularly in ensuring data equity in the outputs of LLM-based systems.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Mental health among older Bhutanese with a refugee life experience: A mixed-methods latent class analysis study.American Journal of Orthopsychiatry, 2023
- Just data representation for Native Hawaiians and Pacific Islanders: A critical review of systemic Indigenous erasure in census and recommendations for psychologistsAmerican Journal of Community Psychology, 2021
- Capturing Racial/Ethnic Diversity in Population-Based Surveys: Data Disaggregation of Health Data for Asian American, Native Hawaiian, and Pacific Islanders (AANHPIs)Population Research and Policy Review, 2021
- Cancer health disparities in racial/ethnic minorities in the United StatesBritish Journal of Cancer, 2020
- Striking Racial/Ethnic Disparities in Liver Cancer Incidence Rates and Temporal Trends in California, 1988–2012JNCI: Journal of the National Cancer Institute, 2018
- Stomach Cancer Disparity among Korean Americans by Tumor Characteristics: Comparison with Non-Hispanic Whites, Japanese Americans, South Koreans, and JapaneseCancer Epidemiology, Biomarkers & Prevention, 2017
- hdbscan: Hierarchical density based clusteringJournal of Open Source Software, 2017
- Trauma and mental health of medics in eastern Myanmar’s conflict zones: a cross-sectional and mixed methods investigationConflict and Health, 2013
- Hmong Mental Health Needs Assessment: A Community‐Based Partnership in a Small Mid‐Western CommunityAmerican Journal of Community Psychology, 2011
- Rates and Correlates of Seeking Mental Health Services Among Cambodian RefugeesAmerican Journal of Public Health, 2006