Yonaira M. Rivera
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Yonaira M. Rivera has a total of 47 co-authors. Here are the top co-authors by number of shared publications:
- Johannes Thrul
Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, USA
5 shared publications - Katherine Clegg Smith
Johns Hopkins University Bloomberg School of Public Health
5 shared publications - Anna Rose Johnson
Division of Plastic and Reconstructive Surgery, Department of Surgery, Washington University School of Medicine, Saint Louis, MO, United States
4 shared publications - Benjamin Ormseth
Department of Plastic Surgery, University of Texas Southwestern Medical Center, Dallas, TX, United States
4 shared publications - Clara Nan-Hi Lee
Department of Surgery, Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
4 shared publications - Corinne E. Joshu
Department of Epidemiology Johns Hopkins Bloomberg School of Public Health Baltimore Maryland USA
4 shared publications
Latest Publications
- Open Access
- 26 May 2026
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 10, e87465
Differences in Safety Risks Across Languages in Health-Relevant Queries: Vulnerability Analysis of Large Language Model Responses
Background: Large language models (LLMs) such as ChatGPT are increasingly used to support health-related queries and decision-making. However, these models can be “jailbroken” through adversarial prompts that bypass safety filters and elicit harmful or medically inappropriate responses. In health care contexts, such vulnerabilities pose serious risks. Understanding how jailbreak susceptibility varies across languages is essential for developing robust safeguards and promoting equitable access to safe health information. This paper may contain examples that may be deemed harmful in terms of violence, self-harm, and drug abuse. Objective: This study aims to systematically compare and contrast the vulnerability of a health LLM for jailbreaking across 3 languages: English, Spanish, and Hindi (transliterated using the Latin alphabet), based on emoji and permutation cipher attacks. Methods: We analyzed 1000 input prompts per language, drawn from the BeaverTails dataset, across 3 harm categories: self-harm, violence, and drug abuse. Each prompt was modified using emoji and permutation cipher techniques, resulting in 6000 input-output pairs. Model responses were evaluated by human coders to determine the success rate of jailbreak attempts across languages and cipher types. Results: Hindi prompts showed the highest vulnerability, with 787 successful jailbreaks using emoji ciphers and 873 using permutation ciphers. Spanish and English followed, with lower success rates across both cipher types. Differences in jailbreak success across languages and cipher strategies were statistically significant. Additionally, attacks targeting violence-related prompts were more successful overall than those targeting drug-related or self-harm content, indicating variation in vulnerability by harm type. Conclusions: The findings of this formative study reveal that LLM safety performance varies substantially across languages and harm categories, raising concerns about equitable protection in multilingual health communication. Disparities in access to harmful content may contribute to downstream health risks. Strengthening multilingual content moderation and developing language-aware safety mechanisms are critical steps toward creating safer and more inclusive health AI systems.
- 26 May 2026
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 10, e87465
Open AccessDifferences in Safety Risks Across Languages in Health-Relevant Queries: Vulnerability Analysis of Large Language Model Responses
Background: Large language models (LLMs) such as ChatGPT are increasingly used to support health-related queries and decision-making. However, these models can be “jailbroken” through adversarial prompts that bypass safety filters and elicit harmful or medically inappropriate responses. In health care contexts, such vulnerabilities pose serious risks. Understanding how jailbreak susceptibility varies across languages is essential for developing robust safeguards and promoting equitable access to safe health information. This paper may contain examples that may be deemed harmful in terms of violence, self-harm, and drug abuse. Objective: This study aims to systematically compare and contrast the vulnerability of a health LLM for jailbreaking across 3 languages: English, Spanish, and Hindi (transliterated using the Latin alphabet), based on emoji and permutation cipher attacks. Methods: We analyzed 1000 input prompts per language, drawn from the BeaverTails dataset, across 3 harm categories: self-harm, violence, and drug abuse. Each prompt was modified using emoji and permutation cipher techniques, resulting in 6000 input-output pairs. Model responses were evaluated by human coders to determine the success rate of jailbreak attempts across languages and cipher types. Results: Hindi prompts showed the highest vulnerability, with 787 successful jailbreaks using emoji ciphers and 873 using permutation ciphers. Spanish and English followed, with lower success rates across both cipher types. Differences in jailbreak success across languages and cipher strategies were statistically significant. Additionally, attacks targeting violence-related prompts were more successful overall than those targeting drug-related or self-harm content, indicating variation in vulnerability by harm type. Conclusions: The findings of this formative study reveal that LLM safety performance varies substantially across languages and harm categories, raising concerns about equitable protection in multilingual health communication. Disparities in access to harmful content may contribute to downstream health risks. Strengthening multilingual content moderation and developing language-aware safety mechanisms are critical steps toward creating safer and more inclusive health AI systems.
- Open Access
- 13 February 2026
- Published by JMIR Publications Inc. in JMIR Cancer
- Vol. 12, e87683
Correction: Social Media as a Platform for Cancer Care Decision-Making Among Women: Internet Survey-Based Study on Trust, Engagement, and Preferences
[This corrects the article DOI: 10.2196/64724.].
- 13 February 2026
- Published by JMIR Publications Inc. in JMIR Cancer
- Vol. 12, e87683
Open AccessCorrection: Social Media as a Platform for Cancer Care Decision-Making Among Women: Internet Survey-Based Study on Trust, Engagement, and Preferences
[This corrects the article DOI: 10.2196/64724.].
- Open Access
- 9 November 2025
- Published by JMIR Publications Inc. in JMIR Preprints
Differences in Safety Risks across Languages for Health Large Language Models: A Cross-Language Vulnerability Study (Preprint)
BACKGROUND Large language models (LLMs) such as ChatGPT are increasingly used to support health-related queries and decision-making. However, these models can be “jailbroken” through adversarial prompts that bypass safety filters and elicit harmful or medically inappropriate responses. In healthcare contexts, such vulnerabilities pose serious risks. Understanding how jailbreak susceptibility varies across languages is essential for developing robust safeguards and promoting equitable access to safe health information. OBJECTIVE This study aims to systematically compare and contrast the vulnerability of a health LLM for jailbreaking across three languages: English, Spanish, and Hindi (transliterated using the Latin alphabet) based on emoji and permutation cipher attacks. METHODS We analyzed 1,000 input prompts per language, drawn from the BeaverTails dataset, across three harm categories: self-harm, violence, and drug abuse. Each prompt was modified using emoji and permutation cipher techniques, resulting in 6,000 input-output pairs. Model responses were evaluated by human coders to determine the success rate of jailbreak attempts across languages and cipher types. RESULTS Hindi prompts showed the highest vulnerability, with 787 successful jailbreaks using emoji ciphers and 873 using permutation ciphers. Spanish and English followed, with lower success rates across both cipher types. Differences in jailbreak success across languages and cipher strategies were statistically significant. Additionally, attacks targeting violence-related prompts were more successful overall than those targeting drug-related or self-harm content, indicating variation in vulnerability by harm type. CONCLUSIONS The findings of this formative study reveal that LLM safety performance varies substantially across languages and harm categories, raising concerns about equitable protection in multilingual health communication. Disparities in access to harmful content may contribute to downstream health risks. Strengthening multilingual content moderation and developing language-aware safety mechanisms are critical steps toward safer and more inclusive health AI systems.
- 9 November 2025
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessDifferences in Safety Risks across Languages for Health Large Language Models: A Cross-Language Vulnerability Study (Preprint)
BACKGROUND Large language models (LLMs) such as ChatGPT are increasingly used to support health-related queries and decision-making. However, these models can be “jailbroken” through adversarial prompts that bypass safety filters and elicit harmful or medically inappropriate responses. In healthcare contexts, such vulnerabilities pose serious risks. Understanding how jailbreak susceptibility varies across languages is essential for developing robust safeguards and promoting equitable access to safe health information. OBJECTIVE This study aims to systematically compare and contrast the vulnerability of a health LLM for jailbreaking across three languages: English, Spanish, and Hindi (transliterated using the Latin alphabet) based on emoji and permutation cipher attacks. METHODS We analyzed 1,000 input prompts per language, drawn from the BeaverTails dataset, across three harm categories: self-harm, violence, and drug abuse. Each prompt was modified using emoji and permutation cipher techniques, resulting in 6,000 input-output pairs. Model responses were evaluated by human coders to determine the success rate of jailbreak attempts across languages and cipher types. RESULTS Hindi prompts showed the highest vulnerability, with 787 successful jailbreaks using emoji ciphers and 873 using permutation ciphers. Spanish and English followed, with lower success rates across both cipher types. Differences in jailbreak success across languages and cipher strategies were statistically significant. Additionally, attacks targeting violence-related prompts were more successful overall than those targeting drug-related or self-harm content, indicating variation in vulnerability by harm type. CONCLUSIONS The findings of this formative study reveal that LLM safety performance varies substantially across languages and harm categories, raising concerns about equitable protection in multilingual health communication. Disparities in access to harmful content may contribute to downstream health risks. Strengthening multilingual content moderation and developing language-aware safety mechanisms are critical steps toward safer and more inclusive health AI systems.
- 4 June 2025
- Published by Oxford University Press (OUP) in Journal of Communication
Leveraging digital spaces and datafication in communication research: contributions of digital qualitative fluidity to ethnographic interviewing
The digitalization of communication practices affords major gains for qualitative methods of communication science. What we term digital qualitative fluidity captures the way traditional boundaries between interviewing and ethnography may dissolve through the use of contemporary, digital technologies. We illustrate how communication scholars have leveraged the digital to open ethnographic spaces within interviews to examine situated communication processes of meaning-making in action, amplifying marginalized voices, evoking affect, and enabling sense making–better meeting key goals of qualitative research. We discuss examples of harnessing the fluidity of digital ethnographic interviews in different corners of communication science, dilemmas that accompany this approach, and institutional investments to ensure that qualitative data and methods are valued alongside computational approaches.
- 4 June 2025
- Published by Oxford University Press (OUP) in Journal of Communication
Leveraging digital spaces and datafication in communication research: contributions of digital qualitative fluidity to ethnographic interviewing
The digitalization of communication practices affords major gains for qualitative methods of communication science. What we term digital qualitative fluidity captures the way traditional boundaries between interviewing and ethnography may dissolve through the use of contemporary, digital technologies. We illustrate how communication scholars have leveraged the digital to open ethnographic spaces within interviews to examine situated communication processes of meaning-making in action, amplifying marginalized voices, evoking affect, and enabling sense making–better meeting key goals of qualitative research. We discuss examples of harnessing the fluidity of digital ethnographic interviews in different corners of communication science, dilemmas that accompany this approach, and institutional investments to ensure that qualitative data and methods are valued alongside computational approaches.
- 1 May 2025
- Published by Taylor & Francis in Health Communication
- Vol. 40 (14) , 3126-3138
Exploring the role ofculture and cognitive heuristics in assessing the credibility of cancer (mis)information on Facebook among U.S. Latinos/as
As health misinformation continues to permeate social media, it is of utmost importance to understand how credibility assessments are made among those most vulnerable to health inequities. Dual-processing models suggest that online credibility assessments are largely made through the use of cognitive heuristics. However, little is known about the role cultural identity plays in these assessments, particularly in light of health (mis)information exposure among communities of color. This mixed-methods study explores the role of culture and cognitive heuristics in how a sample of Latinos/as assessed the credibility of cancer prevention and screening information on Facebook, and the quality of cancer information they were exposed to on their accounts. Participants (N = 20) logged onto their Facebook account, entered the search term "cancer," and walked-through 12 months of cancer-related posts that appeared on their News Feed. When cancer screening and prevention information engagement was recalled, participants were asked if/how they assessed information credibility. Computer screen and audio were recorded. Interviews were analyzed thematically, and the content and scientific credibility of claims in posts were assessed. Despite the majority of posts being coded as having low/very low scientific credibility, credibility assessments were not always accurate. Most participants only verified the accuracy of content when they deemed it important/relevant, instead relying on cognitive heuristics interpreted through a cultural lens to assess (mis)information. Findings highlight the instrumental role of culture in the heuristic interpretation of source and content credibility among Latinos/as, emphasizing the importance of explicitly including culture in theoretical models that explore how individuals assess credibility on social media.
- 1 May 2025
- Published by Taylor & Francis in Health Communication
- Vol. 40 (14) , 3126-3138
Exploring the role ofculture and cognitive heuristics in assessing the credibility of cancer (mis)information on Facebook among U.S. Latinos/as
As health misinformation continues to permeate social media, it is of utmost importance to understand how credibility assessments are made among those most vulnerable to health inequities. Dual-processing models suggest that online credibility assessments are largely made through the use of cognitive heuristics. However, little is known about the role cultural identity plays in these assessments, particularly in light of health (mis)information exposure among communities of color. This mixed-methods study explores the role of culture and cognitive heuristics in how a sample of Latinos/as assessed the credibility of cancer prevention and screening information on Facebook, and the quality of cancer information they were exposed to on their accounts. Participants (N = 20) logged onto their Facebook account, entered the search term "cancer," and walked-through 12 months of cancer-related posts that appeared on their News Feed. When cancer screening and prevention information engagement was recalled, participants were asked if/how they assessed information credibility. Computer screen and audio were recorded. Interviews were analyzed thematically, and the content and scientific credibility of claims in posts were assessed. Despite the majority of posts being coded as having low/very low scientific credibility, credibility assessments were not always accurate. Most participants only verified the accuracy of content when they deemed it important/relevant, instead relying on cognitive heuristics interpreted through a cultural lens to assess (mis)information. Findings highlight the instrumental role of culture in the heuristic interpretation of source and content credibility among Latinos/as, emphasizing the importance of explicitly including culture in theoretical models that explore how individuals assess credibility on social media.
- Open Access
- 5 March 2025
- Published by JMIR Publications Inc. in JMIR Cancer
- Vol. 11 (1) , e64724
Social Media as a Platform for Cancer Care Decision-Making Among Women: Internet Survey-Based Study on Trust, Engagement, and Preferences
Background: Decision aids improve patient and clinician decision-making but are underused and often restricted to clinical settings. Objective: Given limited studies analyzing the feasibility of disseminating decision aids through social media, this study aimed to evaluate the acceptability, trust, and engagement of women with social media as a tool to deliver online decision aids for cancer treatment. Methods: To prepare for potential dissemination of a breast cancer decision aid via social media, a cross-sectional survey in February 2023 was conducted via Prime Panels, an online market research platform, of women aged 35-75 years in the United States. Demographics, health, cancer information-seeking behaviors, social media use, trust in social media for health information, as well as the likelihood of viewing cancer-related health information and clicking on decision aids through social media, were assessed. Statistical analyses included descriptive statistics, correlations, and multivariable ordinal regression. Results: Of 607 respondents, 397 (65.4%) had searched for cancer information, with 185 (46.6%) using the internet as their primary source. Facebook (Meta) was the most popular platform (511/607, 84.2%). Trust in social media for health information was higher among Black (14/72, 19.4%) and Asian respondents (7/27, 25.9%) than among White respondents (49/480, 10.2%; P=.003). Younger respondents aged 35-39 years (17/82, 20.7%) showed higher trust than those aged 70-79 years (12/70, 17.1%; P<.001). Trust in social media for health information was linked to a higher likelihood of viewing cancer information and accessing a decision aid online (P<.001). Participants who rated social media as “Trustworthy” (n=73) were more likely to view cancer information (61/73, 83.6%) and click on decision aids (61/73, 83.6%) than those who found it “Untrustworthy” (n=277; view: 133/277, 48.0%; click: 125/277, 45.1%). Engagement with social media positively correlated with viewing online cancer information (Spearman ρ=0.20, P<.001) and willingness to use decision aids (ρ=0.21, P<.001). Multivariable ordinal regression analyses confirmed that perception of social media’s trustworthiness is a significant predictor of engagement with decision aids (untrustworthy vs trustworthy β=–1.826, P<.001; neutral vs trustworthy β=–0.926, P=.007) and of viewing cancer information (untrustworthy vs trustworthy β=–1.680, P<.001, neutral vs trustworthy β=–0.581, P=.098), while age and employment status were not significant predictors. Conclusions: This exploratory study suggests that social media platforms may increase access to health information and decision aids. No significant differences were observed between demographic variables and the use or trust in social media for health information. However, trust in social media emerged as a mediating factor between demographics and engagement with cancer information online. Before disseminating decision aids on social media, groups should identify existing trust and engagement patterns with different platforms within their target demographic.
- 5 March 2025
- Published by JMIR Publications Inc. in JMIR Cancer
- Vol. 11 (1) , e64724
Open AccessSocial Media as a Platform for Cancer Care Decision-Making Among Women: Internet Survey-Based Study on Trust, Engagement, and Preferences
Background: Decision aids improve patient and clinician decision-making but are underused and often restricted to clinical settings. Objective: Given limited studies analyzing the feasibility of disseminating decision aids through social media, this study aimed to evaluate the acceptability, trust, and engagement of women with social media as a tool to deliver online decision aids for cancer treatment. Methods: To prepare for potential dissemination of a breast cancer decision aid via social media, a cross-sectional survey in February 2023 was conducted via Prime Panels, an online market research platform, of women aged 35-75 years in the United States. Demographics, health, cancer information-seeking behaviors, social media use, trust in social media for health information, as well as the likelihood of viewing cancer-related health information and clicking on decision aids through social media, were assessed. Statistical analyses included descriptive statistics, correlations, and multivariable ordinal regression. Results: Of 607 respondents, 397 (65.4%) had searched for cancer information, with 185 (46.6%) using the internet as their primary source. Facebook (Meta) was the most popular platform (511/607, 84.2%). Trust in social media for health information was higher among Black (14/72, 19.4%) and Asian respondents (7/27, 25.9%) than among White respondents (49/480, 10.2%; P=.003). Younger respondents aged 35-39 years (17/82, 20.7%) showed higher trust than those aged 70-79 years (12/70, 17.1%; P<.001). Trust in social media for health information was linked to a higher likelihood of viewing cancer information and accessing a decision aid online (P<.001). Participants who rated social media as “Trustworthy” (n=73) were more likely to view cancer information (61/73, 83.6%) and click on decision aids (61/73, 83.6%) than those who found it “Untrustworthy” (n=277; view: 133/277, 48.0%; click: 125/277, 45.1%). Engagement with social media positively correlated with viewing online cancer information (Spearman ρ=0.20, P<.001) and willingness to use decision aids (ρ=0.21, P<.001). Multivariable ordinal regression analyses confirmed that perception of social media’s trustworthiness is a significant predictor of engagement with decision aids (untrustworthy vs trustworthy β=–1.826, P<.001; neutral vs trustworthy β=–0.926, P=.007) and of viewing cancer information (untrustworthy vs trustworthy β=–1.680, P<.001, neutral vs trustworthy β=–0.581, P=.098), while age and employment status were not significant predictors. Conclusions: This exploratory study suggests that social media platforms may increase access to health information and decision aids. No significant differences were observed between demographic variables and the use or trust in social media for health information. However, trust in social media emerged as a mediating factor between demographics and engagement with cancer information online. Before disseminating decision aids on social media, groups should identify existing trust and engagement patterns with different platforms within their target demographic.
- Open Access
- 24 February 2025
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 27 (1) , e59387
Engagement With and Use of Health Information on Social Media Among US Latino Individuals: National Cross-Sectional Survey Study
Background: During the COVID-19 pandemic, US Latino individuals were more likely to report accessing coronavirus information on social media than other groups, despite copious amounts of health misinformation documented on these platforms. Among the existing literature on factors associated with engagement and use of health information, racial minority status has been associated with greater susceptibility to health misinformation. However, literature to date has not reported national trends on how Latino individuals engage with or use health information on social media compared to non-Latino White (NLW) individuals, nor whether perceptions of the amount of health misinformation on social media influence health information engagement and usage. Objective: This study aimed to examine differences in engagement with and use of health information on social media among Latino and NLW individuals in the United States. Methods: We examined a nationally representative cross-sectional sample of Latino (n=827) and NLW (n=2563) respondents of the 2022 Health Information National Trends Survey who used social media in 2022 to assess differences in engagement with and use of health information. Items related to the perceived quantity of health misinformation on social media, social media use frequency, health information engagement (sharing content; watching videos), and health information usage (health decision-making; discussions with health care providers) were selected to conduct weighted bivariate analyses and logistic regressions. Results: Latino individuals perceive lower amounts of health misinformation on social media (28.9% perceived little to no misinformation vs 13.6% NLW individuals, P<.001). Latino audiences also reported higher health information engagement compared to NLW individuals (20% vs 10.2% shared information several times a month or more, P<.001; 42.4% vs 27.2% watched videos several times a month or more, P<.001), as well as higher information usage for health decision-making (22.8% vs 13.7%, P=.003). When controlling for ethnicity and other sociodemographic variables, perceiving lower amounts of health misinformation on social media was associated with higher odds of watching videos more frequently, making health decisions, and discussing health-related content with a health care provider (P<.001). Furthermore, Latino audiences were 1.85 times more likely to watch videos (P<.001), when controlling for the perceived amount of health misinformation and other sociodemographic variables. Finally, when compared to NLW individuals perceiving little to no health misinformation, Latino audiences perceiving little to no health misinformation were 2.91 times more likely to watch videos (P<.001). Conclusions: The findings suggest that Latino individuals engage with visual health (mis)information at higher rates. Digital health literacy interventions should consider video formats and preferred social media platforms among Latino individuals. Further research is warranted to understand sociocultural factors important to Latino social media users when consuming health information, as these may impact the success of digital media literacy interventions that teach users how to navigate misinformation online.
- 24 February 2025
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 27 (1) , e59387
Open AccessEngagement With and Use of Health Information on Social Media Among US Latino Individuals: National Cross-Sectional Survey Study
Background: During the COVID-19 pandemic, US Latino individuals were more likely to report accessing coronavirus information on social media than other groups, despite copious amounts of health misinformation documented on these platforms. Among the existing literature on factors associated with engagement and use of health information, racial minority status has been associated with greater susceptibility to health misinformation. However, literature to date has not reported national trends on how Latino individuals engage with or use health information on social media compared to non-Latino White (NLW) individuals, nor whether perceptions of the amount of health misinformation on social media influence health information engagement and usage. Objective: This study aimed to examine differences in engagement with and use of health information on social media among Latino and NLW individuals in the United States. Methods: We examined a nationally representative cross-sectional sample of Latino (n=827) and NLW (n=2563) respondents of the 2022 Health Information National Trends Survey who used social media in 2022 to assess differences in engagement with and use of health information. Items related to the perceived quantity of health misinformation on social media, social media use frequency, health information engagement (sharing content; watching videos), and health information usage (health decision-making; discussions with health care providers) were selected to conduct weighted bivariate analyses and logistic regressions. Results: Latino individuals perceive lower amounts of health misinformation on social media (28.9% perceived little to no misinformation vs 13.6% NLW individuals, P<.001). Latino audiences also reported higher health information engagement compared to NLW individuals (20% vs 10.2% shared information several times a month or more, P<.001; 42.4% vs 27.2% watched videos several times a month or more, P<.001), as well as higher information usage for health decision-making (22.8% vs 13.7%, P=.003). When controlling for ethnicity and other sociodemographic variables, perceiving lower amounts of health misinformation on social media was associated with higher odds of watching videos more frequently, making health decisions, and discussing health-related content with a health care provider (P<.001). Furthermore, Latino audiences were 1.85 times more likely to watch videos (P<.001), when controlling for the perceived amount of health misinformation and other sociodemographic variables. Finally, when compared to NLW individuals perceiving little to no health misinformation, Latino audiences perceiving little to no health misinformation were 2.91 times more likely to watch videos (P<.001). Conclusions: The findings suggest that Latino individuals engage with visual health (mis)information at higher rates. Digital health literacy interventions should consider video formats and preferred social media platforms among Latino individuals. Further research is warranted to understand sociocultural factors important to Latino social media users when consuming health information, as these may impact the success of digital media literacy interventions that teach users how to navigate misinformation online.
- Open Access
- 30 October 2024
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 8 (1) , e60939
Ensuring Accuracy and Equity in Vaccination Information From ChatGPT and CDC: Mixed-Methods Cross-Language Evaluation
Background: In the digital age, large language models (LLMs) like ChatGPT have emerged as important sources of health care information. Their interactive capabilities offer promise for enhancing health access, particularly for groups facing traditional barriers such as insurance and language constraints. Despite their growing public health use, with millions of medical queries processed weekly, the quality of LLM-provided information remains inconsistent. Previous studies have predominantly assessed ChatGPT’s English responses, overlooking the needs of non–English speakers in the United States. This study addresses this gap by evaluating the quality and linguistic parity of vaccination information from ChatGPT and the Centers for Disease Control and Prevention (CDC), emphasizing health equity. Objective: This study aims to assess the quality and language equity of vaccination information provided by ChatGPT and the CDC in English and Spanish. It highlights the critical need for cross-language evaluation to ensure equitable health information access for all linguistic groups. Methods: We conducted a comparative analysis of ChatGPT’s and CDC’s responses to frequently asked vaccination-related questions in both languages. The evaluation encompassed quantitative and qualitative assessments of accuracy, readability, and understandability. Accuracy was gauged by the perceived level of misinformation; readability, by the Flesch-Kincaid grade level and readability score; and understandability, by items from the National Institutes of Health’s Patient Education Materials Assessment Tool (PEMAT) instrument. Results: The study found that both ChatGPT and CDC provided mostly accurate and understandable (eg, scores over 95 out of 100) responses. However, Flesch-Kincaid grade levels often exceeded the American Medical Association’s recommended levels, particularly in English (eg, average grade level in English for ChatGPT=12.84, Spanish=7.93, recommended=6). CDC responses outperformed ChatGPT in readability across both languages. Notably, some Spanish responses appeared to be direct translations from English, leading to unnatural phrasing. The findings underscore the potential and challenges of using ChatGPT for health care access. Conclusions: ChatGPT holds potential as a health information resource but requires improvements in readability and linguistic equity to be truly effective for diverse populations. Crucially, the default user experience with ChatGPT, typically encountered by those without advanced language and prompting skills, can significantly shape health perceptions. This is vital from a public health standpoint, as the majority of users will interact with LLMs in their most accessible form. Ensuring that default responses are accurate, understandable, and equitable is imperative for fostering informed health decisions across diverse communities.
- 30 October 2024
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 8 (1) , e60939
Open AccessEnsuring Accuracy and Equity in Vaccination Information From ChatGPT and CDC: Mixed-Methods Cross-Language Evaluation
Background: In the digital age, large language models (LLMs) like ChatGPT have emerged as important sources of health care information. Their interactive capabilities offer promise for enhancing health access, particularly for groups facing traditional barriers such as insurance and language constraints. Despite their growing public health use, with millions of medical queries processed weekly, the quality of LLM-provided information remains inconsistent. Previous studies have predominantly assessed ChatGPT’s English responses, overlooking the needs of non–English speakers in the United States. This study addresses this gap by evaluating the quality and linguistic parity of vaccination information from ChatGPT and the Centers for Disease Control and Prevention (CDC), emphasizing health equity. Objective: This study aims to assess the quality and language equity of vaccination information provided by ChatGPT and the CDC in English and Spanish. It highlights the critical need for cross-language evaluation to ensure equitable health information access for all linguistic groups. Methods: We conducted a comparative analysis of ChatGPT’s and CDC’s responses to frequently asked vaccination-related questions in both languages. The evaluation encompassed quantitative and qualitative assessments of accuracy, readability, and understandability. Accuracy was gauged by the perceived level of misinformation; readability, by the Flesch-Kincaid grade level and readability score; and understandability, by items from the National Institutes of Health’s Patient Education Materials Assessment Tool (PEMAT) instrument. Results: The study found that both ChatGPT and CDC provided mostly accurate and understandable (eg, scores over 95 out of 100) responses. However, Flesch-Kincaid grade levels often exceeded the American Medical Association’s recommended levels, particularly in English (eg, average grade level in English for ChatGPT=12.84, Spanish=7.93, recommended=6). CDC responses outperformed ChatGPT in readability across both languages. Notably, some Spanish responses appeared to be direct translations from English, leading to unnatural phrasing. The findings underscore the potential and challenges of using ChatGPT for health care access. Conclusions: ChatGPT holds potential as a health information resource but requires improvements in readability and linguistic equity to be truly effective for diverse populations. Crucially, the default user experience with ChatGPT, typically encountered by those without advanced language and prompting skills, can significantly shape health perceptions. This is vital from a public health standpoint, as the majority of users will interact with LLMs in their most accessible form. Ensuring that default responses are accurate, understandable, and equitable is imperative for fostering informed health decisions across diverse communities.
- Open Access
- 12 September 2024
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 8 (1) , e56939
Designing a Culturally Relevant Digital Skin Cancer Prevention Intervention for Hispanic Individuals: Qualitative Exploration
Background: In the past 2 decades, melanoma incidence among Hispanic individuals has risen by 20%. The mortality rate of Hispanic individuals is higher than that for non-Hispanic White individuals. Skin cancer can largely be prevented with regular sun protection, and skin cancer outcomes can be improved through early detection, for example, by skin self-examination. Alarmingly, Hispanic individuals are less aware of the symptoms and harms of skin cancers, tend to have misperceptions regarding the risks and benefits of skin cancer prevention behaviors, and engage in less sun protection behaviors than non-Hispanic White individuals. Objective: This study aimed to use a community-engaged approach and conduct both group and individual interviews among Hispanic individuals and relevant key stakeholders to explore the potential design of a mobile-based skin cancer prevention intervention for Hispanic individuals. Methods: This study used a qualitative design (focus groups and individual interviews). Participants were recruited from local community organizations’ social media, local events, and contact lists (eg, email). Zoom interviews were conducted to examine whether Hispanic individuals would be interested in a mobile-based skin cancer intervention and to explore their preferences and suggestions to inform skin cancer prevention intervention design. Results: Five focus groups (2 in Spanish and 3 in English) among self-identified Hispanic individuals (n=34) and 15 semistructured, in-depth individual interviews among key stakeholders (health care providers and community leaders; eg, dermatologist, nurse practitioner, licensed social worker, and church leader) were conducted. The main themes and subthemes emerging from the group discussions and individual interviews were organized into the following categories: intervention platform, delivery frequency and format, message design, engagement plan, and activities. WhatsApp and Facebook were identified as suitable platforms for the intervention. Messages including short videos, visuals (eg, images and photographs), and simple texts messages were preferred. Recommendations for message design included personalized messages, personal stories and narratives, culturally relevant design (eg, incorporating family values), and community-trusted sources. Potential engagement and retention recommendations were also discussed. Additional details and exemplar quotes of each theme and subtheme are described. Conclusions: This study provides important insights and directions for the design of a mobile, digital skin cancer intervention to modify Hispanic individuals’ sun protection and skin self-examination behaviors to help improve skin cancer outcomes. Insights gathered from community leaders and health care providers provided valuable additions to the community-derived data. Leveraging popular digital platforms among Hispanic individuals such as WhatsApp or Facebook could be a promising approach to skin cancer prevention. Recommendations from the community included the use of concise videos, illustrative images, clear text messages, tailored communications, narratives featuring personal experiences, designs that reflect cultural significance, and information from sources that are trusted by the community, which provided useful strategies for future intervention design among Hispanic individuals.
- 12 September 2024
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 8 (1) , e56939
Open AccessDesigning a Culturally Relevant Digital Skin Cancer Prevention Intervention for Hispanic Individuals: Qualitative Exploration
Background: In the past 2 decades, melanoma incidence among Hispanic individuals has risen by 20%. The mortality rate of Hispanic individuals is higher than that for non-Hispanic White individuals. Skin cancer can largely be prevented with regular sun protection, and skin cancer outcomes can be improved through early detection, for example, by skin self-examination. Alarmingly, Hispanic individuals are less aware of the symptoms and harms of skin cancers, tend to have misperceptions regarding the risks and benefits of skin cancer prevention behaviors, and engage in less sun protection behaviors than non-Hispanic White individuals. Objective: This study aimed to use a community-engaged approach and conduct both group and individual interviews among Hispanic individuals and relevant key stakeholders to explore the potential design of a mobile-based skin cancer prevention intervention for Hispanic individuals. Methods: This study used a qualitative design (focus groups and individual interviews). Participants were recruited from local community organizations’ social media, local events, and contact lists (eg, email). Zoom interviews were conducted to examine whether Hispanic individuals would be interested in a mobile-based skin cancer intervention and to explore their preferences and suggestions to inform skin cancer prevention intervention design. Results: Five focus groups (2 in Spanish and 3 in English) among self-identified Hispanic individuals (n=34) and 15 semistructured, in-depth individual interviews among key stakeholders (health care providers and community leaders; eg, dermatologist, nurse practitioner, licensed social worker, and church leader) were conducted. The main themes and subthemes emerging from the group discussions and individual interviews were organized into the following categories: intervention platform, delivery frequency and format, message design, engagement plan, and activities. WhatsApp and Facebook were identified as suitable platforms for the intervention. Messages including short videos, visuals (eg, images and photographs), and simple texts messages were preferred. Recommendations for message design included personalized messages, personal stories and narratives, culturally relevant design (eg, incorporating family values), and community-trusted sources. Potential engagement and retention recommendations were also discussed. Additional details and exemplar quotes of each theme and subtheme are described. Conclusions: This study provides important insights and directions for the design of a mobile, digital skin cancer intervention to modify Hispanic individuals’ sun protection and skin self-examination behaviors to help improve skin cancer outcomes. Insights gathered from community leaders and health care providers provided valuable additions to the community-derived data. Leveraging popular digital platforms among Hispanic individuals such as WhatsApp or Facebook could be a promising approach to skin cancer prevention. Recommendations from the community included the use of concise videos, illustrative images, clear text messages, tailored communications, narratives featuring personal experiences, designs that reflect cultural significance, and information from sources that are trusted by the community, which provided useful strategies for future intervention design among Hispanic individuals.