Stephen D. McLeod
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
Stephen D. McLeod has a total of 81 co-authors. Here are the top co-authors by number of shared publications:
- Thomas M. Lietman
Francis I Proctor Foundation, University of California, San Francisco, USA
10 shared publications - Michael S. Deiner
Francis I Proctor Foundation University of California San Francisco San Francisco, CA United States
9 shared publications - Travis C. Porco
Francis I. Proctor Foundation, University of California San Francisco, San Francisco, California
9 shared publications - Gerami D. Seitzman
Department of Ophthalmology, University of California, San Francisco, CA
4 shared publications - Gurbani Kaur
School of Medicine, University of California San Francisco, San Francisco, CA, USA
4 shared publications - James Chodosh
Department of Ophthalmology and Visual Sciences, University of New Mexico School of Medicine
4 shared publications
Latest Publications
- 26 October 2026
- Published by Wolters Kluwer Health in Journal of Cataract & Refractive Surgery
Ophthalmologists’ Perspectives on Cataract Surgery Sedation: A National Survey Study
DESCRIPTIVE In this national survey assessing US cataract surgery sedation practices, among 659 of 3,406 (19.3%) respondents, 578 (87.7%) routinely performed cataract surgery with anesthesia care. For sedation, 534 (81.0%) routinely used light-to-moderate IV sedation; 69 (10.5%) routinely used oral or no additional sedation. Forty-two (6.4%) performed cataract surgery in office-based surgery (OBS) suites. For future consideration of OBS, 76.0% (n=501), 76.0% (n=501) and 77.2% (n=509) indicated it was “moderately” or “very important” for Medicare to establish physician payment policies for OBS; to maintain access to ambulatory surgery center or hospital outpatient department privileges for complex cases; and for Medicare to continue allowing anesthesia services in the office when needed, respectively. Ophthalmologists nationwide rely heavily on anesthesia care for cataract surgery sedation. With appropriate training and favorable Medicare policies to make OBS financially viable and safe, ophthalmologists may be willing to assume additional, selected sedation roles traditionally performed by anesthesia teams.
- 26 October 2026
- Published by Wolters Kluwer Health in Journal of Cataract & Refractive Surgery
Ophthalmologists’ Perspectives on Cataract Surgery Sedation: A National Survey Study
DESCRIPTIVE In this national survey assessing US cataract surgery sedation practices, among 659 of 3,406 (19.3%) respondents, 578 (87.7%) routinely performed cataract surgery with anesthesia care. For sedation, 534 (81.0%) routinely used light-to-moderate IV sedation; 69 (10.5%) routinely used oral or no additional sedation. Forty-two (6.4%) performed cataract surgery in office-based surgery (OBS) suites. For future consideration of OBS, 76.0% (n=501), 76.0% (n=501) and 77.2% (n=509) indicated it was “moderately” or “very important” for Medicare to establish physician payment policies for OBS; to maintain access to ambulatory surgery center or hospital outpatient department privileges for complex cases; and for Medicare to continue allowing anesthesia services in the office when needed, respectively. Ophthalmologists nationwide rely heavily on anesthesia care for cataract surgery sedation. With appropriate training and favorable Medicare policies to make OBS financially viable and safe, ophthalmologists may be willing to assume additional, selected sedation roles traditionally performed by anesthesia teams.
- Open Access
- 15 October 2026
- Published by JMIR Publications Inc. in JMIR Preprints
Themes, Stances, and Concerns in Social Media Discussion of LASIK: An Infodemiology Study (Preprint)
BACKGROUND Public perceptions of LASIK are shaped in part by social media, where people describe concerns and experiences without the constraints of a clinical questionnaire. Whether the issues discussed in social media correspond to those emphasized in formal preoperative consent and postoperative patient-reported outcome assessment is unknown. OBJECTIVE To characterize LASIK social media discourse and compare post-surgery concerns with those represented in formal informed-consent and patient-reported outcome instruments. METHODS We collected 16,529 unique posts from blogs, forums, Twitter/X, YouTube, and Instagram (April 2023–March 2025). Six large language models independently proposed and classified themes; latent class analysis combined their ratings, and two blinded human raters validated classifications. Stance and time since surgery were assessed in additional analyses. We compared post-surgery themes with the Ophthalmic Mutual Insurance Company (OMIC) LASIK consent form and the Patient-Reported Outcomes with LASIK (PROWL) questionnaire. RESULTS Two-thirds of posts (66.8%) were classified as reflecting direct patient experience followed by secondary reports of others’ experiences (18%). Stance was divided overall (28.8% supportive, 26.6% opposed, 32.5% neutral) and varied by theme. Across all posts, the most frequent themes were regret (26.7%), psychological impact (23.2%), pain (22.2%), and fear (20.3%), with positive outcomes at 15.0%. Among post-surgery posts, pain (40.4%), regret (35.5%), and positive outcomes (35.3%) were most frequent. Recorrection concerns appeared at a median of 11 years after surgery, compared with 1.5 years for pain. OMIC addressed 7 of 17 themes; its item count showed weak association with social-media frequency (ρ=0.35), with no clear association between theme frequency and presentation order (ρ=0.25; n=7). The full 110-item PROWL battery addressed 9 themes, whereas the 33-item PROWL-SS subset addressed 2. Five themes were absent from both instruments, all decisional or economic. CONCLUSIONS Social media provides a complementary view of the concerns people choose to discuss without prompting. Differences from formal consent and outcome assessment were greatest for psychological, decisional, and economic concerns, identifying areas for attention in counseling and patient-reported assessment.
- 15 October 2026
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessThemes, Stances, and Concerns in Social Media Discussion of LASIK: An Infodemiology Study (Preprint)
BACKGROUND Public perceptions of LASIK are shaped in part by social media, where people describe concerns and experiences without the constraints of a clinical questionnaire. Whether the issues discussed in social media correspond to those emphasized in formal preoperative consent and postoperative patient-reported outcome assessment is unknown. OBJECTIVE To characterize LASIK social media discourse and compare post-surgery concerns with those represented in formal informed-consent and patient-reported outcome instruments. METHODS We collected 16,529 unique posts from blogs, forums, Twitter/X, YouTube, and Instagram (April 2023–March 2025). Six large language models independently proposed and classified themes; latent class analysis combined their ratings, and two blinded human raters validated classifications. Stance and time since surgery were assessed in additional analyses. We compared post-surgery themes with the Ophthalmic Mutual Insurance Company (OMIC) LASIK consent form and the Patient-Reported Outcomes with LASIK (PROWL) questionnaire. RESULTS Two-thirds of posts (66.8%) were classified as reflecting direct patient experience followed by secondary reports of others’ experiences (18%). Stance was divided overall (28.8% supportive, 26.6% opposed, 32.5% neutral) and varied by theme. Across all posts, the most frequent themes were regret (26.7%), psychological impact (23.2%), pain (22.2%), and fear (20.3%), with positive outcomes at 15.0%. Among post-surgery posts, pain (40.4%), regret (35.5%), and positive outcomes (35.3%) were most frequent. Recorrection concerns appeared at a median of 11 years after surgery, compared with 1.5 years for pain. OMIC addressed 7 of 17 themes; its item count showed weak association with social-media frequency (ρ=0.35), with no clear association between theme frequency and presentation order (ρ=0.25; n=7). The full 110-item PROWL battery addressed 9 themes, whereas the 33-item PROWL-SS subset addressed 2. Five themes were absent from both instruments, all decisional or economic. CONCLUSIONS Social media provides a complementary view of the concerns people choose to discuss without prompting. Differences from formal consent and outcome assessment were greatest for psychological, decisional, and economic concerns, identifying areas for attention in counseling and patient-reported assessment.
- Open Access
- 14 May 2026
- Published by Elsevier in American Journal of Ophthalmology
- Vol. 289, 20-30
Visual Field Progression in Glaucoma Patients With Delayed Follow-Up
- 14 May 2026
- Published by Elsevier in American Journal of Ophthalmology
- Vol. 289, 20-30
Open AccessVisual Field Progression in Glaucoma Patients With Delayed Follow-Up
- Open Access
- 14 August 2025
- Published by Association of University Professors of Ophthalmology in Journal of Academic Ophthalmology
- Vol. 17 (2) , 17
Ophthalmology's Role in the Academic Medical Center: A Perspective
- 14 August 2025
- Published by Association of University Professors of Ophthalmology in Journal of Academic Ophthalmology
- Vol. 17 (2) , 17
Open AccessOphthalmology's Role in the Academic Medical Center: A Perspective
- Open Access
- 2 July 2025
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 27, e65226
Use of Large Language Models to Classify Epidemiological Characteristics in Synthetic and Real-World Social Media Posts About Conjunctivitis Outbreaks: Infodemiology Study
Background: The use of web-based search and social media can help identify epidemics, potentially earlier than clinical methods or even potentially identifying unreported outbreaks. Monitoring for eye-related epidemics, such as conjunctivitis outbreaks, can facilitate early public health intervention to reduce transmission and ocular comorbidities. However, monitoring social media content for conjunctivitis outbreaks is costly and laborious. Large language models (LLMs) could overcome these barriers by assessing the likelihood that real-world outbreaks are being described. However, public health actions for likely outbreaks could benefit more by knowing additional epidemiological characteristics, such as outbreak type, size, and severity. Objective: We aimed to assess whether and how well LLMs can classify epidemiological features from social media posts beyond conjunctivitis outbreak probability, including outbreak type, size, severity, etiology, and community setting. We used a validation framework comparing LLM classifications to those of other LLMs and human experts. Methods: We wrote code to generate synthetic conjunctivitis outbreak social media posts, embedded with specific preclassified epidemiological features to simulate various infectious eye disease outbreak and control scenarios. We used these posts to develop effective LLM prompts and test the capabilities of multiple LLMs. For top-performing LLMs, we gauged their practical utility in real-world epidemiological surveillance by comparing their assessments of Twitter/X, forum, and YouTube conjunctivitis posts. Finally, human raters also classified the posts, and we compared their classifications to those of a leading LLM for validation. Comparisons entailed correlation or sensitivity and specificity statistics. Results: We assessed 7 LLMs for effectively classifying epidemiological data from 1152 synthetic posts, 370 Twitter/X posts, 290 forum posts, and 956 YouTube posts. Despite some discrepancies, the LLMs demonstrated a reliable capacity for nuanced epidemiological analysis across various data sources and compared to humans or between LLMs. Notably, GPT-4 and Mixtral 8x22b exhibited high performance, predicting conjunctivitis outbreak characteristics such as probability (GPT-4: correlation=0.73), size (Mixtral 8x22b: correlation=0.82), and type (infectious, allergic, or environmentally caused); however, there were notable exceptions. Assessing synthetic and real-world posts for etiological factors, infectious eye disease specialist validations revealed that GPT-4 had high specificity (0.83-1.00) but variable sensitivity (0.32-0.71). Interrater reliability analyses showed that LLM-expert agreement exceeded expert-expert agreement for severity assessment (intraclass correlation coefficient=0.69 vs 0.38), while agreement varied by condition type (κ=0.37-0.94). Conclusions: This investigation into the potential of LLMs for public health infoveillance suggests effectiveness in classifying key epidemiological characteristics from social media content about conjunctivitis outbreaks. Future studies should further explore LLMs’ potential to support public health monitoring through the automated assessment and classification of potential infectious eye disease or other outbreaks. Their optimal role may be to act as a first line of documentation, alerting public health organizations for the follow-up of LLM-detected and -classified small, early outbreaks, with a focus on the most severe ones.
- 2 July 2025
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 27, e65226
Open AccessUse of Large Language Models to Classify Epidemiological Characteristics in Synthetic and Real-World Social Media Posts About Conjunctivitis Outbreaks: Infodemiology Study
Background: The use of web-based search and social media can help identify epidemics, potentially earlier than clinical methods or even potentially identifying unreported outbreaks. Monitoring for eye-related epidemics, such as conjunctivitis outbreaks, can facilitate early public health intervention to reduce transmission and ocular comorbidities. However, monitoring social media content for conjunctivitis outbreaks is costly and laborious. Large language models (LLMs) could overcome these barriers by assessing the likelihood that real-world outbreaks are being described. However, public health actions for likely outbreaks could benefit more by knowing additional epidemiological characteristics, such as outbreak type, size, and severity. Objective: We aimed to assess whether and how well LLMs can classify epidemiological features from social media posts beyond conjunctivitis outbreak probability, including outbreak type, size, severity, etiology, and community setting. We used a validation framework comparing LLM classifications to those of other LLMs and human experts. Methods: We wrote code to generate synthetic conjunctivitis outbreak social media posts, embedded with specific preclassified epidemiological features to simulate various infectious eye disease outbreak and control scenarios. We used these posts to develop effective LLM prompts and test the capabilities of multiple LLMs. For top-performing LLMs, we gauged their practical utility in real-world epidemiological surveillance by comparing their assessments of Twitter/X, forum, and YouTube conjunctivitis posts. Finally, human raters also classified the posts, and we compared their classifications to those of a leading LLM for validation. Comparisons entailed correlation or sensitivity and specificity statistics. Results: We assessed 7 LLMs for effectively classifying epidemiological data from 1152 synthetic posts, 370 Twitter/X posts, 290 forum posts, and 956 YouTube posts. Despite some discrepancies, the LLMs demonstrated a reliable capacity for nuanced epidemiological analysis across various data sources and compared to humans or between LLMs. Notably, GPT-4 and Mixtral 8x22b exhibited high performance, predicting conjunctivitis outbreak characteristics such as probability (GPT-4: correlation=0.73), size (Mixtral 8x22b: correlation=0.82), and type (infectious, allergic, or environmentally caused); however, there were notable exceptions. Assessing synthetic and real-world posts for etiological factors, infectious eye disease specialist validations revealed that GPT-4 had high specificity (0.83-1.00) but variable sensitivity (0.32-0.71). Interrater reliability analyses showed that LLM-expert agreement exceeded expert-expert agreement for severity assessment (intraclass correlation coefficient=0.69 vs 0.38), while agreement varied by condition type (κ=0.37-0.94). Conclusions: This investigation into the potential of LLMs for public health infoveillance suggests effectiveness in classifying key epidemiological characteristics from social media content about conjunctivitis outbreaks. Future studies should further explore LLMs’ potential to support public health monitoring through the automated assessment and classification of potential infectious eye disease or other outbreaks. Their optimal role may be to act as a first line of documentation, alerting public health organizations for the follow-up of LLM-detected and -classified small, early outbreaks, with a focus on the most severe ones.
- 22 October 2024
- Published by Springer Nature in Nature
- Vol. 634 (8035) , 782
US plan to close the National Eye Institute shows faulty vision
- 22 October 2024
- Published by Springer Nature in Nature
- Vol. 634 (8035) , 782
US plan to close the National Eye Institute shows faulty vision
- Open Access
- 9 August 2024
- Published by JMIR Publications Inc. in JMIR Preprints
Use of Large Language Models to Classify Epidemiological Characteristics in Synthetic and Real-World Social Media Posts About Conjunctivitis Outbreaks: Infodemiology Study (Preprint)
BACKGROUND The use of web-based search and social media can help identify epidemics, potentially earlier than clinical methods or even potentially identifying unreported outbreaks. Monitoring for eye-related epidemics, such as conjunctivitis outbreaks, can facilitate early public health intervention to reduce transmission and ocular comorbidities. However, monitoring social media content for conjunctivitis outbreaks is costly and laborious. Large language models (LLMs) could overcome these barriers by assessing the likelihood that real-world outbreaks are being described. However, public health actions for likely outbreaks could benefit more by knowing additional epidemiological characteristics, such as outbreak type, size, and severity. OBJECTIVE We aimed to assess whether and how well LLMs can classify epidemiological features from social media posts beyond conjunctivitis outbreak probability, including outbreak type, size, severity, etiology, and community setting. We used a validation framework comparing LLM classifications to those of other LLMs and human experts. METHODS We wrote code to generate synthetic conjunctivitis outbreak social media posts, embedded with specific preclassified epidemiological features to simulate various infectious eye disease outbreak and control scenarios. We used these posts to develop effective LLM prompts and test the capabilities of multiple LLMs. For top-performing LLMs, we gauged their practical utility in real-world epidemiological surveillance by comparing their assessments of Twitter/X, forum, and YouTube conjunctivitis posts. Finally, human raters also classified the posts, and we compared their classifications to those of a leading LLM for validation. Comparisons entailed correlation or sensitivity and specificity statistics. RESULTS We assessed 7 LLMs for effectively classifying epidemiological data from 1152 synthetic posts, 370 Twitter/X posts, 290 forum posts, and 956 YouTube posts. Despite some discrepancies, the LLMs demonstrated a reliable capacity for nuanced epidemiological analysis across various data sources and compared to humans or between LLMs. Notably, GPT-4 and Mixtral 8x22b exhibited high performance, predicting conjunctivitis outbreak characteristics such as probability (GPT-4: correlation=0.73), size (Mixtral 8x22b: correlation=0.82), and type (infectious, allergic, or environmentally caused); however, there were notable exceptions. Assessing synthetic and real-world posts for etiological factors, infectious eye disease specialist validations revealed that GPT-4 had high specificity (0.83-1.00) but variable sensitivity (0.32-0.71). Interrater reliability analyses showed that LLM-expert agreement exceeded expert-expert agreement for severity assessment (intraclass correlation coefficient=0.69 vs 0.38), while agreement varied by condition type (κ=0.37-0.94). CONCLUSIONS This investigation into the potential of LLMs for public health infoveillance suggests effectiveness in classifying key epidemiological characteristics from social media content about conjunctivitis outbreaks. Future studies should further explore LLMs’ potential to support public health monitoring through the automated assessment and classification of potential infectious eye disease or other outbreaks. Their optimal role may be to act as a first line of documentation, alerting public health organizations for the follow-up of LLM-detected and -classified small, early outbreaks, with a focus on the most severe ones.
- 9 August 2024
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessUse of Large Language Models to Classify Epidemiological Characteristics in Synthetic and Real-World Social Media Posts About Conjunctivitis Outbreaks: Infodemiology Study (Preprint)
BACKGROUND The use of web-based search and social media can help identify epidemics, potentially earlier than clinical methods or even potentially identifying unreported outbreaks. Monitoring for eye-related epidemics, such as conjunctivitis outbreaks, can facilitate early public health intervention to reduce transmission and ocular comorbidities. However, monitoring social media content for conjunctivitis outbreaks is costly and laborious. Large language models (LLMs) could overcome these barriers by assessing the likelihood that real-world outbreaks are being described. However, public health actions for likely outbreaks could benefit more by knowing additional epidemiological characteristics, such as outbreak type, size, and severity. OBJECTIVE We aimed to assess whether and how well LLMs can classify epidemiological features from social media posts beyond conjunctivitis outbreak probability, including outbreak type, size, severity, etiology, and community setting. We used a validation framework comparing LLM classifications to those of other LLMs and human experts. METHODS We wrote code to generate synthetic conjunctivitis outbreak social media posts, embedded with specific preclassified epidemiological features to simulate various infectious eye disease outbreak and control scenarios. We used these posts to develop effective LLM prompts and test the capabilities of multiple LLMs. For top-performing LLMs, we gauged their practical utility in real-world epidemiological surveillance by comparing their assessments of Twitter/X, forum, and YouTube conjunctivitis posts. Finally, human raters also classified the posts, and we compared their classifications to those of a leading LLM for validation. Comparisons entailed correlation or sensitivity and specificity statistics. RESULTS We assessed 7 LLMs for effectively classifying epidemiological data from 1152 synthetic posts, 370 Twitter/X posts, 290 forum posts, and 956 YouTube posts. Despite some discrepancies, the LLMs demonstrated a reliable capacity for nuanced epidemiological analysis across various data sources and compared to humans or between LLMs. Notably, GPT-4 and Mixtral 8x22b exhibited high performance, predicting conjunctivitis outbreak characteristics such as probability (GPT-4: correlation=0.73), size (Mixtral 8x22b: correlation=0.82), and type (infectious, allergic, or environmentally caused); however, there were notable exceptions. Assessing synthetic and real-world posts for etiological factors, infectious eye disease specialist validations revealed that GPT-4 had high specificity (0.83-1.00) but variable sensitivity (0.32-0.71). Interrater reliability analyses showed that LLM-expert agreement exceeded expert-expert agreement for severity assessment (intraclass correlation coefficient=0.69 vs 0.38), while agreement varied by condition type (κ=0.37-0.94). CONCLUSIONS This investigation into the potential of LLMs for public health infoveillance suggests effectiveness in classifying key epidemiological characteristics from social media content about conjunctivitis outbreaks. Future studies should further explore LLMs’ potential to support public health monitoring through the automated assessment and classification of potential infectious eye disease or other outbreaks. Their optimal role may be to act as a first line of documentation, alerting public health organizations for the follow-up of LLM-detected and -classified small, early outbreaks, with a focus on the most severe ones.
- Open Access
- 1 March 2024
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 26, e49139
Use of Large Language Models to Assess the Likelihood of Epidemics From the Content of Tweets: Infodemiology Study
Background: Previous work suggests that Google searches could be useful in identifying conjunctivitis epidemics. Content-based assessment of social media content may provide additional value in serving as early indicators of conjunctivitis and other systemic infectious diseases. Objective: We investigated whether large language models, specifically GPT-3.5 and GPT-4 (OpenAI), can provide probabilistic assessments of whether social media posts about conjunctivitis could indicate a regional outbreak. Methods: A total of 12,194 conjunctivitis-related tweets were obtained using a targeted Boolean search in multiple languages from India, Guam (United States), Martinique (France), the Philippines, American Samoa (United States), Fiji, Costa Rica, Haiti, and the Bahamas, covering the time frame from January 1, 2012, to March 13, 2023. By providing these tweets via prompts to GPT-3.5 and GPT-4, we obtained probabilistic assessments that were validated by 2 human raters. We then calculated Pearson correlations of these time series with tweet volume and the occurrence of known outbreaks in these 9 locations, with time series bootstrap used to compute CIs. Results: Probabilistic assessments derived from GPT-3.5 showed correlations of 0.60 (95% CI 0.47-0.70) and 0.53 (95% CI 0.40-0.65) with the 2 human raters, with higher results for GPT-4. The weekly averages of GPT-3.5 probabilities showed substantial correlations with weekly tweet volume for 44% (4/9) of the countries, with correlations ranging from 0.10 (95% CI 0.0-0.29) to 0.53 (95% CI 0.39-0.89), with larger correlations for GPT-4. More modest correlations were found for correlation with known epidemics, with substantial correlation only in American Samoa (0.40, 95% CI 0.16-0.81). Conclusions: These findings suggest that GPT prompting can efficiently assess the content of social media posts and indicate possible disease outbreaks to a degree of accuracy comparable to that of humans. Furthermore, we found that automated content analysis of tweets is related to tweet volume for conjunctivitis-related posts in some locations and to the occurrence of actual epidemics. Future work may improve the sensitivity and specificity of these methods for disease outbreak detection.
- 1 March 2024
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 26, e49139
Open AccessUse of Large Language Models to Assess the Likelihood of Epidemics From the Content of Tweets: Infodemiology Study
Background: Previous work suggests that Google searches could be useful in identifying conjunctivitis epidemics. Content-based assessment of social media content may provide additional value in serving as early indicators of conjunctivitis and other systemic infectious diseases. Objective: We investigated whether large language models, specifically GPT-3.5 and GPT-4 (OpenAI), can provide probabilistic assessments of whether social media posts about conjunctivitis could indicate a regional outbreak. Methods: A total of 12,194 conjunctivitis-related tweets were obtained using a targeted Boolean search in multiple languages from India, Guam (United States), Martinique (France), the Philippines, American Samoa (United States), Fiji, Costa Rica, Haiti, and the Bahamas, covering the time frame from January 1, 2012, to March 13, 2023. By providing these tweets via prompts to GPT-3.5 and GPT-4, we obtained probabilistic assessments that were validated by 2 human raters. We then calculated Pearson correlations of these time series with tweet volume and the occurrence of known outbreaks in these 9 locations, with time series bootstrap used to compute CIs. Results: Probabilistic assessments derived from GPT-3.5 showed correlations of 0.60 (95% CI 0.47-0.70) and 0.53 (95% CI 0.40-0.65) with the 2 human raters, with higher results for GPT-4. The weekly averages of GPT-3.5 probabilities showed substantial correlations with weekly tweet volume for 44% (4/9) of the countries, with correlations ranging from 0.10 (95% CI 0.0-0.29) to 0.53 (95% CI 0.39-0.89), with larger correlations for GPT-4. More modest correlations were found for correlation with known epidemics, with substantial correlation only in American Samoa (0.40, 95% CI 0.16-0.81). Conclusions: These findings suggest that GPT prompting can efficiently assess the content of social media posts and indicate possible disease outbreaks to a degree of accuracy comparable to that of humans. Furthermore, we found that automated content analysis of tweets is related to tweet volume for conjunctivitis-related posts in some locations and to the occurrence of actual epidemics. Future work may improve the sensitivity and specificity of these methods for disease outbreak detection.
- Open Access
- 19 May 2023
- Published by JMIR Publications Inc. in JMIR Preprints
Use of Large Language Models to Assess the Likelihood of Epidemics From the Content of Tweets: Infodemiology Study (Preprint)
BACKGROUND Previous work suggests that Google searches could be useful in identifying conjunctivitis epidemics. Content-based assessment of social media content may provide additional value in serving as early indicators of conjunctivitis and other systemic infectious diseases. OBJECTIVE We investigated whether large language models, specifically GPT-3.5 and GPT-4 (OpenAI), can provide probabilistic assessments of whether social media posts about conjunctivitis could indicate a regional outbreak. METHODS A total of 12,194 conjunctivitis-related tweets were obtained using a targeted Boolean search in multiple languages from India, Guam (United States), Martinique (France), the Philippines, American Samoa (United States), Fiji, Costa Rica, Haiti, and the Bahamas, covering the time frame from January 1, 2012, to March 13, 2023. By providing these tweets via prompts to GPT-3.5 and GPT-4, we obtained probabilistic assessments that were validated by 2 human raters. We then calculated Pearson correlations of these time series with tweet volume and the occurrence of known outbreaks in these 9 locations, with time series bootstrap used to compute CIs. RESULTS Probabilistic assessments derived from GPT-3.5 showed correlations of 0.60 (95% CI 0.47-0.70) and 0.53 (95% CI 0.40-0.65) with the 2 human raters, with higher results for GPT-4. The weekly averages of GPT-3.5 probabilities showed substantial correlations with weekly tweet volume for 44% (4/9) of the countries, with correlations ranging from 0.10 (95% CI 0.0-0.29) to 0.53 (95% CI 0.39-0.89), with larger correlations for GPT-4. More modest correlations were found for correlation with known epidemics, with substantial correlation only in American Samoa (0.40, 95% CI 0.16-0.81). CONCLUSIONS These findings suggest that GPT prompting can efficiently assess the content of social media posts and indicate possible disease outbreaks to a degree of accuracy comparable to that of humans. Furthermore, we found that automated content analysis of tweets is related to tweet volume for conjunctivitis-related posts in some locations and to the occurrence of actual epidemics. Future work may improve the sensitivity and specificity of these methods for disease outbreak detection.
- 19 May 2023
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessUse of Large Language Models to Assess the Likelihood of Epidemics From the Content of Tweets: Infodemiology Study (Preprint)
BACKGROUND Previous work suggests that Google searches could be useful in identifying conjunctivitis epidemics. Content-based assessment of social media content may provide additional value in serving as early indicators of conjunctivitis and other systemic infectious diseases. OBJECTIVE We investigated whether large language models, specifically GPT-3.5 and GPT-4 (OpenAI), can provide probabilistic assessments of whether social media posts about conjunctivitis could indicate a regional outbreak. METHODS A total of 12,194 conjunctivitis-related tweets were obtained using a targeted Boolean search in multiple languages from India, Guam (United States), Martinique (France), the Philippines, American Samoa (United States), Fiji, Costa Rica, Haiti, and the Bahamas, covering the time frame from January 1, 2012, to March 13, 2023. By providing these tweets via prompts to GPT-3.5 and GPT-4, we obtained probabilistic assessments that were validated by 2 human raters. We then calculated Pearson correlations of these time series with tweet volume and the occurrence of known outbreaks in these 9 locations, with time series bootstrap used to compute CIs. RESULTS Probabilistic assessments derived from GPT-3.5 showed correlations of 0.60 (95% CI 0.47-0.70) and 0.53 (95% CI 0.40-0.65) with the 2 human raters, with higher results for GPT-4. The weekly averages of GPT-3.5 probabilities showed substantial correlations with weekly tweet volume for 44% (4/9) of the countries, with correlations ranging from 0.10 (95% CI 0.0-0.29) to 0.53 (95% CI 0.39-0.89), with larger correlations for GPT-4. More modest correlations were found for correlation with known epidemics, with substantial correlation only in American Samoa (0.40, 95% CI 0.16-0.81). CONCLUSIONS These findings suggest that GPT prompting can efficiently assess the content of social media posts and indicate possible disease outbreaks to a degree of accuracy comparable to that of humans. Furthermore, we found that automated content analysis of tweets is related to tweet volume for conjunctivitis-related posts in some locations and to the occurrence of actual epidemics. Future work may improve the sensitivity and specificity of these methods for disease outbreak detection.
- Open Access
- 5 July 2022
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 24 (7) , e27310
A Google Trends Approach to Identify Distinct Diurnal and Day-of-Week Web-Based Search Patterns Related to Conjunctivitis and Other Common Eye Conditions: Infodemiology Study
Background: Studies suggest diurnal patterns of occurrence of some eye conditions. Leveraging new information sources such as web-based search data to learn more about such patterns could improve the understanding of patients’ eye-related conditions and well-being, better inform timing of clinical and remote eye care, and improve precision when targeting web-based public health campaigns toward underserved populations. Objective: To investigate our hypothesis that the public is likely to consistently search about different ophthalmologic conditions at different hours of the day or days of week, we conducted an observational study using search data for terms related to ophthalmologic conditions such as conjunctivitis. We assessed whether search volumes reflected diurnal or day-of-week patterns and if those patterns were distinct from each other. Methods: We designed a study to analyze and compare hourly search data for eye-related and control search terms, using time series regression models with trend and periodicity terms to remove outliers and then estimate diurnal effects. We planned a Google Trends setting, extracting data from 10 US states for the entire year of 2018. The exposure was internet search, and the participants were populations who searched through Google’s search engine using our chosen study terms. Our main outcome measures included cyclical hourly and day-of-week web-based search patterns. For statistical analyses, we considered P<.001 to be statistically significant. Results: Distinct diurnal (P<.001 for all search terms) and day-of-week search patterns for eye-related terms were observed but with differing peak time periods and cyclic strengths. Some diurnal patterns represented those reported from prior clinical studies. Of the eye-related terms, “pink eye” showed the largest diurnal amplitude-to-mean ratios. Stronger signal was restricted to and peaked in mornings, and amplitude was higher on weekdays. By contrast, “dry eyes” had a higher amplitude diurnal pattern on weekends, with stronger signal occurring over a broader evening-to-morning period and peaking in early morning. Conclusions: The frequency of web-based searches for various eye conditions can show cyclic patterns according to time of the day or week. Further studies to understand the reasons for these variations may help supplement the current clinical understanding of ophthalmologic symptom presentation and improve the timeliness of patient messaging and care interventions.
- 5 July 2022
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 24 (7) , e27310
Open AccessA Google Trends Approach to Identify Distinct Diurnal and Day-of-Week Web-Based Search Patterns Related to Conjunctivitis and Other Common Eye Conditions: Infodemiology Study
Background: Studies suggest diurnal patterns of occurrence of some eye conditions. Leveraging new information sources such as web-based search data to learn more about such patterns could improve the understanding of patients’ eye-related conditions and well-being, better inform timing of clinical and remote eye care, and improve precision when targeting web-based public health campaigns toward underserved populations. Objective: To investigate our hypothesis that the public is likely to consistently search about different ophthalmologic conditions at different hours of the day or days of week, we conducted an observational study using search data for terms related to ophthalmologic conditions such as conjunctivitis. We assessed whether search volumes reflected diurnal or day-of-week patterns and if those patterns were distinct from each other. Methods: We designed a study to analyze and compare hourly search data for eye-related and control search terms, using time series regression models with trend and periodicity terms to remove outliers and then estimate diurnal effects. We planned a Google Trends setting, extracting data from 10 US states for the entire year of 2018. The exposure was internet search, and the participants were populations who searched through Google’s search engine using our chosen study terms. Our main outcome measures included cyclical hourly and day-of-week web-based search patterns. For statistical analyses, we considered P<.001 to be statistically significant. Results: Distinct diurnal (P<.001 for all search terms) and day-of-week search patterns for eye-related terms were observed but with differing peak time periods and cyclic strengths. Some diurnal patterns represented those reported from prior clinical studies. Of the eye-related terms, “pink eye” showed the largest diurnal amplitude-to-mean ratios. Stronger signal was restricted to and peaked in mornings, and amplitude was higher on weekdays. By contrast, “dry eyes” had a higher amplitude diurnal pattern on weekends, with stronger signal occurring over a broader evening-to-morning period and peaking in early morning. Conclusions: The frequency of web-based searches for various eye conditions can show cyclic patterns according to time of the day or week. Further studies to understand the reasons for these variations may help supplement the current clinical understanding of ophthalmologic symptom presentation and improve the timeliness of patient messaging and care interventions.