Marco Montagna
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Marco Montagna has a total of 59 co-authors. Here are the top co-authors by number of shared publications:
- Patrizia Rovere-Querini
Vita-Salute San Raffaele University, Milan, Italy; Division of Immunology, Transplantation and Infectious Diseases, IRCCS San Raffaele Scientific Institute, Milan, Italy
4 shared publications - Cecilia Mascolo
Department of Computer Science and Technology, University of Cambridge, Cambridge, United Kingdom
3 shared publications - Filippo Chiabrando
School of Medicine, Vita-Salute San Raffaele University, Via Olgettina 58, Milan, 20132, Italy
3 shared publications Medical Students
2 shared publicationsMikael Kågebäck
2 shared publicationsPatrizia Rovere Querini
Department of Medicine, IRCCS San Raffaele Scientific Institute
2 shared publications
Latest Publications
- Open Access
- 20 April 2026
- Published by Elsevier in SSRN Electronic Journal
Short-Term PM2.5 Exposure Is Associated with Increased Nocturnal Cough: A Multi-City Mobile Sensing Study
Background: Air pollution remains a major environmental determinant of health, yet the short-term effects of fine particulate matter (PM2.5) on early respiratory symptoms are poorly characterised because population-scale symptom data are rarely available. We aimed to quantify the association between short-term PM2.5 exposure and nocturnal cough frequency using large-scale mobile sensing data across multiple cities.Methods: We analysed nocturnal audio recordings collected from a widely used mobile sleep application (Sleep Cycle), applying an AI-based cough detection model to estimate nightly cough frequency. Data were aggregated at the city-day level across 32 cities in 12 countries over 500 consecutive days. We used city-specific generalised additive models, multi-city panel regressions adjusting for meteorological conditions and influenza activity, and an event-based analysis during a wildfire episode to assess the association between daily PM2.5 concentrations and nocturnal cough frequency.Findings: Higher daily PM2.5 concentrations were consistently associated with increased nocturnal cough frequency across analytical approaches (relative risk = 1.012, 95% CI 1.006–1.018 per 10 μg/m3). The exposure–response relationship was supralinear, with detectable increases in cough frequency observed even at relatively low PM2.5 levels. The association was strongest at short lags and remained robust in sensitivity analyses.Interpretation: Short-term exposure to PM2.5 is associated with increased nocturnal cough frequency at the population level, suggesting that respiratory symptoms may respond to air pollution at lower exposure levels than previously recognized. Large-scale mobile sensing of respiratory symptoms offers a novel digital biomarker for real-time environmental health surveillance and could complement traditional health monitoring systems.
- 20 April 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessShort-Term PM2.5 Exposure Is Associated with Increased Nocturnal Cough: A Multi-City Mobile Sensing Study
Background: Air pollution remains a major environmental determinant of health, yet the short-term effects of fine particulate matter (PM2.5) on early respiratory symptoms are poorly characterised because population-scale symptom data are rarely available. We aimed to quantify the association between short-term PM2.5 exposure and nocturnal cough frequency using large-scale mobile sensing data across multiple cities.Methods: We analysed nocturnal audio recordings collected from a widely used mobile sleep application (Sleep Cycle), applying an AI-based cough detection model to estimate nightly cough frequency. Data were aggregated at the city-day level across 32 cities in 12 countries over 500 consecutive days. We used city-specific generalised additive models, multi-city panel regressions adjusting for meteorological conditions and influenza activity, and an event-based analysis during a wildfire episode to assess the association between daily PM2.5 concentrations and nocturnal cough frequency.Findings: Higher daily PM2.5 concentrations were consistently associated with increased nocturnal cough frequency across analytical approaches (relative risk = 1.012, 95% CI 1.006–1.018 per 10 μg/m3). The exposure–response relationship was supralinear, with detectable increases in cough frequency observed even at relatively low PM2.5 levels. The association was strongest at short lags and remained robust in sensitivity analyses.Interpretation: Short-term exposure to PM2.5 is associated with increased nocturnal cough frequency at the population level, suggesting that respiratory symptoms may respond to air pollution at lower exposure levels than previously recognized. Large-scale mobile sensing of respiratory symptoms offers a novel digital biomarker for real-time environmental health surveillance and could complement traditional health monitoring systems.
- Open Access
- 25 February 2026
- Published by Frontiers Media SA in Frontiers in Digital Health
- Vol. 8, 1685842
From raw data to actionable insights: preprocessing real-world data for machine learning in diabetes care
Background: Machine Learning (ML) applied to healthcare Real World Data (RWD) may improve patient management. RWD, however, requires extensive preprocessing to make it ML-ready. Our aim was to explore the impact of preprocessing on ML models applied to RWD from 20 years of type 2 diabetes patients visits. Methods: Our cohort consisted of patients with at least two glycated hemoglobin (HbA1c) measurements three years apart. We set up three different experimental settings consisting of different data preprocessing pipelines. Logistic Regression (LR), XGBoost and a Decision Tree Classifier (DTC) were then applied and tuned to optimize precision. Results: The final dataset comprised 12 variables from 1,651 patients treated between 2003 and 2023. 921 (56%) patients had a HbA1c decrease at three years. This group had a higher baseline HbA1c, higher BMI and shorter first visit gap from the date of diagnosis ( p < 0.0001). Precision scores for LR, XGBoost did not vary across different experimental conditions while DTC benefitted from missing data imputation. Shapley Additive Explanations confirmed the Exploratory Data Analysis findings, with worse baseline values being predictors of HbA1c decrease at three years. Conclusions: ML models' performance and their explanation did not vary substantially across experimental conditions, with worse baseline values being predictors of HbA1c decrease at three years. Insights such as this, extracted by ML application to RWD, enable clinical discussion and may foster improvements in patient management.
- 25 February 2026
- Published by Frontiers Media SA in Frontiers in Digital Health
- Vol. 8, 1685842
Open AccessFrom raw data to actionable insights: preprocessing real-world data for machine learning in diabetes care
Background: Machine Learning (ML) applied to healthcare Real World Data (RWD) may improve patient management. RWD, however, requires extensive preprocessing to make it ML-ready. Our aim was to explore the impact of preprocessing on ML models applied to RWD from 20 years of type 2 diabetes patients visits. Methods: Our cohort consisted of patients with at least two glycated hemoglobin (HbA1c) measurements three years apart. We set up three different experimental settings consisting of different data preprocessing pipelines. Logistic Regression (LR), XGBoost and a Decision Tree Classifier (DTC) were then applied and tuned to optimize precision. Results: The final dataset comprised 12 variables from 1,651 patients treated between 2003 and 2023. 921 (56%) patients had a HbA1c decrease at three years. This group had a higher baseline HbA1c, higher BMI and shorter first visit gap from the date of diagnosis ( p < 0.0001). Precision scores for LR, XGBoost did not vary across different experimental conditions while DTC benefitted from missing data imputation. Shapley Additive Explanations confirmed the Exploratory Data Analysis findings, with worse baseline values being predictors of HbA1c decrease at three years. Conclusions: ML models' performance and their explanation did not vary substantially across experimental conditions, with worse baseline values being predictors of HbA1c decrease at three years. Insights such as this, extracted by ML application to RWD, enable clinical discussion and may foster improvements in patient management.
- Open Access
- 19 December 2025
- Published by Springer Nature in Internal and Emergency Medicine
- Vol. 21 (1) , 1-14
Artificial intelligence in medicine: a position paper by the Italian Society of Internal Medicine
Artificial Intelligence (AI) represents an innovative technological support for clinical practice. The Italian Society of Internal Medicine (SIMI) emphasizes the need for clear guidance on the use of AI in medicine, recognizing that knowledge in this field is continuously evolving. This position paper presents a comprehensive vision for the responsible integration of AI into clinical practice. AI should serve as a support tool—not a replacement—for clinicians. It has the potential to improve diagnostic accuracy, reduce administrative workload, and strengthen the physician–patient relationship. In the light of these characteristics, SIMI advocates for transparency, data privacy, equity, and sustainability in the development and implementation of AI systems. SIMI also highlights several ethical, legal, and methodological challenges that must be addressed, including algorithmic bias, environmental impact, and disparities in access. Ultimately, SIMI envisions a future in which AI augments human expertise, enabling more efficient, personalized, and compassionate care. SIMI calls for active clinician participation in the co-design and validation of AI tools to ensure alignment with real-world clinical needs. Key recommendations include the preferential use of certified AI systems, the integration of AI education into medical training, and continuous monitoring after deployment.
- 19 December 2025
- Published by Springer Nature in Internal and Emergency Medicine
- Vol. 21 (1) , 1-14
Open AccessArtificial intelligence in medicine: a position paper by the Italian Society of Internal Medicine
Artificial Intelligence (AI) represents an innovative technological support for clinical practice. The Italian Society of Internal Medicine (SIMI) emphasizes the need for clear guidance on the use of AI in medicine, recognizing that knowledge in this field is continuously evolving. This position paper presents a comprehensive vision for the responsible integration of AI into clinical practice. AI should serve as a support tool—not a replacement—for clinicians. It has the potential to improve diagnostic accuracy, reduce administrative workload, and strengthen the physician–patient relationship. In the light of these characteristics, SIMI advocates for transparency, data privacy, equity, and sustainability in the development and implementation of AI systems. SIMI also highlights several ethical, legal, and methodological challenges that must be addressed, including algorithmic bias, environmental impact, and disparities in access. Ultimately, SIMI envisions a future in which AI augments human expertise, enabling more efficient, personalized, and compassionate care. SIMI calls for active clinician participation in the co-design and validation of AI tools to ensure alignment with real-world clinical needs. Key recommendations include the preferential use of certified AI systems, the integration of AI education into medical training, and continuous monitoring after deployment.
- Open Access
- 2 December 2025
- Published by Springer Nature in Research Square
Uncovering Respiratory Effects of Short-term PM2.5 Exposure through Global Nocturnal Cough Monitoring
Air pollution remains one of the leading environmental determinants of health, yet the short-term effects of fine particulate matter (PM₂.₅) on early respiratory symptoms are poorly characterized owing to the limited availability of high-resolution, population-scale data. Enabled by global mobile sensing through a widely used application (Sleep Cycle) and AI algorithms that automatically detect coughs from nocturnal audio recordings, we derived nighttime cough frequency across 34 cities in 12 countries on five continents over 500 consecutive days, conducting the first multi-city assessment linking daily PM₂.₅ exposure to nocturnal respiratory symptoms. Single-city generalized additive models, multi-city panel regressions adjusting for meteorological variables and influenza rates, and quasi-experimental analyses consistently showed that same-day PM₂.₅ exposure was associated with higher cough frequency (relative risk = 1.004, 95% CI 1.001–1.008 per 10 μg/m³), with amplified effects during influenza peaks. The exposure–response curve was monotonic but nonlinear, with measurable risks evident even below the World Health Organization 24-hour guideline (15 μg/m³) and sharply increasing risks above 50 μg/m³. These findings provide real-world, population-level, multi-city evidence of acute respiratory effects from short-term PM₂.₅ exposure, reveal the absence of a safe threshold, and underscore the urgency of stringent air-quality interventions to protect respiratory health. This study also demonstrates how large-scale, AI-based mobile sensing can transform real-time environmental epidemiology by enabling population-level monitoring of early respiratory symptoms worldwide.
- 2 December 2025
- Published by Springer Nature in Research Square
Open AccessUncovering Respiratory Effects of Short-term PM2.5 Exposure through Global Nocturnal Cough Monitoring
Air pollution remains one of the leading environmental determinants of health, yet the short-term effects of fine particulate matter (PM₂.₅) on early respiratory symptoms are poorly characterized owing to the limited availability of high-resolution, population-scale data. Enabled by global mobile sensing through a widely used application (Sleep Cycle) and AI algorithms that automatically detect coughs from nocturnal audio recordings, we derived nighttime cough frequency across 34 cities in 12 countries on five continents over 500 consecutive days, conducting the first multi-city assessment linking daily PM₂.₅ exposure to nocturnal respiratory symptoms. Single-city generalized additive models, multi-city panel regressions adjusting for meteorological variables and influenza rates, and quasi-experimental analyses consistently showed that same-day PM₂.₅ exposure was associated with higher cough frequency (relative risk = 1.004, 95% CI 1.001–1.008 per 10 μg/m³), with amplified effects during influenza peaks. The exposure–response curve was monotonic but nonlinear, with measurable risks evident even below the World Health Organization 24-hour guideline (15 μg/m³) and sharply increasing risks above 50 μg/m³. These findings provide real-world, population-level, multi-city evidence of acute respiratory effects from short-term PM₂.₅ exposure, reveal the absence of a safe threshold, and underscore the urgency of stringent air-quality interventions to protect respiratory health. This study also demonstrates how large-scale, AI-based mobile sensing can transform real-time environmental epidemiology by enabling population-level monitoring of early respiratory symptoms worldwide.
- Open Access
- 29 October 2025
- Published by MDPI AG in Clinics and Practice
- Vol. 15 (11) , 200
Advancing Toward P6 Medicine: Recommendations for Integrating Artificial Intelligence in Internal Medicine
Background: Internists formulate diagnostic hypotheses and personalized treatment plans by integrating data from a comprehensive clinical interview, reviewing a patient’s medical history, physical examination and findings from complementary tests. The patient treatment life cycle generates a significant volume of data points that can offer valuable insights to improve patient care by guiding clinical decision-making. Artificial Intelligence (AI) and, in particular, Generative AI (GAI), are promising tools in this regard, particularly after the introduction of Large Language Models. The European Federation of Internal Medicine (EFIM) recognizes the transformative impact of AI in leveraging clinical data and advancing the field of internal medicine. This position paper from the EFIM explores how AI can be applied to achieve the goals of P6 Medicine principles in internal medicine. P6 Medicine is an advanced healthcare model that extends the concept of Personalized Medicine toward a holistic, predictive, patient-centered approach that also integrates psycho-cognitive and socially responsible dimensions. An additional concept introduced is that of Digital Therapies (DTx), software applications designed to prevent and manage diseases and disorders through AI, which are used in the clinical setting if validated by rigorous research studies. Methods: The literature examining the relationship between AI and Internal Medicine was investigated through a bibliometric analysis. The themes identified in the literature review were further examined through the Delphi method. Thirty international AI and Internal Medicine experts constituted the Delphi panel. Results: Delphi results were summarized in a SWOT Analysis. The evidence is that through extensive data analysis, diagnostic capacity, drug development and patient tracking are increased. Conclusions: The panel unanimously considered AI in Internal Medicine as an opportunity, achieving a complete consensus on the matter. AI-driven solutions, including clinical applications of GAI and DTx, hold the potential to strongly change internal medicine by streamlining workflows, enhancing patient care and generating valuable data.
- 29 October 2025
- Published by MDPI AG in Clinics and Practice
- Vol. 15 (11) , 200
Open AccessAdvancing Toward P6 Medicine: Recommendations for Integrating Artificial Intelligence in Internal Medicine
Background: Internists formulate diagnostic hypotheses and personalized treatment plans by integrating data from a comprehensive clinical interview, reviewing a patient’s medical history, physical examination and findings from complementary tests. The patient treatment life cycle generates a significant volume of data points that can offer valuable insights to improve patient care by guiding clinical decision-making. Artificial Intelligence (AI) and, in particular, Generative AI (GAI), are promising tools in this regard, particularly after the introduction of Large Language Models. The European Federation of Internal Medicine (EFIM) recognizes the transformative impact of AI in leveraging clinical data and advancing the field of internal medicine. This position paper from the EFIM explores how AI can be applied to achieve the goals of P6 Medicine principles in internal medicine. P6 Medicine is an advanced healthcare model that extends the concept of Personalized Medicine toward a holistic, predictive, patient-centered approach that also integrates psycho-cognitive and socially responsible dimensions. An additional concept introduced is that of Digital Therapies (DTx), software applications designed to prevent and manage diseases and disorders through AI, which are used in the clinical setting if validated by rigorous research studies. Methods: The literature examining the relationship between AI and Internal Medicine was investigated through a bibliometric analysis. The themes identified in the literature review were further examined through the Delphi method. Thirty international AI and Internal Medicine experts constituted the Delphi panel. Results: Delphi results were summarized in a SWOT Analysis. The evidence is that through extensive data analysis, diagnostic capacity, drug development and patient tracking are increased. Conclusions: The panel unanimously considered AI in Internal Medicine as an opportunity, achieving a complete consensus on the matter. AI-driven solutions, including clinical applications of GAI and DTx, hold the potential to strongly change internal medicine by streamlining workflows, enhancing patient care and generating valuable data.
- Open Access
- 18 March 2025
- Published by JMIR Publications Inc. in JMIR Medical Education
- Vol. 11 (1) , e55709
Impact of Clinical Decision Support Systems on Medical Students’ Case-Solving Performance: Comparison Study with a Focus Group
Background: Health care practitioners use clinical decision support systems (CDSS) as an aid in the crucial task of clinical reasoning and decision-making. Traditional CDSS are online repositories (ORs) and clinical practice guidelines (CPG). Recently, large language models (LLMs) such as ChatGPT have emerged as potential alternatives. They have proven to be powerful, innovative tools, yet they are not devoid of worrisome risks. Objective: This study aims to explore how medical students perform in an evaluated clinical case through the use of different CDSS tools. Methods: The authors randomly divided medical students into 3 groups, CPG, n=6 (38%); OR, n=5 (31%); and ChatGPT, n=5 (31%); and assigned each group a different type of CDSS for guidance in answering prespecified questions, assessing how students’ speed and ability at resolving the same clinical case varied accordingly. External reviewers evaluated all answers based on accuracy and completeness metrics (score: 1‐5). The authors analyzed and categorized group scores according to the skill investigated: differential diagnosis, diagnostic workup, and clinical decision-making. Results: Answering time showed a trend for the ChatGPT group to be the fastest. The mean scores for completeness were as follows: CPG 4.0, OR 3.7, and ChatGPT 3.8 ( P =.49). The mean scores for accuracy were as follows: CPG 4.0, OR 3.3, and ChatGPT 3.7 ( P =.02). Aggregating scores according to the 3 students’ skill domains, trends in differences among the groups emerge more clearly, with the CPG group that performed best in nearly all domains and maintained almost perfect alignment between its completeness and accuracy. Conclusions: This hands-on session provided valuable insights into the potential perks and associated pitfalls of LLMs in medical education and practice. It suggested the critical need to include teachings in medical degree courses on how to properly take advantage of LLMs, as the potential for misuse is evident and real.
- 18 March 2025
- Published by JMIR Publications Inc. in JMIR Medical Education
- Vol. 11 (1) , e55709
Open AccessImpact of Clinical Decision Support Systems on Medical Students’ Case-Solving Performance: Comparison Study with a Focus Group
Background: Health care practitioners use clinical decision support systems (CDSS) as an aid in the crucial task of clinical reasoning and decision-making. Traditional CDSS are online repositories (ORs) and clinical practice guidelines (CPG). Recently, large language models (LLMs) such as ChatGPT have emerged as potential alternatives. They have proven to be powerful, innovative tools, yet they are not devoid of worrisome risks. Objective: This study aims to explore how medical students perform in an evaluated clinical case through the use of different CDSS tools. Methods: The authors randomly divided medical students into 3 groups, CPG, n=6 (38%); OR, n=5 (31%); and ChatGPT, n=5 (31%); and assigned each group a different type of CDSS for guidance in answering prespecified questions, assessing how students’ speed and ability at resolving the same clinical case varied accordingly. External reviewers evaluated all answers based on accuracy and completeness metrics (score: 1‐5). The authors analyzed and categorized group scores according to the skill investigated: differential diagnosis, diagnostic workup, and clinical decision-making. Results: Answering time showed a trend for the ChatGPT group to be the fastest. The mean scores for completeness were as follows: CPG 4.0, OR 3.7, and ChatGPT 3.8 ( P =.49). The mean scores for accuracy were as follows: CPG 4.0, OR 3.3, and ChatGPT 3.7 ( P =.02). Aggregating scores according to the 3 students’ skill domains, trends in differences among the groups emerge more clearly, with the CPG group that performed best in nearly all domains and maintained almost perfect alignment between its completeness and accuracy. Conclusions: This hands-on session provided valuable insights into the potential perks and associated pitfalls of LLMs in medical education and practice. It suggested the critical need to include teachings in medical degree courses on how to properly take advantage of LLMs, as the potential for misuse is evident and real.
- Open Access
- 7 October 2024
- Published by JMIR Publications Inc. in JMIR Preprints
A large language model-powered digital assistant for up-to-date COVID-19 outpatient management (Preprint)
UNSTRUCTURED The medical community is both excited and worried by the advent of artificial intelligence in healthcare, whereby two disciplines meet for the benefit of patients. This is especially true when considering large language models (LLM) for generative functions. This letter briefly describes a successful collaboration between physicians and informaticians to develop a LLM-powered chatbot for outpatient management of COVID-19.
- 7 October 2024
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessA large language model-powered digital assistant for up-to-date COVID-19 outpatient management (Preprint)
UNSTRUCTURED The medical community is both excited and worried by the advent of artificial intelligence in healthcare, whereby two disciplines meet for the benefit of patients. This is especially true when considering large language models (LLM) for generative functions. This letter briefly describes a successful collaboration between physicians and informaticians to develop a LLM-powered chatbot for outpatient management of COVID-19.
- 30 August 2024
- Published by Springer Nature in Internal and Emergency Medicine
- Vol. 20 (2) , 515-518
Less is not always more: complex interactions in blood clot mysteries
- 30 August 2024
- Published by Springer Nature in Internal and Emergency Medicine
- Vol. 20 (2) , 515-518
Less is not always more: complex interactions in blood clot mysteries
- Open Access
- 21 December 2023
- Published by JMIR Publications Inc. in JMIR Preprints
Clinical decision support systems during teaching: a hands-on comparison (Preprint)
BACKGROUND Healthcare practitioners use Clinical Decision Support Systems (CDSS) as an aid in the crucial task of clinical reasoning and decision making. Traditional CDSS are Online Repositories (OR) and Clinical Practice Guidelines (CPG). Recently, Large Language Models (LLMs) like ChatGPT have emerged as potential alternatives. They have proven to be powerful innovative tools, yet they are not devoid of worrisome risks. OBJECTIVE This study aims to explore how medical students utilize ChatGPT as a CDSS in a teaching environment. METHODS The authors randomly divided medical students into three groups and assigned each group a different type of CDSS for guidance in answering prespecified questions, assessing how students’ ability at resolving the same clinical case varied accordingly. External reviewers evaluated all answers based on accuracy and completeness metrics (Score: 1-5). The authors analyzed and categorized group scores according to the skill investigated: Differential Diagnosis, Diagnostic Workup, and Clinical Decision Making. RESULTS Answering time showed a trend for the ChatGPT group to be the fastest. The mean scores for completeness were: 4.0 (CPG), 3.7 (OR), 3.8 (ChatGPT). The mean scores for accuracy were: 4.0 (CPG), 3.3 (OR), 3.7 (ChatGPT). Aggregating scores according to the three students’ skill domains, trends in differences among the groups emerge more clearly, with the CPG group that performed best in nearly all domains and maintained almost perfect alignment between its completeness and accuracy. CONCLUSIONS This hands-on session provided valuable insights into the potential perks and associated pitfalls of LLMs in medical education and practice. It suggested the critical need to include teachings in Medical Degree Courses on how to properly take advantage of LLMs, as the potential for misuse is evident and real.
- 21 December 2023
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessClinical decision support systems during teaching: a hands-on comparison (Preprint)
BACKGROUND Healthcare practitioners use Clinical Decision Support Systems (CDSS) as an aid in the crucial task of clinical reasoning and decision making. Traditional CDSS are Online Repositories (OR) and Clinical Practice Guidelines (CPG). Recently, Large Language Models (LLMs) like ChatGPT have emerged as potential alternatives. They have proven to be powerful innovative tools, yet they are not devoid of worrisome risks. OBJECTIVE This study aims to explore how medical students utilize ChatGPT as a CDSS in a teaching environment. METHODS The authors randomly divided medical students into three groups and assigned each group a different type of CDSS for guidance in answering prespecified questions, assessing how students’ ability at resolving the same clinical case varied accordingly. External reviewers evaluated all answers based on accuracy and completeness metrics (Score: 1-5). The authors analyzed and categorized group scores according to the skill investigated: Differential Diagnosis, Diagnostic Workup, and Clinical Decision Making. RESULTS Answering time showed a trend for the ChatGPT group to be the fastest. The mean scores for completeness were: 4.0 (CPG), 3.7 (OR), 3.8 (ChatGPT). The mean scores for accuracy were: 4.0 (CPG), 3.3 (OR), 3.7 (ChatGPT). Aggregating scores according to the three students’ skill domains, trends in differences among the groups emerge more clearly, with the CPG group that performed best in nearly all domains and maintained almost perfect alignment between its completeness and accuracy. CONCLUSIONS This hands-on session provided valuable insights into the potential perks and associated pitfalls of LLMs in medical education and practice. It suggested the critical need to include teachings in Medical Degree Courses on how to properly take advantage of LLMs, as the potential for misuse is evident and real.
- Open Access
- 9 May 2023
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 25 (1) , e44804
Evaluating Listening Performance for COVID-19 Detection by Clinicians and Machine Learning: Comparative Study
Background: To date, performance comparisons between men and machines have been carried out in many health domains. Yet machine learning (ML) models and human performance comparisons in audio-based respiratory diagnosis remain largely unexplored. Objective: The primary objective of this study was to compare human clinicians and an ML model in predicting COVID-19 from respiratory sound recordings. Methods: In this study, we compared human clinicians and an ML model in predicting COVID-19 from respiratory sound recordings. Prediction performance on 24 audio samples (12 tested positive) made by 36 clinicians with experience in treating COVID-19 or other respiratory illnesses was compared with predictions made by an ML model trained on 1162 samples. Each sample consisted of voice, cough, and breathing sound recordings from 1 subject, and the length of each sample was around 20 seconds. We also investigated whether combining the predictions of the model and human experts could further enhance the performance in terms of both accuracy and confidence. Results: The ML model outperformed the clinicians, yielding a sensitivity of 0.75 and a specificity of 0.83, whereas the best performance achieved by the clinicians was 0.67 in terms of sensitivity and 0.75 in terms of specificity. Integrating the clinicians’ and the model’s predictions, however, could enhance performance further, achieving a sensitivity of 0.83 and a specificity of 0.92. Conclusions: Our findings suggest that the clinicians and the ML model could make better clinical decisions via a cooperative approach and achieve higher confidence in audio-based respiratory diagnosis.
- 9 May 2023
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 25 (1) , e44804
Open AccessEvaluating Listening Performance for COVID-19 Detection by Clinicians and Machine Learning: Comparative Study
Background: To date, performance comparisons between men and machines have been carried out in many health domains. Yet machine learning (ML) models and human performance comparisons in audio-based respiratory diagnosis remain largely unexplored. Objective: The primary objective of this study was to compare human clinicians and an ML model in predicting COVID-19 from respiratory sound recordings. Methods: In this study, we compared human clinicians and an ML model in predicting COVID-19 from respiratory sound recordings. Prediction performance on 24 audio samples (12 tested positive) made by 36 clinicians with experience in treating COVID-19 or other respiratory illnesses was compared with predictions made by an ML model trained on 1162 samples. Each sample consisted of voice, cough, and breathing sound recordings from 1 subject, and the length of each sample was around 20 seconds. We also investigated whether combining the predictions of the model and human experts could further enhance the performance in terms of both accuracy and confidence. Results: The ML model outperformed the clinicians, yielding a sensitivity of 0.75 and a specificity of 0.83, whereas the best performance achieved by the clinicians was 0.67 in terms of sensitivity and 0.75 in terms of specificity. Integrating the clinicians’ and the model’s predictions, however, could enhance performance further, achieving a sensitivity of 0.83 and a specificity of 0.92. Conclusions: Our findings suggest that the clinicians and the ML model could make better clinical decisions via a cooperative approach and achieve higher confidence in audio-based respiratory diagnosis.