Chasing sleep physicians: ChatGPT-4o on the interpretation of polysomnographic results
Open Access
- 20 October 2024
- journal article
- research article
- Published by Springer Nature in European Archives of Oto-Rhino-Laryngology
- Vol. 282 (3) , 1631-1639
Abstract
Background: From a healthcare professional's perspective, the use of ChatGPT (Open AI), a large language model (LLM), offers huge potential as a practical and economic digital assistant. However, ChatGPT has not yet been evaluated for the interpretation of polysomnographic results in patients with suspected obstructive sleep apnea (OSA). Aims/objectives: To evaluate the agreement of polysomnographic result interpretation between ChatGPT-4o and a board-certified sleep physician and to shed light into the role of ChatGPT-4o in the field of medical decision-making in sleep medicine. Material and methods: For this proof-of-concept study, 40 comprehensive patient profiles were designed, which represent a broad and typical spectrum of cases, ensuring a balanced distribution of demographics and clinical characteristics. After various prompts were tested, one prompt was used for initial diagnosis of OSA and a further for patients with positive airway pressure (PAP) therapy intolerance. Each polysomnographic result was independently evaluated by ChatGPT-4o and a board-certified sleep physician. Diagnosis and therapy suggestions were analyzed for agreement. Results: ChatGPT-4o and the sleep physician showed 97% (29/30) concordance in the diagnosis of the simple cases. For the same cases the two assessment instances unveiled 100% (30/30) concordance regarding therapy suggestions. For cases with intolerance of treatment with positive airway pressure (PAP) ChatGPT-4o and the sleep physician revealed 70% (7/10) concordance in the diagnosis and 44% (22/50) concordance for therapy suggestions. Conclusion and significance: Precise prompting improves the output of ChatGPT-4o and provides sleep physician-like polysomnographic result interpretation. Although ChatGPT shows some shortcomings in offering treatment advice, our results provide evidence for AI assisted automation and economization of polysomnographic interpretation by LLMs. Further research should explore data protection issues and demonstrate reproducibility with real patient data on a larger scale.Keywords
Funding Information
- Universitätsmedizin der Johannes Gutenberg-Universität Mainz
This publication has 34 references indexed in Scilit:
- The Comparative Diagnostic Capability of Large Language Models in OtolaryngologyThe Laryngoscope, 2024
- Accuracy of ChatGPT-3.5 and -4 in providing scientific references in otolaryngology–head and neck surgeryEuropean Archives of Oto-Rhino-Laryngology, 2024
- Accuracy of ChatGPT in head and neck oncological board decisions: preliminary findingsEuropean Archives of Oto-Rhino-Laryngology, 2023
- Chat GPT for the management of obstructive sleep apnea: do we have a polar star?European Archives of Oto-Rhino-Laryngology, 2023
- Is artificial intelligence ready to replace specialist doctors entirely? ENT specialists vs ChatGPT: 1-0, ball at the centerEuropean Archives of Oto-Rhino-Laryngology, 2023
- Head‐to‐Head Comparison of ChatGPT Versus Google Search for Medical Knowledge AcquisitionOtolaryngology–Head and Neck Surgery, 2023
- “Vertigo, likely peripheral”: the dizzying rise of ChatGPTEuropean Archives of Oto-Rhino-Laryngology, 2023
- Exploring the potential of Chat-GPT as a supportive tool for sialendoscopy clinical decision making and patient information supportEuropean Archives of Oto-Rhino-Laryngology, 2023
- Diagnostic and Management Applications of ChatGPT in Structured Otolaryngology Clinical ScenariosOTO Open, 2023
- ChatGPT’s quiz skills in different otolaryngology subspecialties: an analysis of 2576 single-choice and multiple-choice board certification preparation questionsEuropean Archives of Oto-Rhino-Laryngology, 2023