Comparison of the Audiological Knowledge of Three Chatbots: ChatGPT, Bing Chat, and Bard
Authors:
- W. Wiktor JedrzejczakW. Wiktor Jedrzejczak
- Krzysztof KochanekKKKrzysztof Kochanek
- 6 May 2024
- journal article
- research article
- Published by S. Karger AG in Audiology and Neurotology
- Vol. 29 (6) , 1-7
Abstract
Introduction: The purpose of this study was to evaluate three chatbots – OpenAI ChatGPT, Microsoft Bing Chat (currently Copilot), and Google Bard (currently Gemini) – in terms of their responses to a defined set of audiological questions. Methods: Each chatbot was presented with the same 10 questions. The authors rated the responses on a Likert scale ranging from 1 to 5. Additional features, such as the number of inaccuracies or errors and the provision of references, were also examined. Results: Most responses given by all three chatbots were rated as satisfactory or better. However, all chatbots generated at least a few errors or inaccuracies. ChatGPT achieved the highest overall score, while Bard was the worst. Bard was also the only chatbot unable to provide a response to one of the questions. ChatGPT was the only chatbot that did not provide information about its sources. Conclusions: Chatbots are an intriguing tool that can be used to access basic information in a specialized area like audiology. Nevertheless, one needs to be careful, as correct information is not infrequently mixed in with errors that are hard to pick up unless the user is well versed in the field.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- Assessing the accuracy and completeness of artificial intelligence language models in providing information on methotrexate useRheumatology International, 2023
- The performance of ChatGPT in generating answers to clinical questions in psychiatry: a two‐layer assessmentWorld Psychiatry, 2023
- Validity of the large language model ChatGPT (GPT4) as a patient information source in otolaryngology by a variety of doctors in a tertiary otorhinolaryngology departmentActa Oto-Laryngologica, 2023
- Comparative Performance of ChatGPT and Bard in a Text-Based Radiology Knowledge AssessmentCanadian Association of Radiologists Journal, 2023
- Large language models in education: A focus on the complementary relationship between human teachers and ChatGPTEducation and Information Technologies, 2023
- ChatGPT takes on the European Exam in Core Cardiology: an artificial intelligence success story?European Heart Journal - Digital Health, 2023
- The Rise of AI Chatbots in Hearing Health CareThe Hearing Journal, 2023
- Comparing the Efficacy of Large Language Models ChatGPT, BARD, and Bing AI in Providing Information on Rhinoplasty: An Observational StudyAesthetic Surgery Journal Open Forum, 2023
- Chatbots: History, technology, and applicationsMachine Learning with Applications, 2020
- A Brief History of Artificial Intelligence: On the Past, Present, and Future of Artificial IntelligenceCalifornia Management Review, 2019