Quality of Tinnitus Information From Generative AI Systems and Web Search: An Expert‑Rated Comparative Study
- 20 July 2026
- journal article
- research article
- Published by Wolters Kluwer Health in Otology & Neurotology
- Vol. 47 (9) , 1196-1204
Abstract
Objective: To compare the quality of tinnitus-related information generated by multiple generative artificial intelligence (GenAI) systems and web search using expert evaluation. Study design: Cross-sectional comparative study. Setting: Digital platforms evaluated in their native public interfaces. Patients: Not applicable. Thirty commonly searched tinnitus-related questions derived from United States Google Trends data (2020–2025). Intervention(s): Questions were submitted to 6 GenAI systems (OpenEvidence, Claude, DeepSeek, GPT-5, Gemini, GPT-4) and Google Search (first organic result). Responses were independently rated by 6 experts using the QAMAI framework. Main outcome measure(s): Mean expert-rated quality scores across 5 domains (accuracy, clarity, relevance, completeness, and usefulness). Results: Overall quality differed significantly across systems (Friedman P r =0.49–0.86). Inter-rater reliability was high (ICC=0.82). Readability demonstrated an inverse pattern relative to expert-rated quality. OpenEvidence demonstrated the lowest readability (Flesch-Kincaid Grade Level 17.5; Flesch Reading Ease 2.2), corresponding to a postgraduate reading level, whereas general-purpose LLMs produced more accessible responses at a sixth–seventh grade reading level. Conclusions: The quality of tinnitus information varies substantially across digital platforms. While GenAI systems generally outperform web search, deficiencies in completeness persist. Readability analysis revealed an inverse relationship between expert-rated quality and response accessibility, suggesting that clinician and patient assessments of informational value may not always align. These findings highlight the need for continued evaluation and clinician oversight to ensure safe, comprehensive, and accessible patient-facing information.Keywords
This publication has 61 references indexed in Scilit:
- Paradigm Shift in Online Health Information Search in the Era of Generative AI ? – A Bibliometric Literature Survey and Sentiment AnalysisProceedings of the Association for Information Science and Technology, 2025
- Do large language models meet professional standards in rhinoplasty?? A comparative evaluation with AAO-HNS guidelinesEuropean Journal of Plastic Surgery, 2025
- Surgeon, Trainee, or GPT ? A Blinded Multicentric Study of AI ‐Augmented Operative NotesThe Laryngoscope, 2025
- Can AI Fix the Consent Form? Promise, Pitfalls, and Practical RealitiesWorld Journal of Surgery, 2025
- Are chatbots a reliable source for patient frequently asked questions on neck masses?European Archives of Oto-Rhino-Laryngology, 2025
- Large Language Models for Chatbot Health Advice StudiesJAMA Network Open, 2025
- ESR Essentials: diagnostic strategies in tinnitus—practice recommendations by the European Society of Head and Neck RadiologyEuropean Radiology, 2025
- Validation of the Quality Analysis of Medical Artificial Intelligence (QAMAI) tool: a new tool to assess the quality of health information provided by AI platformsEuropean Archives of Oto-Rhino-Laryngology, 2024
- Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media ForumJAMA Internal Medicine, 2023
- Prevalence, Severity, Exposures, and Treatment Patterns of Tinnitus in the United StatesJAMA Otolaryngology–Head & Neck Surgery, 2016