Validity of a deep learning algorithm for detecting wheezes and crackles from lung sound recordings in adults
Preprint
Abstract
We validated our state-of-the-art deep learning algorithm for detection of wheezes and crackles in sound files by comparing the classification of our algorithm with those of human experts. We had two validation sets classified by experienced raters that were not used to train the algorithm with 615 (A) and 120 (B) sound files, respectively. We calculated Area Under Curve (AUC) of the algorithm’s probability scores for wheezes and crackles. We dichotomized the scores and calculated sensitivity and specificity as well as kappa agreement. In set A, the AUC was 0.88 (95% CI 0.84 – 0.92) for wheezes and 0.88 (95% CI 0.84 – 0.92) for crackles. The sensitivities and specificities of the labels were 81% and 89% for wheezes and 67% and 96% for crackles. In set B, the kappa agreement between the algorithm and the validation set was 0.78 (95% CI 0.58 – 0.99) for wheezes and 0.75 (95% CI 0.59 – 0.92) for crackles. The 24 observers who had rated the same 120 sound files agreed less with the reference classification with a mean kappa of 0.68 for wheezes and 0.55 for crackles. We found the algorithm to be superior to doctors in detecting wheezes and crackles in lung sound files.Keywords
This publication has 29 references indexed in Scilit:
- Practical implementation of artificial intelligence algorithms in pulmonary auscultation examinationEuropean Journal of Pediatrics, 2019
- International perception of lung sounds: a comparison of classification across some European bordersBMJ Open Respiratory Research, 2017
- Listening panel agreement and characteristics of lung sounds digitally recorded from children aged 1–59 months enrolled in the Pneumonia Etiology Research for Child Health (PERCH) case–control studyBMJ Open Respiratory Research, 2017
- Automatic adventitious respiratory sound analysis: A systematic reviewPLOS ONE, 2017
- Rethinking the Inception Architecture for Computer Vision2016
- Wheezes, crackles and rhonchi: simplifying description of lung sounds increases the agreement on their classification: a study of 12 physicians' classification of lung sounds from video recordingsBMJ Open Respiratory Research, 2016
- STARD 2015: an updated list of essential items for reporting diagnostic accuracy studiesBMJ, 2015
- Fundamentals of Lung AuscultationNew England Journal of Medicine, 2014
- Use of serum C reactive protein and procalcitonin concentrations in addition to symptoms and signs to predict pneumonia in patients presenting to primary care with acute cough: diagnostic studyBMJ, 2013
- Narrative Review: Should Teaching of the Respiratory Physical Examination Be Restricted Only to Signs with Proven Reliability and Validity?Journal of General Internal Medicine, 2010