Detecting eczema areas in digital images: an impossible task?
Authors:
Preprint
Abstract
Assessing the severity of atopic dermatitis (AD, or eczema) traditionally relies on a face-to-face assessment by healthcare professionals, and may suffer from inter- and intra-rater variability. With the expanding role of telemedicine, several machine learning algorithms have been proposed to automatically assess AD severity from digital images. Those algorithms usually detect and then delineate (“segment”) AD lesions before assessing lesional severity, and are trained using the data of AD areas detected by healthcare professionals. To evaluate the reliability of such data, we estimated the inter-rater reliability of AD segmentation in digital images. Four dermatologists independently segmented AD lesions in 80 digital images collected in a published clinical trial. We estimated the inter-rater reliability of the AD segmentation using the intra-class correlation coefficients (ICCs) at the pixel-level and the area-levels for different resolutions of the images. The average ICC was 0.45 ( SE = 0.04) corresponding to a “poor” agreement between raters, while the degree of agreement for AD segmentation varied from image to image. The AD segmentation in digital images is highly rater-dependent even between dermatologists. Such limitations need to be taken into consideration when the AD segmentation data are used to train machine learning algorithms that assess eczema severity.Keywords
All Related Versions
- Published version: JID Innovations, 2 (5), 100133.
This publication has 15 references indexed in Scilit:
- Deep learning with noisy labels: Exploring techniques and remedies in medical image analysisMedical Image Analysis, 2020
- Opportunities and obstacles for deep learning in biology and medicineJournal of The Royal Society Interface, 2018
- Atopic dermatitis in diverse racial and ethnic groups—Variations in epidemiology, genetics, clinical presentation and treatmentExperimental Dermatology, 2018
- Automatic detection and severity measurement of eczema using image processing2016
- A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability ResearchJournal of Chiropractic Medicine, 2016
- Computing Inter-Rater Reliability for Observational Data: An Overview and TutorialThe Quantitative Methods for Psychology, 2012
- A Randomised Controlled Trial of Ion-Exchange Water Softeners for the Treatment of Eczema in ChildrenPLOS Medicine, 2011
- Repeatability for Gaussian and non‐Gaussian data: a practical guide for biologistsBiological Reviews, 2010
- Measurement of body surface area involvement in atopic eczema: an impossible task?British Journal of Dermatology, 1999
- A protocol for recording the sign of flexural dermatitis in childrenBritish Journal of Dermatology, 1995