Generalizability and usefulness of artificial intelligence for skin cancer diagnostics: An algorithm validation study
Open Access
- 3 October 2022
- journal article
- research article
- Published by Wiley in JEADV Clinical Practice
- Vol. 1 (4) , 344-354
Abstract
Background: Artificial intelligence can be trained to outperform dermatologists in image‐based skin cancer diagnostics. However, the networks' sensitivity to biases and overfitting may hamper their clinical applicability.Objectives: The aim of this study was to explain the potential consequences of implementing convolutional neural networks for stand‐alone melanoma diagnostics and skin lesion triage.Methods: In this algorithm validation study on retrospective data, we reproduced and evaluated the performance of state‐of‐the‐art artificial intelligence (convolutional neural networks) for skin cancer diagnostics. The networks were trained on 25,331 annotated dermoscopic skin lesion images from an open‐source data set (ISIC‐2019) and tested using a novel data set (AISC‐2021) consisting of 26,591 annotated dermoscopic skin lesion images. We tested the trained algorithms' ability to generalize to new data and their diagnostic performance in two simulations (melanoma diagnostics and skin lesion triage).Results: The trained algorithms performed significantly less accurate diagnostics on images of nevi, melanomas and actinic keratoses from the AISC‐2021 data set than the ISIC‐2019 data set (p < 0.003). Almost one‐third (31.1%) of the melanomas were misclassified during the melanoma diagnostics simulation, irrespective of their Breslow thickness. Furthermore, the algorithms marked 92.7% of the lesions ‘suspicious’ during the triage simulation, which yielded a triage sensitivity and specificity of 99.7% and 8.2%, respectively.Conclusions: Although state‐of‐the‐art artificial intelligence outperforms dermatologists on image‐based skin lesion classification within an artificial setting, additional data and technological advances are needed before clinical implementation.Keywords
Funding Information
- Innovationsfonden (9093‐00005B)
This publication has 37 references indexed in Scilit:
- The Growing Burden of Invasive Melanoma: Projections of Incidence Rates and Numbers of New Cases in Six Susceptible Populations through 2031Journal of Investigative Dermatology, 2016
- Global teledermatologyGlobal Dermatology, 2016
- Prevalence and Costs of Skin Cancer Treatment in the U.S., 2002−2006 and 2007−2011American Journal of Preventive Medicine, 2015
- Clinical performance of the Nevisense system in cutaneous melanoma detection: an international, multicentre, prospective and blinded clinical trial on efficacy and safetyBritish Journal of Dermatology, 2014
- Segmentation of Skin Lesions From Digital Images Using Joint Statistical Texture DistinctivenessIEEE Transactions on Biomedical Engineering, 2014
- The role of dynamic infrared imaging in melanoma diagnosisExpert Review of Dermatology, 2013
- A novel method for color correction in epiluminescence microscopyComputerized Medical Imaging and Graphics, 2011
- Prognostic Significance of Mitotic Rate in Localized Primary Cutaneous Melanoma: An Analysis of Patients in the Multi-Institutional American Joint Committee on Cancer Melanoma Staging DatabaseJournal of Clinical Oncology, 2011
- The Performance of MelaFindArchives of Dermatology, 2011
- The meaning and use of the area under a receiver operating characteristic (ROC) curve.Radiology, 1982