Performance of the ‘DermDx’ algorithm in triaging suspected skin cancers: results of the AI-SCSS study
Open Access
- 27 August 2026
- journal article
- Published by Oxford University Press (OUP) in Skin Health and Disease
Abstract
Background: The evidence base for the use of artificial intelligence (AI) in skin cancer diagnosis consists largely of artificial ‘reader’ studies and incomplete postimplementation survey data, leading to the National Institute for Clinical Excellence placing restrictions on use in the UK and raising questions regarding the value of AI as a medical device vs. traditional human teledermatology. Objectives: To assess the real-world performance of a convolutional neural network-based system (‘DermDx’; MetaOptima Technology) in triaging skin lesions referred to secondary care on a UK suspected skin cancer pathway. Methods: This was a blinded observational study to evaluate the performance of an AI algorithm on a consecutive series of referred patients with skin lesions of concern (March–April 2024). It was conducted at a single-centre secondary care National Health Service dermatology department in Cornwall, UK. All lesions referred from primary care on a suspected skin cancer pathway and imaged in a community lesion imaging clinic were eligible for inclusion. Exclusion criteria included the absence of research consent; acral, mucosal, umbilical and eyelid lesions; and inadequate image quality or artefacts. Dermoscopic images were captured with an Apple iPad and a DermLite DL4 dermatoscope, and dermoscopic images were assessed retrospectively in a blinded manner using the DermDx algorithm. Notes review was performed at 1 year to identify misdiagnoses. The main outcome measure for this study was to determine the sensitivity for detection of malignant and premalignant lesions. Results: A total of 1024 lesions were included; 298 (29.1%) lesions were malignant, 271 (26.5%) were premalignant and 455 (44.4%) benign. The algorithm categorized 273 (26.7%) as low risk and 751 (73.3%) as high risk. Sensitivity for malignant and premalignant lesions was 98.1% [n = 558/569, 95% confidence interval (CI) 96.5–99.0]. Specificity for malignant and premalignant lesions was 57.6% (n = 262/455, 95% CI 53.0–62.0). The area under the receiver operating characteristic curve was 0.918 (95% CI 0.900–0.935). DermDx missed 2 of 298 (0.7%) skin cancers. Dermatologists missed 5 of 298 (1.7%) skin cancers (P = 0.25). Use of DermDx as a triage tool would have reduced face-to-face consultations by 26.7%. Conclusions: The DermDx algorithm performed extremely well as a binary classification tool, with sensitivity for malignancy at least as good as that of dermatologists in this real-world setting.Keywords
Funding Information
- MetaOptima Ltd
- Islacare Ltd
- Royal Cornwall Hospitals Trust
- Eurostars EUREKA (6384)
This publication has 31 references indexed in Scilit:
- Artificial intelligence and skin melanomaClinics in Dermatology, 2024
- Assessment of Diagnostic Performance of Dermatologists Cooperating With a Convolutional Neural Network in a Prospective Clinical StudyJAMA Dermatology, 2023
- Generalizability and usefulness of artificial intelligence for skin cancer diagnostics: An algorithm validation studyJEADV Clinical Practice, 2022
- Validation of artificial intelligence prediction models for skin cancer diagnosis using dermoscopy images: the 2019 International Skin Imaging Collaboration Grand ChallengeThe Lancet Digital Health, 2022
- Skin cancer classification via convolutional neural networks: systematic review of studies involving human expertsEuropean Journal of Cancer, 2021
- Skin lesions of face and scalp – Classification by a market-approved convolutional neural network in comparison with 64 dermatologistsEuropean Journal of Cancer, 2021
- Man against machine reloaded: performance of a market-approved convolutional neural network in classifying a broad spectrum of skin lesions in comparison with 96 dermatologists working under less artificial conditionsAnnals of Oncology, 2020
- Assessment of Accuracy of an Artificial Intelligence Algorithm to Detect Melanoma in Images of Skin LesionsJAMA Network Open, 2019
- Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification: an open, web-based, international, diagnostic studyThe Lancet Oncology, 2019
- Expert-Level Diagnosis of Nonpigmented Skin Cancer by Combined Convolutional Neural NetworksJAMA Dermatology, 2019