Validation of a Market-Approved Artificial Intelligence Mobile Health App for Skin Cancer Screening: A Prospective Multicenter Diagnostic Accuracy Study
Open Access
- 4 February 2022
- journal article
- research article
- Published by S. Karger AG in Dermatology
- Vol. 238 (4) , 649-656
Abstract
Background: Mobile health (mHealth) consumer applications (apps) have been integrated with deep learning for skin cancer risk assessments. However, prospective validation of these apps is lacking. Objectives: To identify the diagnostic accuracy of an app integrated with a convolutional neural network for the detection of premalignant and malignant skin lesions. Methods: We performed a prospective multicenter diagnostic accuracy study of a CE-marked mHealth app from January 1 until August 31, 2020, among adult patients with at least one suspicious skin lesion. Skin lesions were assessed by the app on an iOS or Android device after clinical diagnosis and before obtaining histopathology. The app outcome was compared to the histopathological diagnosis, or if not available, the clinical diagnosis by a dermatologist. The primary outcome was the sensitivity and specificity of the app to detect premalignant and malignant skin lesions. Subgroup analyses were conducted for different smartphone types, the lesion’s origin, indication for dermatological consultation, and lesion location. Results: In total, 785 lesions, including 418 suspicious and 367 benign control lesions, among 372 patients (50.8% women) with a median age of 71 years were included. The app performed at an overall 86.9% (95% CI 82.3–90.7) sensitivity and 70.4% (95% CI 66.2–74.3) specificity. The sensitivity was significantly higher on the iOS device compared to the Android device (91.0 vs. 83.0%; p = 0.02). Specificity calculated on benign control lesions was significantly higher than suspicious skin lesions (80.1 vs. 45.5%; p < 0.001). Sensitivity was higher in skin fold areas compared to smooth skin areas (92.9 vs. 84.2%; p = 0.01), while the specificity was higher for lesions in smooth skin areas (72.0 vs. 56.6%; p = 0.02). Conclusion: The diagnostic accuracy of the mHealth app is far from perfect, but is potentially promising to empower patients to self-assess skin lesions before consulting a health care professional. An additional prospective validation study, particularly for suspicious pigmented skin lesions, is warranted. Furthermore, studies investigating mHealth implementation in the lay population are needed to demonstrate the impact on health care systems.Keywords
This publication has 18 references indexed in Scilit:
- Teledermatology for diagnosing skin cancer in adultsCochrane Database of Systematic Reviews, 2018
- mHealth App for Risk Assessment of Pigmented and Nonpigmented Skin Lesions—A Study on Sensitivity and Specificity in Detecting MalignancyTelemedicine and e-Health, 2017
- Current approach of skin lesions suspected of malignancy in general practice in the Netherlands: a quantitative overviewJournal of the European Academy of Dermatology and Venereology, 2017
- The global burden of melanoma: results from the Global Burden of Disease Study 2015British Journal of Dermatology, 2017
- Dermatologist-level classification of skin cancer with deep neural networksNature, 2017
- STARD 2015: an updated list of essential items for reporting diagnostic accuracy studiesBMJ, 2015
- Skin lesions suspected of malignancy: an increasing burden on general practiceBMC Family Practice, 2014
- Dermoscopy compared with naked eye examination for the diagnosis of primary melanoma: a meta-analysis of studies performed in a clinical settingBritish Journal of Dermatology, 2008
- Superiority of a cognitive education with photographs over ABCD criteria in the education of the general population to the early detection of melanoma: A randomized studyInternational Journal of Cancer, 2005
- Diagnostic accuracy of dermoscopyThe Lancet Oncology, 2002