Multimodal Masked Autoencoder Based on Adaptive Masking for Vitiligo Stage Classification
Authors:
Top Cited Papers
- 29 April 2025
- journal article
- research article
- Published by Springer Nature in Journal of Imaging Informatics in Medicine
- Vol. 39 (1) , 655-668
Abstract
Vitiligo, a prevalent skin condition characterized by depigmentation, presents challenges in staging due to its inherent complexity. Multimodal skin images can provide complementary information, and in this study, the integration of clinical images of vitiligo and those obtained under Wood’s lamp is conducive to the classification of vitiligo stages. However, difficulties in annotating multimodal data and the scarcity of multimodal data limit the performance of deep learning models in related classification tasks. To address these issues, a Multimodal Masked Autoencoder (Multi-MAE) based on adaptive masking is proposed in annotating multimodal data and the problem of multimodal data scarcity, and enhances the model’s ability to extract characteristics from multimodal data. Specifically, an image reconstruction task is constructed to diminish reliance on annotated multimodal data, and a pre-training strategy is employed to alleviate the scarcity of multimodal data. Experimental results demonstrate that the proposed model achieves a vitiligo stage classification accuracy of 95.48% on a dataset of unlabeled dermatological images, an improvement of 5.16%, 4.51%, 3.87%, 2.58%, 4.51%, 4.51%, 3.87%, and 2.58% over that of MobileNet, DenseNet, VGG, ResNet-50, BEIT, MaskFeat, SimMIM, and MAE, respectively. These results verify the effectiveness of the proposed Multi-MAE model in assessing the stable and active vitiligo stages, making it a suitable clinical aid for evaluating the severity of vitiligo lesions.Keywords
Funding Information
- Innovative Research Group Project of the National Natural Science Foundation of China (12126606)
This publication has 25 references indexed in Scilit:
- Learning with limited annotations: A survey on deep semi-supervised learning for medical image segmentationComputers in Biology and Medicine, 2023
- Advances in Deep Learning Models for Resolving Medical Image Segmentation Data Scarcity Problem: A Topical ReviewArchives of Computational Methods in Engineering, 2023
- Biomarkers and clinical indicators of disease activity in vitiligoDermatological Reviews, 2022
- MDFNet: application of multimodal fusion method based on skin image and clinical data to skin cancer classificationJournal of Cancer Research and Clinical Oncology, 2022
- Adversarial multimodal fusion with attention mechanism for skin lesion classification using clinical and dermoscopic imagesMedical Image Analysis, 2022
- SimMIM: a Simple Framework for Masked Image Modeling2022
- Multimodal Co-learning: Challenges, applications with datasets, recent advances and future directionsInformation Fusion, 2022
- Melanocytes and keratinocytes morphological changes in vitiligo patients. A histological, immunohistochemical and ultrastructural analysisUltrastructural Pathology, 2022
- A review on multimodal medical image fusion: Compendious analysis of medical modalities, multimodal databases, fusion techniques and quality metricsComputers in Biology and Medicine, 2022
- Image-to-Image Translation with Conditional Adversarial Networks2017