An Integrated Multimodal Deep Learning Framework for Accurate Skin Disease Classification
Open Access
- 14 February 2024
- journal article
- research article
- Published by International Association of Online Engineering (IAOE) in International Journal of Online and Biomedical Engineering (iJOE)
- Vol. 20 (02) , 78-94
Abstract
In order to effectively treat skin diseases, an accurate and prompt diagnosis is required. In this article, a novel method for classifying skin disorders using a multimodal classifier is presented. The proposed classifier utilizes multiple information sources to enhance the accuracy of disease classification. It incorporates images of skin lesions and patient-specific data. The multimodal classifier simultaneously classifies diseases by combining image and structured data inputs. The effectiveness of the proposed classifier was evaluated using the ISIC 2018 dataset, which includes images and clinical data for seven categories of skin diseases. The results indicate that the proposed model outperforms conventional single-modal and single-task classifiers, achieving an accuracy of 98.66% for image classification and 94.40% for clinical data classification. In addition, we compare the performance of the proposed model with that of other methodologies, demonstrating its superiority. Despite yielding promising results, the proposed method has limitations in terms of data requirements and generalizability. Future research directions include incorporating additional information sources, investigating genetic data integration, and applying the method to various medical conditions. This study illustrates the potential of integrating multimodal techniques with transfer learning in deep neural networks to enhance the classification accuracy of cutaneous diseases.Keywords
This publication has 37 references indexed in Scilit:
- 2-HDCNN: A two-tier hybrid dual convolution neural network feature fusion approach for diagnosing malignant melanomaComputers in Biology and Medicine, 2022
- SSD-KD: A self-supervised diverse knowledge distillation method for lightweight skin lesion classification using dermoscopic imagesMedical Image Analysis, 2022
- Handwritten computer science words vocabulary recognition using concatenated convolutional neural networksMultimedia Tools and Applications, 2022
- RETRACTED ARTICLE: Model hybridization & learning rate annealing for skin cancer detectionMultimedia Tools and Applications, 2022
- A multimodal transformer to fuse images and metadata for skin disease classificationThe Visual Computer, 2022
- Ensemble-based bag of features for automated classification of normal and COVID-19 CXR imagesBiomedical Signal Processing and Control, 2021
- Multimodal Multi-tasking for Skin Lesion Classification Using Deep Neural Networks2021
- Convolutional Neural Network Architecture for Offline Handwritten Characters Recognition2019
- Similarity encoding for learning with dirty categorical variablesMachine Learning, 2018
- Evaluation Metrics and Evaluation2018