Multimodal Decision Support System for Improved Diagnosis and Healthcare Decision Making
Preprint
- 14 April 2025
- preprint
- Published by Springer Nature in Research Square
Abstract
Access to quality health care continues to be a major challenge for remote or overlooked regions that do not have the necessary medical infrastructures and human resources. This research proposes integrating structured data-laboratory results, vitals, and demographics- with unstructured medical data-clinical notes, free-text diagnosis-finalized into Multimodal Decision Support Systems (MDSS)-that closes the healthcare gap by enhancing the diagnostic accuracy and treatment recommendations. Our innovative approach entails employing Random Forest Classifiers for structured data and BERT-based embeddings for unstructured data and fuses their predictive outputs through late fusion technique. Among various evaluated fusion methods, including simple average, weighted average, and stacked fusion, the stacked fusion approach resulted in achieving maximum diagnostic accuracy, i.e., 87 against individual models, thus taking diagnostic accuracy improvement into huge consideration as well as significant reductions in misdiagnosis, in addition to last but not least, personalized healthcare recommendation, especially to rural populations. The evaluation of this system used the MIMIC-IV dataset and showed improved performance in risk prediction and analysis of patient outcomes. The first generation of our smart health care assistant will feature video consultations and multi-lingual support, as well as real-time processing capabilities to allow access to high-quality health care for these populations. Future improvements in optimizing data imputation, enhancing interpretability, and ensuring HIPAA and GDPR compliance will make for secure and ethical data usage. This work will build ground work for AI-Personal Healthcare Solutions with a long-term goal of bridging the gap between rural and urban patient populations in access to care.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- Multimodal Machine Learning in Image-Based and Clinical Biomedicine: Survey and ProspectsInternational Journal of Computer Vision, 2024
- Artificial intelligence and multimodal data fusion for smart healthcare: topic modeling and bibliometricsArtificial Intelligence Review, 2024
- Artificial Intelligence and Decision-Making in Healthcare: A Thematic Analysis of a Systematic Review of ReviewsHealth Services Research and Managerial Epidemiology, 2024
- Harnessing the power of clinical decision support systems: challenges and opportunitiesOpen Heart, 2023
- Multi-modality approaches for medical support systems: A systematic review of the last decadeInformation Fusion, 2023
- A survey of multimodal information fusion for smart healthcare: Mapping the journey from data to wisdomInformation Fusion, 2023