A Novel Dual-Modality Dual-View Hybrid Deep Learning–Machine Learning Framework for the Prediction of Carotid Plaque Vulnerability via Late Fusion
Open Access
- 9 March 2026
- journal article
- research article
- Published by MDPI AG in Diagnostics
- Vol. 16 (5) , 807
Abstract
Background: Ultrasound imaging is an ideal tool for regular carotid plaque screening to identify individuals at high risk of stroke for clinical intervention. However, no existing study leverages multi-modal multi-view ultrasound imaging for AI-enabled auto-classification of carotid plaque vulnerability. This study aims to develop and validate an effective AI model for carotid plaque vulnerability classification through the applications of dual-modal (B-Mode and contrast-enhanced mode) dual-view (longitudinal and cross-sectional) settings to maximize the utility and potential of ultrasound imaging. Methods: Hybrid deep-learning (DL) and machine-learning (ML) methods were employed to balance between model discriminability and interpretability. B-Mode ultrasound (BMUS) and contrast-enhanced ultrasound (CEUS) images from 241 patients were retrospectively analyzed using the proposed hybrid-DL-ML variants. Results: Our findings suggest the hybrid VGG-RF model developed from a dual-modal dual-view setting outperforms those developed from other settings for identifying vulnerable carotid plaques. The VGG-RF model emerged as the best-performing model, achieving an optimal performance with an AUC of 0.908, precision of 0.765, recall of 0.929, specificity of 0.886, and F1 score of 0.839. The inherent interpretability of the VGG-RF model divulged that long-axis views of BMUS and CEUS images were the major contributing features for discriminating vulnerable carotid plaques against their counterparts. Conclusions: The present study underscored the effectiveness of AI models developed from dual-modal dual-view settings of ultrasound images. Notably, the hybrid VGG-RF model was benchmarked as the best-performing model among other studied hybrid DL-ML variants. Further studies on a larger cohort in a prospective setting are warranted to validate the findings of the current study.Keywords
Funding Information
- Hong Kong Polytechnic University (P0053754)
- Hong Kong Polytechnic University (P0043132)
- Hong Kong Polytechnic University (P0043005)
This publication has 50 references indexed in Scilit:
- A Robust and Explainable Structure-Based Algorithm for Detecting the Organ Boundary From Ultrasound Multi-DatasetsJournal of Digital Imaging, 2023
- A new xAI framework with feature explainability for tumors decision-making in Ultrasound data: comparing with Grad-CAMComputer Methods and Programs in Biomedicine, 2023
- H-ProSeg: Hybrid ultrasound prostate segmentation based on explainability-guided mathematical modelComputer Methods and Programs in Biomedicine, 2022
- Collaboration based multi-modal multi-label learningApplied Intelligence, 2022
- Ultrasound-based internal carotid artery plaque characterization using deep learning paradigm on a supercomputer: a cardiovascular disease/stroke risk assessment systemThe International Journal of Cardiovascular Imaging, 2021
- Cervical Carotid Plaque MRIClinical Neuroradiology, 2021
- Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based LocalizationInternational Journal of Computer Vision, 2019
- Multiparametric ultrasound evaluation with CEUS and shear wave elastography for carotid plaque risk stratificationJournal of Ultrasound, 2018
- Atherosclerotic plaque tissue characterization in 2D ultrasound longitudinal carotid scans for automated classification: a paradigm for stroke risk assessmentMedical & Biological Engineering & Computing, 2013
- Completely automated robust edge snapper for carotid ultrasound IMT measurement on a multi-institutional database of 300 imagesMedical & Biological Engineering & Computing, 2011