Graph-Based Fusion of Imaging and Non-Imaging Data for Disease Trajectory Prediction
Authors:
- 24 April 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-4
Abstract
This study proposes a graph convolutional neural networks (GCN) architecture for fusion of radiological imaging and non-imaging tabular electronic health records (EHR) for the purpose of clinical event prediction. We focused on a cohort of hospitalized patients with positive RT-PCR test for COVID-19 and developed GCN based models to predict three dependent clinical events (discharge from hospital, admission into ICU, and mortality) using demographics, billing codes for procedures and diagnoses and chest X-rays. We hypothesized that the two-fold learning opportunity provided by the GCN is ideal for fusion of imaging information and tabular data as node and edge features, respectively. Our experiments indicate the validity of our hypothesis where GCN based predictive models outperform single modality and traditional fusion models. We compared the proposed models against two variations of imaging-based models, including DenseNet-121 architecture with learnable classification layers and Random Forest classifiers using disease severity score estimated by pre-trained convolutional neural network. GCN based model outperforms both imaging-only methods. We also validated our models on an external dataset where GCN showed valuable generalization capabilities. We noticed that edge-formation function can be adapted even after training the GCN model without limiting application scope of the model. Our models take advantage of this fact for generalization to external data.Keywords
Funding Information
- National Science Foundation
- National Institute of Health
- NIBIB
- NHLBI
This publication has 8 references indexed in Scilit:
- Using DeepGCN to identify the autism spectrum disorder from multi-site resting-state dataBiomedical Signal Processing and Control, 2021
- Machine Learning to Predict Mortality and Critical Events in a Cohort of Patients With COVID-19 in New York City: Model Development and ValidationJournal of Medical Internet Research, 2020
- COVIDiagnosis-Net: Deep Bayes-SqueezeNet based diagnosis of the coronavirus disease 2019 (COVID-19) from X-ray imagesMedical Hypotheses, 2020
- Using Machine Learning to Predict ICU Transfer in Hospitalized COVID-19 PatientsJournal of Clinical Medicine, 2020
- An interpretable mortality prediction model for COVID-19 patientsNature Machine Intelligence, 2020
- CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert ComparisonProceedings of the AAAI Conference on Artificial Intelligence, 2019
- Disease prediction using graph convolutional networks: Application to Autism Spectrum Disorder and Alzheimer’s diseaseMedical Image Analysis, 2018
- Densely Connected Convolutional Networks2017