CAMR: cross-aligned multimodal representation learning for cancer survival prediction
Open Access
- 1 January 2023
- journal article
- research article
- Published by Oxford University Press (OUP) in Bioinformatics
- Vol. 39 (1)
Abstract
Accurately predicting cancer survival is crucial for helping clinicians to plan appropriate treatments, which largely improves the life quality of cancer patients and spares the related medical costs. Recent advances of survival prediction methods suggest that integrating complementary information from different modalities, e.g., histopathological images and genomic data, plays a key role in enhancing predictive performance. Despite promising results obtained by existing multimodal methods, the disparate and heterogeneous characteristics of multimodal data cause so-called modality gap problem, which brings in dramatically diverse modality representations in feature space. Consequently, detrimental modality gaps make it difficult for comprehensive integration of multimodal information via representation learning, and therefore pose a great challenge to further improvements of cancer survival prediction. To solve the above problems, we propose a novel method called cross-aligned multimodal representation learning (CAMR), which generates both modality-invariant and -specific representations for more accurate cancer survival prediction. Specifically, a cross-modality representation alignment learning network is introduced to reduce modality gaps by effectively learning modality-invariant representations in a common subspace, which is achieved by aligning the distributions of different modality representations through adversarial training. Besides, we adopt a cross-modality fusion module to fuse modality-invariant representations into a unified cross-modality representation for each patient. Meanwhile, CAMR learns modality-specific representations which complement modality-invariant representations and therefore provides a holistic view of the multimodal data for cancer survival prediction. Comprehensive experiment results demonstrate that CAMR can successfully narrow modality gaps and consistently yields better performance than other survival prediction methods using multimodal data. CAMR is freely available at https://github.com/wxq-ustc/CAMR Supplementary data are available at Bioinformatics online.Keywords
Funding Information
- National Natural Science Foundation of China (61971393, 61871361, 61571414, 61471331)
This publication has 40 references indexed in Scilit:
- Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image featuresNature Communications, 2016
- Evaluating the molecule-based prediction of clinical drug responses in cancerBioinformatics, 2016
- Open Access to Large Scale Datasets Is Needed to Translate Knowledge of Cancer Heterogeneity into Better Patient OutcomesPLOS Medicine, 2015
- TCGA-Assembler: open-source software for retrieving and processing TCGA dataNature Methods, 2014
- A cocktail algorithm for solving the elastic net penalized Cox’s regression in high dimensionsStatistics and Its Interface, 2013
- Quantitative Image Analysis of Cellular Heterogeneity in Breast Tumors Complements Genomic ProfilingScience Translational Medicine, 2012
- Survival prediction using gene expression data: A review and comparisonComputational Statistics & Data Analysis, 2009
- Predicting the prognosis of breast cancer by integrating clinical and microarray data with Bayesian networksBioinformatics, 2006
- CellProfiler: image analysis software for identifying and quantifying cell phenotypesGenome Biology, 2006
- THE LASSO METHOD FOR VARIABLE SELECTION IN THE COX MODELStatistics in Medicine, 1997