Pretrained transformers applied to clinical studies improve predictions of treatment efficacy and associated biomarkers
Preprint
Abstract
Cancer treatment has made significant advancements in recent decades, leading to improved outcomes and quality of life for many patients. Despite the array of available therapies, including targeted, hormone, and checkpoint blockade immunotherapy, many patients experience treatment failure or eventual resistance. Attempts to predict the efficacy of therapies, particularly immuno-oncology therapies, have suffered from limited accuracy and difficulties in identifying molecular and other determinants of response. Improving treatment prediction alone is insufficient to create clinically meaningful research tools; additional prerequisites for this goal involve accommodating small data sets, effectively handling sparse features, integrating diverse clinical data, addressing missing measurements, ensuring interpretability, and extracting valuable biological insights for both clinical context and further research. Multimodal deep-learning models offer a promising avenue to surmount these challenges by leveraging their capacity and flexibility to learn from expansive and varied clinical and molecular data sets. Similar to their application in natural language and other domains, deep-learning models can uncover complex relationships within data that are pertinent to survival and treatment response. In this study, we introduce an explainable transformer-based deep-learning framework that addresses these challenges. This framework yields predictions of survival outcomes, as quantified by concordance index, that surpass the performance of state-of-the-art methods such as Cox proportional hazards, survival random forest, and tumor mutation burden, across diverse independent data sets. We developed the clinical transformer, a deep neural-network survival prediction framework that capitalizes on the flexibility of the deep-learning model, including training strategies like gradual and transfer learning, to maximize the use of available data to enhance survival predictions and generate actionable biological insights. Finally, we illustrate the future potential of the clinical transformer’s generative capability in early-stage clinical studies. By perturbing molecular features associated with immune checkpoint inhibition treatment in immunotherapy-naive patient profiles, we identified a subset of patients who may benefit from immunotherapy. These findings were subsequently validated across three independent immunotherapy treatment cohorts. We anticipate that this research will empower the scientific community to further harness data for the benefit of patients.Keywords
All Related Versions
- Published version: Nature Communications, 16 (1), 1.
This publication has 50 references indexed in Scilit:
- UMAP: Uniform Manifold Approximation and ProjectionJournal of Open Source Software, 2018
- DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural networkBMC Medical Research Methodology, 2018
- Tumor and Microenvironment Evolution during Immunotherapy with NivolumabCell, 2017
- Predicting clinical outcomes from large scale cancer genomic profiles with deep survival modelsScientific Reports, 2017
- Biomarker development in the precision medicine era: lung cancer as a case studyNature Reviews Cancer, 2016
- Emerging applications of metabolomics in drug discovery and precision medicineNature Reviews Drug Discovery, 2016
- Genomic correlates of response to CTLA-4 blockade in metastatic melanomaScience, 2015
- Pembrolizumab for the Treatment of Non–Small-Cell Lung CancerNew England Journal of Medicine, 2015
- Mutational landscape determines sensitivity to PD-1 blockade in non–small cell lung cancerScience, 2015
- Random ForestsMachine Learning, 2001