Generalising uncertainty improves accuracy and safety of deep learning analytics applied to oncology
Authors:
Open Access
- 6 May 2023
- journal article
- research article
- Published by Springer Nature in Scientific Reports
- Vol. 13 (1) , 1-14
Abstract
Uncertainty estimation is crucial for understanding the reliability of deep learning (DL) predictions, and critical for deploying DL in the clinic. Differences between training and production datasets can lead to incorrect predictions with underestimated uncertainty. To investigate this pitfall, we benchmarked one pointwise and three approximate Bayesian DL models for predicting cancer of unknown primary, using three RNA-seq datasets with 10,968 samples across 57 cancer types. Our results highlight that simple and scalable Bayesian DL significantly improves the generalisation of uncertainty estimation. Moreover, we designed a prototypical metric—the area between development and production curve (ADP), which evaluates the accuracy loss when deploying models from development to production. Using ADP, we demonstrate that Bayesian DL improves accuracy under data distributional shifts when utilising ‘uncertainty thresholding’. In summary, Bayesian DL is a promising approach for generalising uncertainty, improving performance, transparency, and safety of DL models for deployment in the real world.Keywords
All Related Versions
Funding Information
- Cooperative Research Centres, Australian Government Department of Industry (CRCPFIVE000176, CRCPFIVE000176, CRCPFIVE000176, CRCPFIVE000176, CRCPFIVE000176)
- Australian Research Council Industrial Transformation Training Centre for Information Resilience (IC200100022, IC200100022)
- National Health and Medical Research Council (APP1139071)
- NHMRC Emerging Leader 1 Investigator Grant (APP2008631)
This publication has 71 references indexed in Scilit:
- Intratumoural Heterogeneity Underlies Distinct Therapy Responses and Treatment Resistance in GlioblastomaCancers, 2019
- Cell-of-Origin Patterns Dominate the Molecular Classification of 10,000 Tumors from 33 Types of CancerCell, 2018
- Deep Learning and Its Applications in BiomedicineGenomics, Proteomics & Bioinformatics, 2018
- Integrative clinical genomics of metastatic cancerNature, 2017
- Whole-genome landscapes of major melanoma subtypesNature, 2017
- Whole-genome landscape of pancreatic neuroendocrine tumoursNature, 2017
- Genomic analyses identify molecular subtypes of pancreatic cancerNature, 2016
- Whole–genome characterization of chemoresistant ovarian cancerNature, 2015
- Review The Cancer Genome Atlas (TCGA): an immeasurable source of knowledgeContemporary Oncology, 2015
- Catastrophic forgetting in connectionist networksTrends in Cognitive Sciences, 1999