Artificial Intelligence Approach for Variant Reporting
Authors:
Open Access
- 1 December 2018
- journal article
- research article
- Published by American Society of Clinical Oncology (ASCO) in JCO Clinical Cancer Informatics
- Vol. 2 (2) , 1-13
Abstract
Next-generation sequencing technologies are actively applied in clinical oncology. Bioinformatics pipeline analysis is an integral part of this process; however, humans cannot yet realize the full potential of the highly complex pipeline output. As a result, the decision to include a variant in the final report during routine clinical sign-out remains challenging. We used an artificial intelligence approach to capture the collective clinical sign-out experience of six board-certified molecular pathologists to build and validate a decision support tool for variant reporting. We extracted all reviewed and reported variants from our clinical database and tested several machine learning models. We used 10-fold cross-validation for our variant call prediction model, which derives a contiguous prediction score from 0 to 1 (no to yes) for clinical reporting. For each of the 19,594 initial training variants, our pipeline generates approximately 500 features, which results in a matrix of > 9 million data points. From a comparison of naive Bayes, decision trees, random forests, and logistic regression models, we selected models that allow human interpretability of the prediction score. The logistic regression model demonstrated 1% false negativity and 2% false positivity. The final models’ Youden indices were 0.87 and 0.77 for screening and confirmatory cutoffs, respectively. Retraining on a new assay and performance assessment in 16,123 independent variants validated our approach (Youden index, 0.93). We also derived individual pathologist-centric models (virtual consensus conference function), and a visual drill-down functionality allows assessment of how underlying features contributed to a particular score or decision branch for clinical implementation. Our decision support tool for variant reporting is a practically relevant artificial intelligence approach to harness the next-generation sequencing bioinformatics pipeline output when the complexity of data interpretation exceeds human capabilities.Keywords
This publication has 43 references indexed in Scilit:
- Variant Callers for Next-Generation Sequencing Data: A Comparison StudyPLOS ONE, 2013
- Sensitive detection of somatic point mutations in impure and heterogeneous cancer samplesNature Biotechnology, 2013
- Implementing multiplexed genotyping of non-small-cell lung cancers into routine clinical practiceAnnals of Oncology, 2011
- The Evolving War on CancerCell, 2011
- Rapid targeted mutational analysis of human tumours: a clinical platform to guide personalized cancer medicineEMBO Molecular Medicine, 2010
- A survey of cross-validation procedures for model selectionStatistics Surveys, 2010
- Fast and accurate short read alignment with Burrows–Wheeler transformBioinformatics, 2009
- The Evolution of Molecular Genetic PathologyThe Journal of Molecular Diagnostics, 2008
- A review of feature selection techniques in bioinformaticsBioinformatics, 2007
- Cognitive biases and decision support systems development: a design science approachInformation Systems Journal, 2005