What went wrong with variant effect predictor performance for the PCM1 challenge
Open Access
- 3 July 2019
- journal article
- research article
- Published by Wiley in Human Mutation
- Vol. 40 (9) , 1486-1494
Abstract
The recent years have seen a drastic increase in the amount of available genomic sequences. Alongside this explosion, hundreds of computational tools were developed to assess the impact of observed genetic variation. Critical Assessment of Genome Interpretation (CAGI) provides a platform to evaluate the performance of these tools in experimentally relevant contexts. In the CAGI‐5 challenge assessing the 38 missense variants affecting the human Pericentriolar material 1 protein (PCM1), our SNAP‐based submission was the top performer, although it did worse than expected from other evaluations. Here, we compare the CAGI‐5 submissions, and 24 additional commonly used variant effect predictors, to analyze the reasons for this observation. We identified per residue conservation, structural, and functional PCM1 characteristics, which may be responsible. As expected, predictors had a hard time distinguishing effect variants in nonconserved positions. They were also better able to call effect variants in a structurally rich region than in a less‐structured one; in the latter, they more often correctly identified benign than effect variants. Curiously, most of the protein was predicted to be functionally robust to mutation—a feature that likely makes it a harder problem for generalized variant effect predictors.Keywords
Funding Information
- Clinical Center (NIH U01 GM115486)
- Health and Social Care Delivery Research Programme (NIH R01 MH115958 01)
This publication has 49 references indexed in Scilit:
- CADD: predicting the deleteriousness of variants throughout the human genomeNucleic Acids Research, 2018
- Neuroanatomical and behavioral deficits in mice haploinsufficient for Pericentriolar material 1 (Pcm1)Neuroscience Research, 2015
- A Statistical Framework to Predict Functional Non-Coding Regions in the Human Genome Through Integrated Analysis of Annotation DataScientific Reports, 2015
- Neutral and weakly nonneutral sequence variants may define individualityProceedings of the National Academy of Sciences, 2013
- Predicting the functional impact of protein mutations: application to cancer genomicsNucleic Acids Research, 2011
- A threonine to isoleucine missense mutation in the pericentriolar material 1 gene is strongly associated with schizophreniaMolecular Psychiatry, 2008
- Recruitment of PCM1 to the Centrosome by the Cooperative Action of DISC1 and BBS4Archives of General Psychiatry, 2008
- SNAP predicts effect of mutations on protein functionBioinformatics, 2008
- SNAP: predict effect of non-synonymous polymorphisms on functionNucleic Acids Research, 2007
- Combining evolutionary information and neural networks to predict protein secondary structureProteins: Structure, Function, and Bioinformatics, 1994