Detailed estimation of bioinformatics prediction reliability through the Fragmented Prediction Performance Plots
Open Access
- 11 October 2007
- journal article
- research article
- Published by Springer Nature in BMC Bioinformatics
- Vol. 8 (1) , 1-7
Abstract
An important and yet rather neglected question related to bioinformatics predictions is the estimation of the amount of data that is needed to allow reliable predictions. Bioinformatics predictions are usually validated through a series of figures of merit, like for example sensitivity and precision, and little attention is paid to the fact that their performance may depend on the amount of data used to make the predictions themselves. Here I describe a tool, named Fragmented Prediction Performance Plot (FPPP), which monitors the relationship between the prediction reliability and the amount of information underling the prediction themselves. Three examples of FPPPs are presented to illustrate their principal features. In one example, the reliability becomes independent, over a certain threshold, of the amount of data used to predict protein features and the intrinsic reliability of the predictor can be estimated. In the other two cases, on the contrary, the reliability strongly depends on the amount of data used to make the predictions and, thus, the intrinsic reliability of the two predictors cannot be determined. Only in the first example it is thus possible to fully quantify the prediction performance. It is thus highly advisable to use FPPPs to determine the performance of any new bioinformatics prediction protocol, in order to fully quantify its prediction power and to allow comparisons between two or more predictors based on different types of data.Keywords
This publication has 11 references indexed in Scilit:
- The Universal Protein Resource (UniProt)Nucleic Acids Research, 2006
- The Ten Most Wanted Solutions in Protein Bioinformatics2005
- Integrating Protein Secondary Structure Prediction and Multiple Sequence AlignmentCurrent Protein & Peptide Science, 2004
- The PredictProtein serverNucleic Acids Research, 2004
- The Protein Data BankNucleic Acids Research, 2000
- Evaluation and improvement of multiple sequence methods for protein secondary structure predictionProteins: Structure, Function, and Bioinformatics, 1999
- JPred: a consensus secondary structure prediction server.Bioinformatics, 1998
- Knowledge‐based protein secondary structure assignmentProteins: Structure, Function, and Bioinformatics, 1995
- The protein data bank: A computer-based archival file for macromolecular structuresJournal of Molecular Biology, 1977
- Prediction of protein conformationBiochemistry, 1974