A Phylogeny-Based Benchmarking Test for Orthology Inference Reveals the Limitations of Function-Based Validation
Open Access
- 4 November 2014
- journal article
- research article
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 9 (11) , e111122
Abstract
Accurate orthology prediction is crucial for many applications in the post-genomic era. The lack of broadly accepted benchmark tests precludes a comprehensive analysis of orthology inference. So far, functional annotation between orthologs serves as a performance proxy. However, this violates the fundamental principle of orthology as an evolutionary definition, while it is often not applicable due to limited experimental evidence for most species. Therefore, we constructed high quality "gold standard" orthologous groups that can serve as a benchmark set for orthology inference in bacterial species. Herein, we used this dataset to demonstrate 1) why a manually curated, phylogeny-based dataset is more appropriate for benchmarking orthology than other popular practices and 2) how it guides database design and parameterization through careful error quantification. More specifically, we illustrate how function-based tests often fail to identify false assignments, misjudging the true performance of orthology inference methods. We also examined how our dataset can instruct the selection of a “core” species repertoire to improve detection accuracy. We conclude that including more genomes at the proper evolutionary distances can influence the overall quality of orthology detection. The curated gene families, called Reference Orthologous Groups, are publicly available at http://eggnog.embl.de/orthobench2.Keywords
This publication has 56 references indexed in Scilit:
- A Tight Link between Orthologs and Bidirectional Best Hits in Bacterial and Archaeal GenomesGenome Biology and Evolution, 2012
- Toward community standards in the quest for orthologsBioinformatics, 2012
- The Genomes OnLine Database (GOLD) v.4: status of genomic and metagenomic projects and their associated metadataNucleic Acids Research, 2011
- Orthology prediction methods: A quality assessment using curated protein familiesBioEssays, 2011
- Conceptual framework and pilot study to benchmark phylogenomic databases based on reference gene treesBriefings in Bioinformatics, 2011
- Computational methods for Gene Orthology inferenceBriefings in Bioinformatics, 2011
- OMA 2011: orthology inference among 1000 complete genomesNucleic Acids Research, 2010
- PhylomeDB v3.0: an expanding repository of genome-wide collections of trees, alignments and phylogeny-based orthology and paralogy predictionsNucleic Acids Research, 2010
- TreeFam: 2008 UpdateNucleic Acids Research, 2007
- MUSCLE: multiple sequence alignment with high accuracy and high throughputNucleic Acids Research, 2004