No one tool to rule them all: prokaryotic gene prediction tool annotations are highly dependent on the organism of study
Open Access
- 7 December 2021
- journal article
- research article
- Published by Oxford University Press (OUP) in Bioinformatics
- Vol. 38 (5) , 1198-1207
Abstract
The biases in CoDing Sequence (CDS) prediction tools, which have been based on historic genomic annotations from model organisms, impact our understanding of novel genomes and metagenomes. This hinders the discovery of new genomic information as it results in predictions being biased towards existing knowledge. To date, users have lacked a systematic and replicable approach to identify the strengths and weaknesses of any CDS prediction tool and allow them to choose the right tool for their analysis. We present an evaluation framework (ORForise) based on a comprehensive set of 12 primary and 60 secondary metrics that facilitate the assessment of the performance of CDS prediction tools. This makes it possible to identify which performs better for specific use-cases. We use this to assess 15 ab initio- and model-based tools representing those most widely used (historically and currently) to generate the knowledge in genomic databases. We find that the performance of any tool is dependent on the genome being analysed, and no individual tool ranked as the most accurate across all genomes or metrics analysed. Even the top-ranked tools produced conflicting gene collections, which could not be resolved by aggregation. The ORForise evaluation framework provides users with a replicable, data-led approach to make informed tool choices for novel genome annotations and for refining historical annotations. Code and datasets for reproduction and customisation are available at https://github.com/NickJD/ORForise. Supplementary data are available at Bioinformatics online.Keywords
All Related Versions
Funding Information
- Institute of Biological, Environmental and Rural Sciences Aberystwyth PhD fellowship
- Biotechnology and Biological Sciences Research Council (BB/E/W/10964A01, BBS/OS/GC/000011B)
- Department of Agriculture, Food and the Marine Ireland/DAERA Northern Ireland
- European Commission via Horizon 2020
This publication has 74 references indexed in Scilit:
- FragGeneScan: predicting genes in short and error-prone readsNucleic Acids Research, 2010
- Comparison of 61 Sequenced Escherichia coli GenomesMicrobial Ecology, 2010
- Ab initio gene identification in metagenomic sequencesNucleic Acids Research, 2010
- Missing genes in the annotation of prokaryotic genomesBMC Bioinformatics, 2010
- Prodigal: prokaryotic gene recognition and translation initiation site identificationBMC Bioinformatics, 2010
- Meeting report: a workshop on Best Practices in Genome AnnotationDatabase: The Journal of Biological Databases and Curation, 2010
- MetaGeneAnnotator: Detecting Species-Specific Patterns of Ribosomal Binding Site for Precise Gene Prediction in Anonymous Prokaryotic and Phage GenomesDNA Research, 2008
- The paradox of model organismsEMBO Reports, 2008
- Identifying bacterial genes and endosymbiont DNA with GlimmerBioinformatics, 2007
- MetaGene: prokaryotic gene finding from environmental genome shotgun sequencesNucleic Acids Research, 2006