Towards a standard benchmark for phenotype-driven variant and gene prioritisation algorithms: PhEval - Phenotypic inference Evaluation framework
Authors:
Open Access
- 22 March 2025
- journal article
- software
- Published by Springer Nature in BMC Bioinformatics
- Vol. 26 (1) , 1-18
Abstract
Computational approaches to support rare disease diagnosis are challenging to build, requiring the integration of complex data types such as ontologies, gene-to-phenotype associations, and cross-species data into variant and gene prioritisation algorithms (VGPAs). However, the performance of VGPAs has been difficult to measure and is impacted by many factors, for example, ontology structure, annotation completeness or changes to the underlying algorithm. Assertions of the capabilities of VGPAs are often not reproducible, in part because there is no standardised, empirical framework and openly available patient data to assess the efficacy of VGPAs—ultimately hindering the development of effective prioritisation tools. In this paper, we present our benchmarking tool, PhEval, which aims to provide a standardised and empirical framework to evaluate phenotype-driven VGPAs. The inclusion of standardised test corpora and test corpus generation tools in the PhEval suite of tools allows open benchmarking and comparison of methods on standardised data sets. PhEval and the standardised test corpora solve the issues of patient data availability and experimental tooling configuration when benchmarking and comparing rare disease VGPAs. By providing standardised data on patient cohorts from real-world case-reports and controlling the configuration of evaluated VGPAs, PhEval enables transparent, portable, comparable and reproducible benchmarking of VGPAs. As these tools are often a key component of many rare disease diagnostic pipelines, a thorough and standardised method of assessment is essential for improving patient diagnosis and careKeywords
All Related Versions
Funding Information
- Office of the Director, National Institutes of Health (#5R24 OD011883, #5R24 OD011883, #5R24 OD011883, #5R24 OD011883, #5R24 OD011883, #5R24 OD011883, #5R24 OD011883, #5R24 OD011883, #5R24 OD011883, #5R24 OD011883)
- US Department of Energy, Office of Science, Office of Basic Energy Sciences (DE-AC02-05CH11231, DE-AC02-05CH11231)
- National Human Genome Research Institute Phenomics First Resource, NIH-NHGRI (#5RM1 HG010860, #5RM1 HG010860, #5RM1 HG010860)
This publication has 38 references indexed in Scilit:
- Phenotype-aware prioritisation of rare Mendelian disease variantsTrends in Genetics, 2022
- Phenotype‐driven approaches to enhance variant prioritization and diagnosis of rare diseaseHuman Mutation, 2022
- Interpretable Clinical Genomics with a Likelihood Ratio ParadigmThe American Journal of Human Genetics, 2020
- Increasing phenotypic annotation improves the diagnostic rate of exome sequencing in a rare neuromuscular disorderHuman Mutation, 2019
- Population Genetics2019
- Towards efficiency in rare disease research: what is distinctive and important?Science China Life Sciences, 2017
- Computational evaluation of exome sequence data using human and model organism phenotypes improves diagnostic efficiencyGenetics in Medicine, 2016
- Next-generation diagnostics and disease-gene discovery with the ExomiserNature Protocols, 2015
- A global reference for human genetic variationNature, 2015
- Genetic diagnosis of developmental disorders in the DDD study: a scalable analysis of genome-wide research dataThe Lancet, 2015