Methods for evaluating unsupervised vector representations of genomic regions
Open Access
- 2 July 2024
- journal article
- research article
- Published by Oxford University Press (OUP) in NAR Genomics and Bioinformatics
- Vol. 6 (3) , lqae086
Abstract
Representation learning models have become a mainstay of modern genomics. These models are trained to yield vector representations, or embeddings, of various biological entities, such as cells, genes, individuals, or genomic regions. Recent applications of unsupervised embedding approaches have been shown to learn relationships among genomic regions that define functional elements in a genome. Unsupervised representation learning of genomic regions is free of the supervision from curated metadata and can condense rich biological knowledge from publicly available data to region embeddings. However, there exists no method for evaluating the quality of these embeddings in the absence of metadata, making it difficult to assess the reliability of analyses based on the embeddings, and to tune model training to yield optimal results. To bridge this gap, we propose four evaluation metrics: the cluster tendency score (CTS), the reconstruction score (RCS), the genome distance scaling score (GDSS), and the neighborhood preserving score (NPS). The CTS and RCS statistically quantify how well region embeddings can be clustered and how well the embeddings preserve information in training data. The GDSS and NPS exploit the biological tendency of regions close in genomic space to have similar biological functions; they measure how much such information is captured by individual region embeddings in a set. We demonstrate the utility of these statistical and biological scores for evaluating unsupervised genomic region embeddings and provide guidelines for learning reliable embeddings.Keywords
Funding Information
- National Institute of General Medical Sciences (R35-GM128636)
- National Human Genome Research Institute (R01-HG012558)
This publication has 22 references indexed in Scilit:
- GEOfetch: a command-line tool for downloading data and standardized metadata from GEO and SRABioinformatics, 2023
- Embeddings of genomic region sets capture rich biological associations in lower dimensionsBioinformatics, 2021
- Analytical Approaches for ATAC‐seq Data AnalysisCurrent Protocols in Human Genetics, 2020
- Non-coding mutations in the driver seatNature Reviews Genetics, 2014
- Transposition of native chromatin for fast and sensitive epigenomic profiling of open chromatin, DNA-binding proteins and nucleosome positionNature Methods, 2013
- ChIP–seq and beyond: new and improved methodologies to detect and characterize protein–DNA interactionsNature Reviews Genetics, 2012
- An integrated encyclopedia of DNA elements in the human genomeNature, 2012
- Epigenetic modifications and human diseaseNature Biotechnology, 2010
- Principal component analysisWIREs Computational Statistics, 2010
- Comprehensive genomic characterization defines human glioblastoma genes and core pathwaysNature, 2008