STEAM: Spatial Transcriptomics Evaluation Algorithm and Metric for clustering performance
Authors:
Preprint
Abstract
Motivation Spatial transcriptomic technologies allow researchers to explore the diversity and specificity of gene expression within their original tissue structure. Accurately identifying regions that are spatially coherent in both gene expression and physical tissue structures is an emerging topic, but challenging due to the lack of ground truth labels which renders complicating validation of clustering consistency and reproducibility. This highlights a need for a computational evaluation framework to rigorously and unbiasedly assess clustering performance. Results To address this gap, we propose STEAM (Spatial Transcriptomics Evaluation Algorithm and Metric), a user-friendly computational pipeline designed to evaluate the consistency and reliability of clustering results by leveraging machine learning classification and prediction methods, with the goal of maintaining the spatial proximity and gene expression patterns within clusters. We benchmarked STEAM on various public datasets, spanning multi-cell to single-cell resolution, as well as spatial transcriptomics and proteomics. The results highlighted its robustness and generalizability through comprehensive statistical evaluation metrics, such as Kappa score, F1 score, accuracy, and adjusted rand index. Notably, STEAM supports multi-sample training, enabling cross-replicate clustering consistency assessment. Moreover, STEAM provides practical guidance by comparing clustering results across multiple approaches; here, we evaluated four different methods, including spatial-aware and spatial-ignorant approaches. In summary, we believe that STEAM provides researchers a promising tool for evaluating clustering robustness and benchmarking clustering performance for spatial omics data, offering valuable insights to drive reproducible discoveries in spatial biology. Availability and implementation Source code and the R software tool STEAM are available from https://github.com/fanzhanglab/STEAM.Keywords
All Related Versions
- Published version: Briefings in Bioinformatics, 26 (5).
This publication has 22 references indexed in Scilit:
- A high-resolution transcriptomic and spatial atlas of cell types in the whole mouse brainNature, 2023
- Integration of spatial and single-cell data across modalities with weakly linked featuresNature Biotechnology, 2023
- Organization of the human intestine at single-cell resolutionNature, 2023
- Spatially aware dimension reduction for spatial transcriptomicsNature Communications, 2022
- Author Correction: Visualizing group II intron dynamics between the first and second steps of splicingNature Communications, 2022
- Spatial transcriptomics at subspot resolution with BayesSpaceNature Biotechnology, 2021
- Robust decomposition of cell type mixtures in spatial transcriptomicsNature Biotechnology, 2021
- Spatially resolved transcriptomics adds a new dimension to genomicsNature Methods, 2021
- Fast, sensitive and accurate integration of single-cell data with HarmonyNature Methods, 2019
- Dimensionality reduction for visualizing single-cell data using UMAPNature Biotechnology, 2018