A Label-Free Multi-Metric Pipeline for Benchmarking Single-Cell RNA-Sequencing Clustering and Testing the Reproducibility of Cell-Type Heterogeneity
Authors:
Preprint
Abstract
A discovered sub-population from single-cell transcriptomic data is only meaningful if it is reproducible, yet clustering is usually done with one method on one embedding and rarely tested. We present a label- free, multi-metric pipeline that reframes clustering as an auditable, methods-blind decision and separates two notions of stability that are commonly conflated: reproducibility under cell resampling (bootstrap) and reproducibility under re-embedding (retraining the representation). The pipeline evaluates seven clustering configurations across cluster counts using five non-redundant quality metrics. As a whole- dataset control on a mouse retinal atlas, it recovers an eight-cell-type annotation at 96.3% accuracy (adjusted Rand index, ARI = 0.91) without labels. We then validate the discovery mode on two cell types with opposite ground truth. On bipolar cells, which have well-established subtypes, the pipeline accepts the sub-structure: across-embedding reproducibility rises with cluster number to a high plateau (mean pairwise ARI ∼0.93 near the ∼15 known bipolar subtypes), with quality metrics improving in parallel. On rod photoreceptors, treated as homogeneous, it rejects over-clustering: the metric-selected partition passes a bootstrap-stability check but is not reproducible when the embedding is retrained (mean pairwise ARI = 0.69), and the metrics do not improve with cluster number. On synthetic data, the test recovers real structure down to a 5% subpopulation while rejecting null data (high sensitivity and specificity). Bootstrap stability alone is therefore insufficient evidence for sub-population; the across-embedding test discriminates real sub-structure from over-clustering and applies to any cell type as a reproducible alternative to single-method, single-embedding clustering.Keywords
This publication has 28 references indexed in Scilit:
- Fast and precise single-cell data analysis using a hierarchical autoencoderNature Communications, 2021
- An entropy-based metric for assessing the purity of single cell populationsNature Communications, 2020
- Exponential scaling of single-cell RNA-seq in the past decadeNature Protocols, 2018
- Massively parallel digital transcriptional profiling of single cellsNature Communications, 2017
- Comprehensive Classification of Retinal Bipolar Neurons by Single-Cell TranscriptomicsCell, 2016
- Highly Parallel Genome-wide Expression Profiling of Individual Cells Using Nanoliter DropletsCell, 2015
- mRNA-Seq whole-transcriptome analysis of a single cellNature Methods, 2009
- Cluster-wise assessment of cluster stabilityComputational Statistics & Data Analysis, 2007
- Silhouettes: A graphical aid to the interpretation and validation of cluster analysisJournal of Computational and Applied Mathematics, 1987
- Comparing partitionsJournal of Classification, 1985