NCTDA: Nearest Neighbor Gaussian Process-Based Cell Type-Specific Spatially Variable Gene Detection Analysis
Authors:
- 10 July 2026
- journal article
- research article
- Published by SAGE Publications in Journal of Computational Biology
- Vol. 33 (10) , 929-942
Abstract
A primary task of spatial transcriptomics is detecting spatially variable genes (SVGs). Many genes may show spatially heterogeneous expression in specific cell types while showing spatial randomness across the whole tissue, thereby defining cell type-specific SVGs (ctSVGs). This study aims at detecting not only ctSVGs but also SVGs. Here, we construct a novel analysis framework NCTDA, which employs a nearest neighbor Gaussian process (NNGP) model to incorporate cell type composition into the spatial modeling of gene expression, is capable of linearly scaling with the number of spatial spots, unlike the cubic scalability of most methods. NCTDA performs hypothesis testing for different detection purposes, namely, obtaining SVGs by testing the variance components associated with overall spatial effects and obtaining ctSVGs by testing the coefficients associated with cell types. As a computationally scalable framework, NCTDA enables robust research of large-scale spatial transcriptomics data. Through simulation and real data applications, the results confirm the accuracy and efficiency of NCTDA, enabling it to characterize distinct cellular states and gene modules within structurally complex tissues comprising multiple cell types. It delivers more profound insights into the spatial expression characteristics of genes and cellular functional heterogeneity during tissue development processes and disease states.Keywords
Funding Information
- National Natural Science Foundation of China (11831015)
- National Natural Science Foundation of China (12271216)
- National Natural Science Foundation of China (92370131)
This publication has 42 references indexed in Scilit:
- Recent advances in spatially variable gene detection in spatial transcriptomicsComputational and Structural Biotechnology Journal, 2024
- Identification of spatially variable genes with graph cutsNature Communications, 2022
- Single-cell RNA sequencing depicts the local cell landscape in thyroid-associated ophthalmopathyCell Reports Medicine, 2022
- Reference-free cell type deconvolution of multi-cellular pixel-resolution spatially resolved transcriptomics dataNature Communications, 2022
- Spatial deconvolution of HER2-positive breast cancer delineates tumor-associated cell type interactionsNature Communications, 2021
- Single-Cell Map of Diverse Immune Phenotypes in the Breast Tumor MicroenvironmentCell, 2018
- SpatialDE: identification of spatially variable genesNature Methods, 2018
- BRISC: bootstrap for rapid inference on spatial covariancesStat, 2018
- In Situ Transcription Profiling of Single Cells Reveals Spatial Organization of Cells in the Mouse HippocampusNeuron, 2016
- Comparing partitionsJournal of Classification, 1985