scLong: a billion-parameter foundation model for capturing long-range gene context in single-cell transcriptomics
Top Cited Papers
Open Access
- 5 February 2026
- journal article
- research article
- Published by Springer Nature in Nature Communications
- Vol. 17 (1)
Abstract
Single-cell RNA sequencing (scRNA-seq) has revolutionized the study of cellular heterogeneity by providing gene expression data at single-cell resolution, uncovering insights into rare cell populations, cell-cell interactions, and gene regulation. Foundation models pretrained on large-scale scRNA-seq datasets have shown great promise in analyzing such data, but existing approaches are often limited to modeling a small subset of highly expressed genes and lack the integration of external gene-specific knowledge. To address these limitations, we present scLong, a billion-parameter foundation model pretrained on 48 million cells. scLong performs self-attention across the entire set of 28,000 genes in the human genome. This enables the model to capture long-range dependencies between all genes, including lowly expressed ones (containing unexpressed genes with zero expressions), which often play critical roles in cellular processes but are typically excluded by existing foundation models. Additionally, scLong integrates gene knowledge from the Gene Ontology using a graph convolutional network, enriching its contextual understanding of gene functions and relationships. In extensive evaluations, scLong surpasses both state-of-the-art scRNA-seq foundation models and task-specific models across diverse tasks, including predicting transcriptional responses to genetic and chemical perturbations, forecasting cancer drug responses, and inferring gene regulatory networks.Keywords
All Related Versions
This publication has 71 references indexed in Scilit:
- Towards a Human Cell Atlas: Taking Notes from the PastTrends in Genetics, 2021
- Predicting Drug Response and Synergy Using a Deep Learning Model of Human Cancer CellsCancer Cell, 2020
- A Genome-wide Framework for Mapping Gene Regulation via Cellular Genetic ScreensCell, 2019
- Lowly-expressed lncRNA GAS5 facilitates progression of ovarian cancer through targeting miR-196-5p and thereby regulating HOXA5Gynecologic Oncology, 2018
- A Next Generation Connectivity Map: L1000 Platform and the First 1,000,000 ProfilesCell, 2017
- Perturb-Seq: Dissecting Molecular Circuits with Scalable Single-Cell RNA Profiling of Pooled Genetic ScreensCell, 2016
- A Landscape of Pharmacogenomic Interactions in CancerCell, 2016
- Prediction of individual response to anticancer therapy: historical and future perspectivesCellular and Molecular Life Sciences, 2014
- High-Resolution Profiling of Histone Methylations in the Human GenomeCell, 2007
- Gene Ontology: tool for the unification of biologyNature Genetics, 2000