Systema: a framework for evaluating genetic perturbation response prediction beyond systematic variation
Top Cited Papers
Open Access
- 25 August 2025
- journal article
- research article
- Published by Springer Nature in Nature Biotechnology
- Vol. 44 (6) , 1050-1059
Abstract
Predicting transcriptional responses to genetic perturbations is challenging in functional genomics. While recent methods aim to infer effects of untested perturbations, their true predictive power remains unclear. Here, we show that current methods struggle to generalize beyond systematic variation, the consistent transcriptional differences between perturbed and control cells arising from selection biases or confounders. We quantify this variation in ten datasets, spanning three technologies and five cell lines, and show that common metrics are susceptible to these biases, leading to overestimated performance. To address this, we introduce Systema, an evaluation framework that emphasizes perturbation-specific effects and identifies predictions that correctly reconstruct the perturbation landscape. Using this framework, we uncover insights into the predictive capabilities of existing methods and show that predicting responses to unseen perturbations is substantially harder than standard metrics suggest. Our work highlights the importance of heterogeneous gene panels and disentangles predictive performance from systematic effects, enabling biologically meaningful developments in perturbation response modeling.Keywords
Funding Information
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (TMSGI2_226252)
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (IC00I0-231922)
- Peter und Traudl Engelhorn Stiftung
- European Commission
This publication has 34 references indexed in Scilit:
- Dissecting key regulators of transcriptome kinetics through scalable single-cell RNA profiling of pooled CRISPR screensNature Biotechnology, 2023
- Predicting transcriptional outcomes of novel multigene perturbations with GEARSNature Biotechnology, 2023
- Mapping information-rich genotype-phenotype landscapes with genome-scale Perturb-seqCell, 2022
- Identifying Systematic Variation at the Single-Cell Level by Leveraging Low-Resolution Population-Level Data2022
- Spatial CRISPR genomics identifies regulators of the tumor microenvironmentCell, 2022
- Optical Pooled Screens in Human CellsCell, 2019
- CRISPR Interference-Based Platform for Multimodal Genetic Screens in Human iPSC-Derived NeuronsNeuron, 2019
- SCENIC: single-cell regulatory network inference and clusteringNature Methods, 2017
- A Multiplexed Single-Cell CRISPR Screening Platform Enables Systematic Dissection of the Unfolded Protein ResponseCell, 2016
- Gene Ontology: tool for the unification of biologyNature Genetics, 2000