Pengtao Xie
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Pengtao Xie has a total of 280 co-authors. Here are the top co-authors by number of shared publications:
Ruiyi Zhang
Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, USA
17 shared publicationsLi Zhang
Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, USA
12 shared publications- Eric P. Xing
Carnegie Mellon University, USA
9 shared publications Han Guo
Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, USA
9 shared publicationsSai Ashish Somayajula
UC San Diego
9 shared publicationsDigvijay Singh
School of Biological Sciences, UC San Diego, La Jolla, CA 92093, USA
7 shared publications
Latest Publications
- 4 November 2026
- Published by Springer Nature in Nature Methods
- Vol. 23 (11) , 2239-2249
RobPicker: a meta-learning framework for robust identification of macromolecules in cryo-electron tomograms
- 4 November 2026
- Published by Springer Nature in Nature Methods
- Vol. 23 (11) , 2239-2249
RobPicker: a meta-learning framework for robust identification of macromolecules in cryo-electron tomograms
- Open Access
- 18 August 2026
- Published by MDPI AG in Preprints.org
The Landscape of Single-Cell Foundation Models: Design Principles, Applications, and Open Challenges
Recent advances in large-scale single-cell atlases and large language models (LLMs) have enabled the emergence of single-cell foundation models (scFMs)-pretrained models that aim to learn transferable representations of cellular states across tissues, conditions, and molecular modalities. These models hold promise for moving single-cell analysis beyond task-specific pipelines towards unified, generalizable and predictive frameworks, yet their biological fidelity, generalization and scaling capacity, as well as utility for perturbation modeling remain challenging. In this review, we summarize recent progress in scFMs across key design dimensions, including data representation, model architecture and learning objective, and examine how these frameworks are being applied to biological discovery. We further critically assess current challenges and opportunities from multimodal pretraining atlas, emerging pretraining paradigms, downstream applications, and real-world barriers. This review provides a structured guide for both computational scientists and experimental biologists seeking to understand, develop, and apply foundation models for single-cell research.
- 18 August 2026
- Published by MDPI AG in Preprints.org
Open AccessThe Landscape of Single-Cell Foundation Models: Design Principles, Applications, and Open Challenges
Recent advances in large-scale single-cell atlases and large language models (LLMs) have enabled the emergence of single-cell foundation models (scFMs)-pretrained models that aim to learn transferable representations of cellular states across tissues, conditions, and molecular modalities. These models hold promise for moving single-cell analysis beyond task-specific pipelines towards unified, generalizable and predictive frameworks, yet their biological fidelity, generalization and scaling capacity, as well as utility for perturbation modeling remain challenging. In this review, we summarize recent progress in scFMs across key design dimensions, including data representation, model architecture and learning objective, and examine how these frameworks are being applied to biological discovery. We further critically assess current challenges and opportunities from multimodal pretraining atlas, emerging pretraining paradigms, downstream applications, and real-world barriers. This review provides a structured guide for both computational scientists and experimental biologists seeking to understand, develop, and apply foundation models for single-cell research.
- Open Access
- 21 June 2026
- Published by Springer Nature in npj Artificial Intelligence
- Vol. 2 (1)
Interleaving multi-task neural architecture search
Multi-task neural architecture search (MTNAS), which searches for a shared architecture for multiple tasks, has been broadly investigated. In these methods, multiple tasks are learned simultaneously by minimizing the weighted sum of their losses. How to balance these losses by finding the optimal loss weights requires a lot of tuning, which is time-consuming and labor intensive. To address this problem, we propose an interleaving MTNAS framework, where no tuning of loss weights is needed. In our method, a set of tasks (e.g., A, B, C) are performed in an interleaving loop (e.g., ABCABCABC...) where each task transfers its knowledge to the next task. Each task is learned by minimizing its loss function alone, without intervening with losses of other tasks. Loss functions of individual tasks are organized into a multi-level optimization framework which enables all tasks performed end-to-end. The effectiveness of our method is demonstrated in a variety of experiments.
- 21 June 2026
- Published by Springer Nature in npj Artificial Intelligence
- Vol. 2 (1)
Open AccessInterleaving multi-task neural architecture search
Multi-task neural architecture search (MTNAS), which searches for a shared architecture for multiple tasks, has been broadly investigated. In these methods, multiple tasks are learned simultaneously by minimizing the weighted sum of their losses. How to balance these losses by finding the optimal loss weights requires a lot of tuning, which is time-consuming and labor intensive. To address this problem, we propose an interleaving MTNAS framework, where no tuning of loss weights is needed. In our method, a set of tasks (e.g., A, B, C) are performed in an interleaving loop (e.g., ABCABCABC...) where each task transfers its knowledge to the next task. Each task is learned by minimizing its loss function alone, without intervening with losses of other tasks. Loss functions of individual tasks are organized into a multi-level optimization framework which enables all tasks performed end-to-end. The effectiveness of our method is demonstrated in a variety of experiments.
- Open Access
- 15 April 2026
- Published by Elsevier in Cell Reports Methods
- Vol. 6 (5) , 101407
A tri-modal contrastive learning framework for protein representation learning
- 15 April 2026
- Published by Elsevier in Cell Reports Methods
- Vol. 6 (5) , 101407
Open AccessA tri-modal contrastive learning framework for protein representation learning
- 5 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Artificial Intelligence
- Vol. PP (9) , 1-14
Improving Neural Architecture Search by Minimizing Worst-Case Validation Loss
Neural architecture search (NAS) aims at automatically searching for high-performance architectures and has achieved considerable progress. Existing NAS methods learn architectures by minimizing average-case validation losses. As a result, the searched architectures are less capable of making correct predictions under worst-case scenarios. To address this problem, we propose a framework which leverages a deep generative model to generate adversarial validation examples to measure the worst-case validation performance of an architecture and improves the architecture by minimizing the loss on the generated adversarial validation data. Our framework is based on multi-level optimization, which performs multiple learning stages end-to-end. Experiments on a variety of datasets demonstrate the effectiveness of our method.
- 5 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Artificial Intelligence
- Vol. PP (9) , 1-14
Improving Neural Architecture Search by Minimizing Worst-Case Validation Loss
Neural architecture search (NAS) aims at automatically searching for high-performance architectures and has achieved considerable progress. Existing NAS methods learn architectures by minimizing average-case validation losses. As a result, the searched architectures are less capable of making correct predictions under worst-case scenarios. To address this problem, we propose a framework which leverages a deep generative model to generate adversarial validation examples to measure the worst-case validation performance of an architecture and improves the architecture by minimizing the loss on the generated adversarial validation data. Our framework is based on multi-level optimization, which performs multiple learning stages end-to-end. Experiments on a variety of datasets demonstrate the effectiveness of our method.
- Top Cited PapersOpen Access
- 5 February 2026
- Published by Springer Nature in Nature Communications
- Vol. 17 (1)
scLong: a billion-parameter foundation model for capturing long-range gene context in single-cell transcriptomics
Single-cell RNA sequencing (scRNA-seq) has revolutionized the study of cellular heterogeneity by providing gene expression data at single-cell resolution, uncovering insights into rare cell populations, cell-cell interactions, and gene regulation. Foundation models pretrained on large-scale scRNA-seq datasets have shown great promise in analyzing such data, but existing approaches are often limited to modeling a small subset of highly expressed genes and lack the integration of external gene-specific knowledge. To address these limitations, we present scLong, a billion-parameter foundation model pretrained on 48 million cells. scLong performs self-attention across the entire set of 28,000 genes in the human genome. This enables the model to capture long-range dependencies between all genes, including lowly expressed ones (containing unexpressed genes with zero expressions), which often play critical roles in cellular processes but are typically excluded by existing foundation models. Additionally, scLong integrates gene knowledge from the Gene Ontology using a graph convolutional network, enriching its contextual understanding of gene functions and relationships. In extensive evaluations, scLong surpasses both state-of-the-art scRNA-seq foundation models and task-specific models across diverse tasks, including predicting transcriptional responses to genetic and chemical perturbations, forecasting cancer drug responses, and inferring gene regulatory networks.
- 5 February 2026
- Published by Springer Nature in Nature Communications
- Vol. 17 (1)
Top Cited PapersOpen AccessscLong: a billion-parameter foundation model for capturing long-range gene context in single-cell transcriptomics
Single-cell RNA sequencing (scRNA-seq) has revolutionized the study of cellular heterogeneity by providing gene expression data at single-cell resolution, uncovering insights into rare cell populations, cell-cell interactions, and gene regulation. Foundation models pretrained on large-scale scRNA-seq datasets have shown great promise in analyzing such data, but existing approaches are often limited to modeling a small subset of highly expressed genes and lack the integration of external gene-specific knowledge. To address these limitations, we present scLong, a billion-parameter foundation model pretrained on 48 million cells. scLong performs self-attention across the entire set of 28,000 genes in the human genome. This enables the model to capture long-range dependencies between all genes, including lowly expressed ones (containing unexpressed genes with zero expressions), which often play critical roles in cellular processes but are typically excluded by existing foundation models. Additionally, scLong integrates gene knowledge from the Gene Ontology using a graph convolutional network, enriching its contextual understanding of gene functions and relationships. In extensive evaluations, scLong surpasses both state-of-the-art scRNA-seq foundation models and task-specific models across diverse tasks, including predicting transcriptional responses to genetic and chemical perturbations, forecasting cancer drug responses, and inferring gene regulatory networks.
- Open Access
DAJ: Data-Reweighted LLM Judge for Test-Time Scaling in Code Generation
Test-time scaling for code generation commonly relies on Best-of-N selection, in which multiple candidate solutions are sampled from a base model, and the best one is selected by an LLM judge. However, training reliable LLM judges is challenging due to severe distribution shifts, including imbalances between easy and hard problems, mismatches between training tasks and evaluation benchmarks, and trajectory mismatch arising from training data generated by cheaper models whose behavior differs from that of inference-time models. We propose DAJ, a reasoning-based LLM judge trained with verifiable rewards under a bi-level data-reweighted learning framework. The proposed framework learns data-importance weights (either domain-level or instance-level) to optimize generalization performance on a held-out meta set aligned with target benchmarks. To the best of our knowledge, this is the first application of data reweighting to LLM-as-a-Judge training for test-time scaling. Our approach automatically emphasizes hard problems, in-distribution samples, and trajectory-aligned data, without relying on hand-crafted heuristics. Empirically, DAJ achieves state-of-the-art performance on LiveCodeBench and BigCodeBench, outperforming strong test-time scaling baselines as well as leading proprietary models.
Open AccessDAJ: Data-Reweighted LLM Judge for Test-Time Scaling in Code Generation
Test-time scaling for code generation commonly relies on Best-of-N selection, in which multiple candidate solutions are sampled from a base model, and the best one is selected by an LLM judge. However, training reliable LLM judges is challenging due to severe distribution shifts, including imbalances between easy and hard problems, mismatches between training tasks and evaluation benchmarks, and trajectory mismatch arising from training data generated by cheaper models whose behavior differs from that of inference-time models. We propose DAJ, a reasoning-based LLM judge trained with verifiable rewards under a bi-level data-reweighted learning framework. The proposed framework learns data-importance weights (either domain-level or instance-level) to optimize generalization performance on a held-out meta set aligned with target benchmarks. To the best of our knowledge, this is the first application of data reweighting to LLM-as-a-Judge training for test-time scaling. Our approach automatically emphasizes hard problems, in-distribution samples, and trajectory-aligned data, without relying on hand-crafted heuristics. Empirically, DAJ achieves state-of-the-art performance on LiveCodeBench and BigCodeBench, outperforming strong test-time scaling baselines as well as leading proprietary models.
- Open Access
FunPRM: Function-as-Step Process Reward Model with Meta Reward Correction for Code Generation
Code generation is a core application of large language models (LLMs), yet LLMs still frequently fail on complex programming tasks. Given its success in mathematical reasoning, test-time scaling approaches such as Process Reward Model (PRM)-based Best-of-N selection offer a promising way to improve performance. However, existing PRMs remain ineffective for code generation due to the lack of meaningful step decomposition in code and the noise of Monte Carlo-estimated partial-solution correctness scores (rewards). To address these challenges, we propose FunPRM. FunPRM prompts LLMs to encourage modular code generation organized into functions, with functions treated as PRM reasoning steps. Furthermore, FunPRM introduces a novel meta-learning-based reward correction mechanism that leverages clean final-solution rewards obtained via a unit-test-based evaluation system to purify noisy partial-solution rewards. Experiments on LiveCodeBench and BigCodeBench demonstrate that FunPRM consistently outperforms existing test-time scaling methods across five base LLMs, notably achieving state-of-the-art performance on LiveCodeBench when combined with O4-mini. Furthermore, FunPRM produces code that is more readable and reusable for developers.
Open AccessFunPRM: Function-as-Step Process Reward Model with Meta Reward Correction for Code Generation
Code generation is a core application of large language models (LLMs), yet LLMs still frequently fail on complex programming tasks. Given its success in mathematical reasoning, test-time scaling approaches such as Process Reward Model (PRM)-based Best-of-N selection offer a promising way to improve performance. However, existing PRMs remain ineffective for code generation due to the lack of meaningful step decomposition in code and the noise of Monte Carlo-estimated partial-solution correctness scores (rewards). To address these challenges, we propose FunPRM. FunPRM prompts LLMs to encourage modular code generation organized into functions, with functions treated as PRM reasoning steps. Furthermore, FunPRM introduces a novel meta-learning-based reward correction mechanism that leverages clean final-solution rewards obtained via a unit-test-based evaluation system to purify noisy partial-solution rewards. Experiments on LiveCodeBench and BigCodeBench demonstrate that FunPRM consistently outperforms existing test-time scaling methods across five base LLMs, notably achieving state-of-the-art performance on LiveCodeBench when combined with O4-mini. Furthermore, FunPRM produces code that is more readable and reusable for developers.
- Open Access
BLO-Inst: Bi-Level Optimization Based Alignment of YOLO and SAM for Robust Instance Segmentation
The Segment Anything Model has revolutionized image segmentation with its zero-shot capabilities, yet its reliance on manual prompts hinders fully automated deployment. While integrating object detectors as prompt generators offers a pathway to automation, existing pipelines suffer from two fundamental limitations: objective mismatch, where detectors optimized for geometric localization do not correspond to the optimal prompting context required by SAM, and alignment overfitting in standard joint training, where the detector simply memorizes specific prompt adjustments for training samples rather than learning a generalizable policy. To bridge this gap, we introduce BLO-Inst, a unified framework that aligns detection and segmentation objectives by bi-level optimization. We formulate the alignment as a nested optimization problem over disjoint data splits. In the lower level, the SAM is fine-tuned to maximize segmentation fidelity given the current detection proposals on a subset ($D_1$). In the upper level, the detector is updated to generate bounding boxes that explicitly minimize the validation loss of the fine-tuned SAM on a separate subset ($D_2$). This effectively transforms the detector into a segmentation-aware prompt generator, optimizing the bounding boxes not just for localization accuracy, but for downstream mask quality. Extensive experiments demonstrate that BLO-Inst achieves superior performance, outperforming standard baselines on tasks in general and biomedical domains.
Open AccessBLO-Inst: Bi-Level Optimization Based Alignment of YOLO and SAM for Robust Instance Segmentation
The Segment Anything Model has revolutionized image segmentation with its zero-shot capabilities, yet its reliance on manual prompts hinders fully automated deployment. While integrating object detectors as prompt generators offers a pathway to automation, existing pipelines suffer from two fundamental limitations: objective mismatch, where detectors optimized for geometric localization do not correspond to the optimal prompting context required by SAM, and alignment overfitting in standard joint training, where the detector simply memorizes specific prompt adjustments for training samples rather than learning a generalizable policy. To bridge this gap, we introduce BLO-Inst, a unified framework that aligns detection and segmentation objectives by bi-level optimization. We formulate the alignment as a nested optimization problem over disjoint data splits. In the lower level, the SAM is fine-tuned to maximize segmentation fidelity given the current detection proposals on a subset ($D_1$). In the upper level, the detector is updated to generate bounding boxes that explicitly minimize the validation loss of the fine-tuned SAM on a separate subset ($D_2$). This effectively transforms the detector into a segmentation-aware prompt generator, optimizing the bounding boxes not just for localization accuracy, but for downstream mask quality. Extensive experiments demonstrate that BLO-Inst achieves superior performance, outperforming standard baselines on tasks in general and biomedical domains.
- 17 January 2026
- Published by Springer Nature in International Journal of Machine Learning and Cybernetics
- Vol. 17 (1) , 35
Am-sam: a spatially-aware prompt learning and mask calibration framework for few-shot semantic segmentation
- 17 January 2026
- Published by Springer Nature in International Journal of Machine Learning and Cybernetics
- Vol. 17 (1) , 35
Am-sam: a spatially-aware prompt learning and mask calibration framework for few-shot semantic segmentation