Pei Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Pei Zhang has a total of 101 co-authors. Here are the top co-authors by number of shared publications:
- Massimiliano Lupo Pasini
Computational Sciences and Engineering Division, Oak Ridge National Laboratory, Oak Ridge, USA
9 shared publications - Jong Youl Choi
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
7 shared publications - Haifeng Wang
School of Aeronautics and Astronautics, Purdue University, West Lafayette, USA
6 shared publications - Kshitij Mehta
Computer Science and Mathematics Division, Oak Ridge National Laboratory, Oak Ridge, USA
6 shared publications - Jong Youl Choi
Oak Ridge National Laboratory
4 shared publications - M. Paul Laiu
Oak Ridge National Laboratory
4 shared publications
Latest Publications
- Open Access
- 15 May 2026
- Published by IOP Publishing in Machine Learning: Science and Technology
- Vol. 7 (3) , 035016
MATEY: multiscale adaptive transformer models for spatiotemporal physical systems *
Accurate representation of the multiscale features in spatiotemporal physical systems using vision transformer architectures requires extremely long, computationally prohibitive token sequences. To address this issue, we propose two novel adaptive tokenization schemes that dynamically adjust patch sizes based on local features: one ensures convergent behavior to uniform patch refinement, while the other offers better computational efficiency. Moreover, we present a set of spatiotemporal attention schemes, where the temporal or axial spatial dimensions are decoupled, to evaluate their baseline computational and data efficiencies and to determine whether adaptive tokenization can improve this performance. We assess the performance of the proposed multiscale adaptive model, MATEY, in a sequence of experiments. Compared to a full spatiotemporal attention scheme or a scheme that decouples only the temporal dimension, we find that fully decoupled axial attention is less efficient and expressive, requiring more training time and model parameters to achieve the same accuracy. The experiments on the adaptive tokenization schemes show that, compared to a uniformly refined model, the proposed schemes achieve comparable or improved accuracy at a much lower cost in the tested two-dimensional settings. While the asymptotic analysis suggests the potential for favorable scaling, empirical validation at substantially longer sequence lengths remains to be performed in future work. Finally, we demonstrate in two fine-tuning tasks featuring different physics that models pretrained on PDEBench data outperform the ones trained from scratch, especially in the low data regime with frozen attention.
- 15 May 2026
- Published by IOP Publishing in Machine Learning: Science and Technology
- Vol. 7 (3) , 035016
Open AccessMATEY: multiscale adaptive transformer models for spatiotemporal physical systems *
Accurate representation of the multiscale features in spatiotemporal physical systems using vision transformer architectures requires extremely long, computationally prohibitive token sequences. To address this issue, we propose two novel adaptive tokenization schemes that dynamically adjust patch sizes based on local features: one ensures convergent behavior to uniform patch refinement, while the other offers better computational efficiency. Moreover, we present a set of spatiotemporal attention schemes, where the temporal or axial spatial dimensions are decoupled, to evaluate their baseline computational and data efficiencies and to determine whether adaptive tokenization can improve this performance. We assess the performance of the proposed multiscale adaptive model, MATEY, in a sequence of experiments. Compared to a full spatiotemporal attention scheme or a scheme that decouples only the temporal dimension, we find that fully decoupled axial attention is less efficient and expressive, requiring more training time and model parameters to achieve the same accuracy. The experiments on the adaptive tokenization schemes show that, compared to a uniformly refined model, the proposed schemes achieve comparable or improved accuracy at a much lower cost in the tested two-dimensional settings. While the asymptotic analysis suggests the potential for favorable scaling, empirical validation at substantially longer sequence lengths remains to be performed in future work. Finally, we demonstrate in two fine-tuning tasks featuring different physics that models pretrained on PDEBench data outperform the ones trained from scratch, especially in the low data regime with frozen attention.
- Open Access
- 14 May 2026
- Published by IOP Publishing in Nuclear Fusion
- Vol. 66 (6) , 066013
Autoregressive long-horizon prediction of plasma edge dynamics *
Accurate modeling of scrape-off layer (SOL) and divertor-edge dynamics is vital for designing plasma-facing components in fusion devices. High-fidelity edge fluid/neutral codes such as SOLPS-ITER capture SOL physics with high accuracy, but their computational cost limits broad parameter scans and long transient studies. We present transformer-based, autoregressive surrogates for efficient prediction of 2D, time-dependent plasma edge state fields. Trained on SOLPS-ITER spatiotemporal data for the KSTAR tokamak, the surrogates forecast electron temperature, electron density, and radiated power over extended horizons. We evaluate model variants trained with increasing autoregressive horizons (1–100 steps) on short- and long-horizon prediction tasks. Longer-horizon training systematically improves rollout stability and mitigates error accumulation, enabling stable predictions over hundreds to thousands of steps and reproducing key dynamical features such as the motion of high-radiation regions. Measured end-to-end wall-clock times show the surrogate is orders of magnitude faster than SOLPS-ITER, enabling rapid parameter exploration. Prediction accuracy degrades when the surrogate enters physical regimes not represented in the training dataset, motivating future work on data enrichment and physics-informed constraints. Overall, this approach provides a fast, accurate surrogate for computationally intensive plasma edge simulations, supporting rapid scenario exploration, control-oriented studies, and progress toward real-time applications in fusion devices.
- 14 May 2026
- Published by IOP Publishing in Nuclear Fusion
- Vol. 66 (6) , 066013
Open AccessAutoregressive long-horizon prediction of plasma edge dynamics *
Accurate modeling of scrape-off layer (SOL) and divertor-edge dynamics is vital for designing plasma-facing components in fusion devices. High-fidelity edge fluid/neutral codes such as SOLPS-ITER capture SOL physics with high accuracy, but their computational cost limits broad parameter scans and long transient studies. We present transformer-based, autoregressive surrogates for efficient prediction of 2D, time-dependent plasma edge state fields. Trained on SOLPS-ITER spatiotemporal data for the KSTAR tokamak, the surrogates forecast electron temperature, electron density, and radiated power over extended horizons. We evaluate model variants trained with increasing autoregressive horizons (1–100 steps) on short- and long-horizon prediction tasks. Longer-horizon training systematically improves rollout stability and mitigates error accumulation, enabling stable predictions over hundreds to thousands of steps and reproducing key dynamical features such as the motion of high-radiation regions. Measured end-to-end wall-clock times show the surrogate is orders of magnitude faster than SOLPS-ITER, enabling rapid parameter exploration. Prediction accuracy degrades when the surrogate enters physical regimes not represented in the training dataset, motivating future work on data enrichment and physics-informed constraints. Overall, this approach provides a fast, accurate surrogate for computationally intensive plasma edge simulations, supporting rapid scenario exploration, control-oriented studies, and progress toward real-time applications in fusion devices.
- 1 May 2026
- Published by Office of Scientific and Technical Information (OSTI)
User Manual - HydraGNN v5.0: Distributed Implementation of Multi-Tasking Graph Neural Networks
This document serves as the user manual for HydraGNN v5.0, a scalable graph neural network (GNN) architecture for simultaneous prediction of multiple target properties using multi-task learning (MTL). This version of HydraGNN has been developed primarily to support the development, training, and deployment of predictive graph-based deep learning (DL) models for atomistic materials modeling. HydraGNN is templated over 13 message-passing policies, including invariant models (GIN, PNA, PNAPlus, GAT, MFC, CGCNN, SAGE, SchNet, DimeNet) and equivariant models (EGNN, PNAEq, PAINN, MACE), and supports distributed training via distributed data parallelism (DDP), DeepSpeed, and Fully Sharded Data Parallelism (FSDP) on leadership-class supercomputers. Although HydraGNN can be applied to problems beyond atomistic materials modeling, its current use is confined to homogeneous graphs. Additional capabilities include machine-learned interatomic potentials with energy-conserving forces, General, Powerful, and Scalable Graph Transformer (GraphGPS) global attention, periodic boundary conditions, hyperparameter optimization, mixed-precision training, and uncertainty quantification.
- 1 May 2026
- Published by Office of Scientific and Technical Information (OSTI)
User Manual - HydraGNN v5.0: Distributed Implementation of Multi-Tasking Graph Neural Networks
This document serves as the user manual for HydraGNN v5.0, a scalable graph neural network (GNN) architecture for simultaneous prediction of multiple target properties using multi-task learning (MTL). This version of HydraGNN has been developed primarily to support the development, training, and deployment of predictive graph-based deep learning (DL) models for atomistic materials modeling. HydraGNN is templated over 13 message-passing policies, including invariant models (GIN, PNA, PNAPlus, GAT, MFC, CGCNN, SAGE, SchNet, DimeNet) and equivariant models (EGNN, PNAEq, PAINN, MACE), and supports distributed training via distributed data parallelism (DDP), DeepSpeed, and Fully Sharded Data Parallelism (FSDP) on leadership-class supercomputers. Although HydraGNN can be applied to problems beyond atomistic materials modeling, its current use is confined to homogeneous graphs. Additional capabilities include machine-learned interatomic potentials with energy-conserving forces, General, Powerful, and Scalable Graph Transformer (GraphGPS) global attention, periodic boundary conditions, hyperparameter optimization, mixed-precision training, and uncertainty quantification.
- Open Access
Autoregressive long-horizon prediction of plasma edge dynamics
Accurate modeling of scrape-off layer (SOL) and divertor-edge dynamics is vital for designing plasma-facing components in fusion devices. High-fidelity edge fluid/neutral codes such as SOLPS-ITER capture SOL physics with high accuracy, but their computational cost limits broad parameter scans and long transient studies. We present transformer-based, autoregressive surrogates for efficient prediction of 2D, time-dependent plasma edge state fields. Trained on SOLPS-ITER spatiotemporal data, the surrogates forecast electron temperature, electron density, and radiated power over extended horizons. We evaluate model variants trained with increasing autoregressive horizons (1-100 steps) on short- and long-horizon prediction tasks. Longer-horizon training systematically improves rollout stability and mitigates error accumulation, enabling stable predictions over hundreds to thousands of steps and reproducing key dynamical features such as the motion of high-radiation regions. Measured end-to-end wall-clock times show the surrogate is orders of magnitude faster than SOLPS-ITER, enabling rapid parameter exploration. Prediction accuracy degrades when the surrogate enters physical regimes not represented in the training dataset, motivating future work on data enrichment and physics-informed constraints. Overall, this approach provides a fast, accurate surrogate for computationally intensive plasma edge simulations, supporting rapid scenario exploration, control-oriented studies, and progress toward real-time applications in fusion devices.
Open AccessAutoregressive long-horizon prediction of plasma edge dynamics
Accurate modeling of scrape-off layer (SOL) and divertor-edge dynamics is vital for designing plasma-facing components in fusion devices. High-fidelity edge fluid/neutral codes such as SOLPS-ITER capture SOL physics with high accuracy, but their computational cost limits broad parameter scans and long transient studies. We present transformer-based, autoregressive surrogates for efficient prediction of 2D, time-dependent plasma edge state fields. Trained on SOLPS-ITER spatiotemporal data, the surrogates forecast electron temperature, electron density, and radiated power over extended horizons. We evaluate model variants trained with increasing autoregressive horizons (1-100 steps) on short- and long-horizon prediction tasks. Longer-horizon training systematically improves rollout stability and mitigates error accumulation, enabling stable predictions over hundreds to thousands of steps and reproducing key dynamical features such as the motion of high-radiation regions. Measured end-to-end wall-clock times show the surrogate is orders of magnitude faster than SOLPS-ITER, enabling rapid parameter exploration. Prediction accuracy degrades when the surrogate enters physical regimes not represented in the training dataset, motivating future work on data enrichment and physics-informed constraints. Overall, this approach provides a fast, accurate surrogate for computationally intensive plasma edge simulations, supporting rapid scenario exploration, control-oriented studies, and progress toward real-time applications in fusion devices.
- Open Access
- 1 December 2025
- Published by Office of Scientific and Technical Information (OSTI)
Phase-field predictions of the influence of cooling rates during AM on the Evolution of Microstructures in Nickel-Based Single Crystal Superalloys
Additive manufacturing (AM) of single crystals (SX) made of Ni-based superalloys offers major cost savings for gas turbine engines with the inclusion of internal cooling channels. However, the lack of understanding of the effect of transient thermal conditions on solidification grain structure during AM hinders the potential for process control to maintain the SX quality. The use of HPC high-fidelity simulations to predict the evolution of the solidification microstructure will enhance the abilities to tailor the microstructures through process optimization. Phase field (PF) simulations are used to determine the effect local thermal conditions and defects on the stability of the solidification morphology, specifically with respect to the onset of columnar-to-equiaxed transition (CET) that results in the loss of the SX. The results are expected to be instrumental for developing future surrogate models to speed up the integration of design and manufacturing of turbine blades under the harsh in-service conditions.
- 1 December 2025
- Published by Office of Scientific and Technical Information (OSTI)
Open AccessPhase-field predictions of the influence of cooling rates during AM on the Evolution of Microstructures in Nickel-Based Single Crystal Superalloys
Additive manufacturing (AM) of single crystals (SX) made of Ni-based superalloys offers major cost savings for gas turbine engines with the inclusion of internal cooling channels. However, the lack of understanding of the effect of transient thermal conditions on solidification grain structure during AM hinders the potential for process control to maintain the SX quality. The use of HPC high-fidelity simulations to predict the evolution of the solidification microstructure will enhance the abilities to tailor the microstructures through process optimization. Phase field (PF) simulations are used to determine the effect local thermal conditions and defects on the stability of the solidification morphology, specifically with respect to the onset of columnar-to-equiaxed transition (CET) that results in the loss of the SX. The results are expected to be instrumental for developing future surrogate models to speed up the integration of design and manufacturing of turbine blades under the harsh in-service conditions.
- Open Access
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-10
Intelligent Sampling of Extreme-Scale Turbulence Datasets for Accurate and Efficient Spatiotemporal Model Training
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-10
Open AccessIntelligent Sampling of Extreme-Scale Turbulence Datasets for Accurate and Efficient Spatiotemporal Model Training
- 24 September 2025
- Published by Wiley in Journal of the American Ceramic Society
- Vol. 109 (1)
Modeling MTS pyrolysis and SiC deposition kinetics using principal component analysis and neural networks
Accurate chemical kinetics modeling is crucial for improving the efficiency of chemical processing and synthesis of ceramic matrix composites. Detailed kinetic models are computationally expensive due to the large number of transported chemical species, while the simplified physics‐based models, such as single‐step global mechanisms, are efficient but often overlook key chemical intermediates and pathways. Recent deep learning approaches promise accurate and cost‐effective models. Yet, they require additional closures for the transported nonlinear latent variables, complicating integration with existing solvers. In this work, we develop a hybrid linear—nonlinear reduced model for silicon carbide deposition from methyltrichlorosilane precursor by combining principal component analysis (PCA) and autoencoder (AE) neural network (NN) approaches. PCA is used to identify a smaller set of linear transport variables, enabling direct reuse of conventional transport solvers. NNs then reconstruct the full chemical state from these reduced variables. We demonstrate the method on a chemical vapor deposition reactor—comprising a gas‐phase pyrolysis plug flow reactor and a heterogeneous surface reactor—over a wide range of temperatures, pressures, and residence times. Our PCA–AE model achieves high accuracy with only five transported scalars, achieving an eightfold cost reduction compared to detailed mechanisms, in both a priori (using data from the test set only) and a posteriori (coupled with a differential equation solver). Notable errors arise primarily near training domain boundaries and for long residence times, indicating the need for domain shift indicators and better long‐horizon predictions in future reduced chemistry model development.
- 24 September 2025
- Published by Wiley in Journal of the American Ceramic Society
- Vol. 109 (1)
Modeling MTS pyrolysis and SiC deposition kinetics using principal component analysis and neural networks
Accurate chemical kinetics modeling is crucial for improving the efficiency of chemical processing and synthesis of ceramic matrix composites. Detailed kinetic models are computationally expensive due to the large number of transported chemical species, while the simplified physics‐based models, such as single‐step global mechanisms, are efficient but often overlook key chemical intermediates and pathways. Recent deep learning approaches promise accurate and cost‐effective models. Yet, they require additional closures for the transported nonlinear latent variables, complicating integration with existing solvers. In this work, we develop a hybrid linear—nonlinear reduced model for silicon carbide deposition from methyltrichlorosilane precursor by combining principal component analysis (PCA) and autoencoder (AE) neural network (NN) approaches. PCA is used to identify a smaller set of linear transport variables, enabling direct reuse of conventional transport solvers. NNs then reconstruct the full chemical state from these reduced variables. We demonstrate the method on a chemical vapor deposition reactor—comprising a gas‐phase pyrolysis plug flow reactor and a heterogeneous surface reactor—over a wide range of temperatures, pressures, and residence times. Our PCA–AE model achieves high accuracy with only five transported scalars, achieving an eightfold cost reduction compared to detailed mechanisms, in both a priori (using data from the test set only) and a posteriori (coupled with a differential equation solver). Notable errors arise primarily near training domain boundaries and for long residence times, indicating the need for domain shift indicators and better long‐horizon predictions in future reduced chemistry model development.
- Open Access
Pixel-Resolved Long-Context Learning for Turbulence at Exascale: Resolving Small-scale Eddies Toward the Viscous Limit
Turbulence plays a crucial role in multiphysics applications, including aerodynamics, fusion, and combustion. Accurately capturing turbulence's multiscale characteristics is essential for reliable predictions of multiphysics interactions, but remains a grand challenge even for exascale supercomputers and advanced deep learning models. The extreme-resolution data required to represent turbulence, ranging from billions to trillions of grid points, pose prohibitive computational costs for models based on architectures like vision transformers. To address this challenge, we introduce a multiscale hierarchical Turbulence Transformer that reduces sequence length from billions to a few millions and a novel RingX sequence parallelism approach that enables scalable long-context learning. We perform scaling and science runs on the Frontier supercomputer. Our approach demonstrates excellent performance up to 1.1 EFLOPS on 32,768 AMD GPUs, with a scaling efficiency of 94%. To our knowledge, this is the first AI model for turbulence that can capture small-scale eddies down to the dissipative range.
Open AccessPixel-Resolved Long-Context Learning for Turbulence at Exascale: Resolving Small-scale Eddies Toward the Viscous Limit
Turbulence plays a crucial role in multiphysics applications, including aerodynamics, fusion, and combustion. Accurately capturing turbulence's multiscale characteristics is essential for reliable predictions of multiphysics interactions, but remains a grand challenge even for exascale supercomputers and advanced deep learning models. The extreme-resolution data required to represent turbulence, ranging from billions to trillions of grid points, pose prohibitive computational costs for models based on architectures like vision transformers. To address this challenge, we introduce a multiscale hierarchical Turbulence Transformer that reduces sequence length from billions to a few millions and a novel RingX sequence parallelism approach that enables scalable long-context learning. We perform scaling and science runs on the Frontier supercomputer. Our approach demonstrates excellent performance up to 1.1 EFLOPS on 32,768 AMD GPUs, with a scaling efficiency of 94%. To our knowledge, this is the first AI model for turbulence that can capture small-scale eddies down to the dissipative range.
- Open Access
Multi-task parallelism for robust pre-training of graph foundation models on multi-source, multi-fidelity atomistic modeling data
Graph foundation models using graph neural networks promise sustainable, efficient atomistic modeling. To tackle challenges of processing multi-source, multi-fidelity data during pre-training, recent studies employ multi-task learning, in which shared message passing layers initially process input atomistic structures regardless of source, then route them to multiple decoding heads that predict data-specific outputs. This approach stabilizes pre-training and enhances a model's transferability to unexplored chemical regions. Preliminary results on approximately four million structures are encouraging, yet questions remain about generalizability to larger, more diverse datasets and scalability on supercomputers. We propose a multi-task parallelism method that distributes each head across computing resources with GPU acceleration. Implemented in the open-source HydraGNN architecture, our method was trained on over 24 million structures from five datasets and tested on the Perlmutter, Aurora, and Frontier supercomputers, demonstrating efficient scaling on all three highly heterogeneous super-computing architectures.
Open AccessMulti-task parallelism for robust pre-training of graph foundation models on multi-source, multi-fidelity atomistic modeling data
Graph foundation models using graph neural networks promise sustainable, efficient atomistic modeling. To tackle challenges of processing multi-source, multi-fidelity data during pre-training, recent studies employ multi-task learning, in which shared message passing layers initially process input atomistic structures regardless of source, then route them to multiple decoding heads that predict data-specific outputs. This approach stabilizes pre-training and enhances a model's transferability to unexplored chemical regions. Preliminary results on approximately four million structures are encouraging, yet questions remain about generalizability to larger, more diverse datasets and scalability on supercomputers. We propose a multi-task parallelism method that distributes each head across computing resources with GPU acceleration. Implemented in the open-source HydraGNN architecture, our method was trained on over 24 million structures from five datasets and tested on the Perlmutter, Aurora, and Frontier supercomputers, demonstrating efficient scaling on all three highly heterogeneous super-computing architectures.