Jiayi Huang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Jiayi Huang has a total of 61 co-authors. Here are the top co-authors by number of shared publications:
- Le Qin
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China
4 shared publications - Weilin Cai
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China
4 shared publications - Eun Jung Kim
Department of Computer Science and Engineering, Texas A&M University College Station, 14736 College Station, Texas, United States
3 shared publications - Hongyuan Liu
Stevens Institute of Technology, Hoboken, USA
3 shared publications - Ki Hwan Yum
Department of Computer Science and Engineering, Texas A&M University College Station, 14736 College Station, Texas, United States
3 shared publications - Lang Feng
School of Integrated Circuits, Sun Yat-sen University, Shenzhen, China
3 shared publications
Latest Publications
- Open Access
- 20 July 2026
- Published by Association for Computing Machinery (ACM) in ACM SIGMETRICS Performance Evaluation Review
- Vol. 54 (1) , 126-128
cuPTW: Leveraging Idle Compute Units for Massively Parallel GPU Page Table Walks
Virtual memory is essential for modern GPUs, yet address translation remains a major bottleneck. In irregular workloads, frequent TLB misses and costly page table walks (PTWs) dominate memory latency. We propose C ompute U nit P age T able W alk (cuPTW), which repurposes idle GPU compute resources to accelerate PTWs. By offloading requests to idle functional units, cuPTW transforms address translation into a massively parallel task. We further optimize this with cuPTW-SW, which caches walks in local data store (LDS) memory, and cuPTW-MT, which parallelizes walks across SIMD lanes. Evaluation shows that cuPTW-Full achieves a 4.43× average speedup (up to 76.09×) and improves PTW throughput by 9.92×. Compared to state-of-the-art designs, Marching Page Walks and SnakeByte, cuPTW-Full delivers a 2.08× and 1.97× performance gain, respectively.
- 20 July 2026
- Published by Association for Computing Machinery (ACM) in ACM SIGMETRICS Performance Evaluation Review
- Vol. 54 (1) , 126-128
Open AccesscuPTW: Leveraging Idle Compute Units for Massively Parallel GPU Page Table Walks
Virtual memory is essential for modern GPUs, yet address translation remains a major bottleneck. In irregular workloads, frequent TLB misses and costly page table walks (PTWs) dominate memory latency. We propose C ompute U nit P age T able W alk (cuPTW), which repurposes idle GPU compute resources to accelerate PTWs. By offloading requests to idle functional units, cuPTW transforms address translation into a massively parallel task. We further optimize this with cuPTW-SW, which caches walks in local data store (LDS) memory, and cuPTW-MT, which parallelizes walks across SIMD lanes. Evaluation shows that cuPTW-Full achieves a 4.43× average speedup (up to 76.09×) and improves PTW throughput by 9.92×. Compared to state-of-the-art designs, Marching Page Walks and SnakeByte, cuPTW-Full delivers a 2.08× and 1.97× performance gain, respectively.
- Open Access
- 8 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 126-128
cuPTW: Leveraging Idle Compute Units for Massively Parallel GPU Page Table Walks
- 8 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 126-128
Open AccesscuPTW: Leveraging Idle Compute Units for Massively Parallel GPU Page Table Walks
- Open Access
- 29 May 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Measurement and Analysis of Computing Systems
- Vol. 10 (2) , 1-26
cuPTW: Leveraging Idle Compute Units for Massively Parallel GPU Page Table Walks
Virtual memory has become a cornerstone of modern GPUs, enabling unified address spaces and advanced memory management techniques. However, the performance of address translation has emerged as a critical bottleneck, particularly under irregular workloads with massive memory footprints, where frequent TLB misses and costly page table walks dominate total memory access latency. Prior work has primarily focused on improving TLB effectiveness or optimizing page table walks through batching and coalescing, but these approaches remain limited by the lack of locality and memory bandwidth constraints. In this work, we propose Compute Unit Page Table Walk ( cuPTW ), a novel address translation architecture that repurposes idle GPU compute resources to accelerate page table walks. Specifically, we introduce (1) a single-threaded synchronous cuPTW , which offloads translation requests to idle functional units within compute units, and (2) two optimizations that further reduce latency and improve throughput by caching page table walks in local data store memory ( cuPTW-SW ) and parallelizing them across multiple SIMD lanes ( cuPTW-MT ). When combined, cuPTW-Full transforms low-parallelism page table walks into a massively parallel computation task. Our evaluation across 15 representative GPU workloads, including deep learning, graph analytics and scientific simulations, demonstrates that cuPTW-Full achieves a performance speedup of 4.43× on average (up to 76.09×) by improving page table walk throughput by 9.92× on average over our baseline GPU architecture. Compared to state-of-the-art GPU address translation proposals, cuPTW-Full achieves a 1.97× to 2.08× average speedup.
- 29 May 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Measurement and Analysis of Computing Systems
- Vol. 10 (2) , 1-26
Open AccesscuPTW: Leveraging Idle Compute Units for Massively Parallel GPU Page Table Walks
Virtual memory has become a cornerstone of modern GPUs, enabling unified address spaces and advanced memory management techniques. However, the performance of address translation has emerged as a critical bottleneck, particularly under irregular workloads with massive memory footprints, where frequent TLB misses and costly page table walks dominate total memory access latency. Prior work has primarily focused on improving TLB effectiveness or optimizing page table walks through batching and coalescing, but these approaches remain limited by the lack of locality and memory bandwidth constraints. In this work, we propose Compute Unit Page Table Walk ( cuPTW ), a novel address translation architecture that repurposes idle GPU compute resources to accelerate page table walks. Specifically, we introduce (1) a single-threaded synchronous cuPTW , which offloads translation requests to idle functional units within compute units, and (2) two optimizations that further reduce latency and improve throughput by caching page table walks in local data store memory ( cuPTW-SW ) and parallelizing them across multiple SIMD lanes ( cuPTW-MT ). When combined, cuPTW-Full transforms low-parallelism page table walks into a massively parallel computation task. Our evaluation across 15 representative GPU workloads, including deep learning, graph analytics and scientific simulations, demonstrates that cuPTW-Full achieves a performance speedup of 4.43× on average (up to 76.09×) by improving page table walk throughput by 9.92× on average over our baseline GPU architecture. Compared to state-of-the-art GPU address translation proposals, cuPTW-Full achieves a 1.97× to 2.08× average speedup.
- Open Access
- 17 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 659-674
Optimizing All-to-All Collective Communication with Fault Tolerance on Torus Networks
- 17 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 659-674
Open AccessOptimizing All-to-All Collective Communication with Fault Tolerance on Torus Networks
- Open Access
- 17 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 156-169
Software Prefetch Multicast: Sharer-Exposed Prefetching for Bandwidth Efficiency in Manycore Processors
- 17 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 156-169
Open AccessSoftware Prefetch Multicast: Sharer-Exposed Prefetching for Bandwidth Efficiency in Manycore Processors
- Open Access
- 20 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 498-513
Chimera: Communication Fusion for Hybrid Parallelism in Large Language Models
- 20 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 498-513
Open AccessChimera: Communication Fusion for Hybrid Parallelism in Large Language Models
- Open Access
- 20 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 1880-1893
TRACI: Network Acceleration of Input-Dynamic Communication for Large-Scale Deep Learning Recommendation Model
- 20 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 1880-1893
Open AccessTRACI: Network Acceleration of Input-Dynamic Communication for Large-Scale Deep Learning Recommendation Model
- Open Access
- 30 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 655-671
MoC-System: Efficient Fault Tolerance for Sparse Mixture-of-Experts Model Training
- 30 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 655-671
Open AccessMoC-System: Efficient Fault Tolerance for Sparse Mixture-of-Experts Model Training
- Top Cited Papers
- 25 March 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. PP (10414347) , 1-20
A Survey on Mixture of Experts in Large Language Models
Large language models (LLMs) have garnered unprecedented advancements across diverse fields, ranging from natural language processing to computer vision and beyond. The prowess of LLMs is underpinned by their substantial model size, extensive and diverse datasets, and the vast computational power harnessed during training, all of which contribute to the emergent abilities of LLMs (e.g., in-context learning) that are not present in small models. Within this context, the mixture of experts (MoE) has emerged as an effective method for substantially scaling up model capacity with minimal computation overhead, gaining significant attention from academia and industry. Despite its growing prevalence, there lacks a systematic and comprehensive review of the literature on MoE. This survey seeks to bridge that gap, serving as an essential resource for researchers delving into the intricacies of MoE. We first briefly introduce the structure of the MoE layer, followed by proposing a new taxonomy of MoE. Next, we overview the core designs for various MoE models including both algorithmic and systemic aspects, alongside collections of available open-source implementations, hyperparameter configurations and empirical evaluations. Furthermore, we delineate the multifaceted applications of MoE in practice, and outline some potential directions for future research. To facilitate ongoing updates and the sharing of cutting-edge advances in MoE research, we have established a resource repository at https://github.com/withinmiaov/A-Survey-on-Mixture-of-Experts-in-LLMs.
- 25 March 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. PP (10414347) , 1-20
Top Cited PapersA Survey on Mixture of Experts in Large Language Models
Large language models (LLMs) have garnered unprecedented advancements across diverse fields, ranging from natural language processing to computer vision and beyond. The prowess of LLMs is underpinned by their substantial model size, extensive and diverse datasets, and the vast computational power harnessed during training, all of which contribute to the emergent abilities of LLMs (e.g., in-context learning) that are not present in small models. Within this context, the mixture of experts (MoE) has emerged as an effective method for substantially scaling up model capacity with minimal computation overhead, gaining significant attention from academia and industry. Despite its growing prevalence, there lacks a systematic and comprehensive review of the literature on MoE. This survey seeks to bridge that gap, serving as an essential resource for researchers delving into the intricacies of MoE. We first briefly introduce the structure of the MoE layer, followed by proposing a new taxonomy of MoE. Next, we overview the core designs for various MoE models including both algorithmic and systemic aspects, alongside collections of available open-source implementations, hyperparameter configurations and empirical evaluations. Furthermore, we delineate the multifaceted applications of MoE in practice, and outline some potential directions for future research. To facilitate ongoing updates and the sharing of cutting-edge advances in MoE research, we have established a resource repository at https://github.com/withinmiaov/A-Survey-on-Mixture-of-Experts-in-LLMs.