cuPTW: Leveraging Idle Compute Units for Massively Parallel GPU Page Table Walks
Open Access
- 29 May 2026
- journal article
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Measurement and Analysis of Computing Systems
- Vol. 10 (2) , 1-26
Abstract
Virtual memory has become a cornerstone of modern GPUs, enabling unified address spaces and advanced memory management techniques. However, the performance of address translation has emerged as a critical bottleneck, particularly under irregular workloads with massive memory footprints, where frequent TLB misses and costly page table walks dominate total memory access latency. Prior work has primarily focused on improving TLB effectiveness or optimizing page table walks through batching and coalescing, but these approaches remain limited by the lack of locality and memory bandwidth constraints. In this work, we propose Compute Unit Page Table Walk ( cuPTW ), a novel address translation architecture that repurposes idle GPU compute resources to accelerate page table walks. Specifically, we introduce (1) a single-threaded synchronous cuPTW , which offloads translation requests to idle functional units within compute units, and (2) two optimizations that further reduce latency and improve throughput by caching page table walks in local data store memory ( cuPTW-SW ) and parallelizing them across multiple SIMD lanes ( cuPTW-MT ). When combined, cuPTW-Full transforms low-parallelism page table walks into a massively parallel computation task. Our evaluation across 15 representative GPU workloads, including deep learning, graph analytics and scientific simulations, demonstrates that cuPTW-Full achieves a performance speedup of 4.43× on average (up to 76.09×) by improving page table walk throughput by 9.92× on average over our baseline GPU architecture. Compared to state-of-the-art GPU address translation proposals, cuPTW-Full achieves a 1.97× to 2.08× average speedup.Keywords
Funding Information
- National Natural Science Foundation of China (62402411)
- CCF-Phytium Fund (CCF-Phytium202302)
- Guangdong Provincial Project (2023QN10X252)
This publication has 43 references indexed in Scilit:
- Orchestrated Scheduling and Partitioning for Improved Address Translation in GPUs2023
- Demystifying the Nvidia Ampere Architecture through Microbenchmarking and Instruction-level Analysis2022
- Hetero-mark, a benchmark suite for CPU-GPU collaborative computing2016
- Towards high performance paged memory for GPUs2016
- Unlocking bandwidth for GPUs in CC-NUMA systems2015
- Supporting x86-64 address translation for 100s of GPU lanes2014
- Improving GPGPU resource utilization through alternative thread block scheduling2014
- Pannotia: Understanding irregular GPGPU graph applications2013
- A quantitative study of irregular programs on GPUs2012
- Optimized HPL for AMD GPU and multi-core CPU usageComputer Science - Research and Development, 2011