SnakeByte: A TLB Design with Adaptive and Recursive Page Merging in GPUs
Authors:
- 1 February 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1195-1207
Abstract
This paper presents an address translation scheme in GPUs named SnakeByte that can dynamically manage variable-sized pages and maximize TLB reach by recursively merging contiguous pages. Memory virtualization has become an integral part of GPUs to enhance programmability and memory management efficiency. However, conventional memory virtualization methods using multi-level page tables and caching them in TLBs are insufficient to provide GPUs with enough address translation coverage for the massive volume of data. SnakeByte implements a hardware-based address translation mechanism that recursively merges contiguous pages into larger page groups and effectively extends TLB coverage. SnakeByte allows multiple equal-sized pages coalescing into a page table entry (PTE). It records the validity of pages to be merged using a bit vector, and few bits are annexed to indicate the size of merged pages. If all pages covered by the PTE are allocated with contiguity, the PTE is promoted to be further coalesced into a larger page group. The recursive coalescence of contiguous pages enables SnakeByte to handle variable-sized page groups with the exponentially increasing TLB reach. Associated with a contiguity-aware memory allocator, SnakeByte can consolidate vastly contiguous address spaces into a few TLB entries. Consequently, it significantly reduces TLB misses for large working sets in GPUs and achieves substantial performance improvements. Experiment results show that SnakeByte decreases the number of page table walks by 6.5x and enhances the GPU performance by 2.0x on average over the conventional paging scheme.Keywords
Funding Information
- Ministry of Education
This publication has 47 references indexed in Scilit:
- Observations and opportunities in architecting shared virtual memory for heterogeneous systems2016
- Towards high performance paged memory for GPUs2016
- NVMMU: A Non-volatile Memory Management Unit for Heterogeneous GPU-SSD Architectures2015
- Redundant memory mappings for fast access to large memories2015
- Ramulator: A Fast and Extensible DRAM SimulatorIEEE Computer Architecture Letters, 2015
- VGRIS: Virtualized GPU Resource Isolation and Scheduling in Cloud GamingACM Transactions on Architecture and Code Optimization, 2014
- Architectural support for address translation on GPUs2014
- Supporting x86-64 address translation for 100s of GPU lanes2014
- Pannotia: Understanding irregular GPGPU graph applications2013
- FreePDK: An Open-Source Variation-Aware Design Kit2007