UMap: An application-oriented user level memory mapping library
- 5 December 2024
- journal article
- research article
- Published by SAGE Publications in The International Journal of High Performance Computing Applications
- Vol. 39 (2) , 269-282
Abstract
Exploiting the prominent role of complex memories in exascale node architecture, the UMap page fault handler offers new capabilities to access large memory-mapped data sets directly. UMap provides flexible configuration options to customize page handling to each application, including analysis of massive observational and simulation data sets. The high-performance design features I/O decoupling, dynamic load balancing, and application-level controls. Page faults triggered by application threads and processes accessing data mapped to a UMapp’ed region are handled via the Linux userfaultfd protocol, an asynchronous message-oriented kernel-user communication mechanism that avoids the context switch penalty of traditional signal fault handlers. UMap is fully open source. In this paper, we give an overview of the UMap library architecture, its extensible plugin architecture, and the use/performance of UMap in emerging heterogeneous memory hierarchies such as near-node Non-volatile Memory (NVM) and network attached memories. We highlight new capabilities in two pagefault management plugins, the NetworkStore and SparseStore. We demonstrate the integration between UMap and multiple ECP products including Caliper, Metall, ZFP, Mochi, and Ripples.Keywords
Funding Information
- Lawrence Livermore National Laboratory (DE-AC52-07NA27344, LLNL-JRNL-860428)
- Exascale Computing Project (17-SC-20-SC)
This publication has 19 references indexed in Scilit:
- Evaluating Emerging CXL-enabled Memory Pooling for HPC Systems2022
- FAM-Graph: Graph Analytics on Disaggregated Memory2022
- Metall: A persistent memory allocator for data-centric analyticsParallel Computing, 2022
- On the Memory Underutilization: Exploring Disaggregated Memory on HPC Systems2020
- Mochi: Composing Data Services for High-Performance Computing EnvironmentsJournal of Computer Science and Technology, 2020
- UMap: Enabling Application-driven Optimizations for Page Management2019
- Fast and Scalable Implementations of Influence Maximization Algorithms2019
- RAM as a Network Managed Resource2018
- DI-MMAP—a scalable memory-map runtime for out-of-core data-intensive applicationsCluster Computing, 2013
- Community Structure in Large Networks: Natural Cluster Sizes and the Absence of Large Well-Defined ClustersInternet Mathematics, 2009