From Good to Great: Parameter Tuning in Memory Tiering Systems
- 21 January 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 75 (4) , 1378-1390
Abstract
To meet the memory demands of modern data-intensive applications such as graph processing and machine learning, datacenters are adopting tiered memory architectures that include fast tiers such as locally attached DRAM and slower tiers built using CXL-attached memory. While these newer tiers expand capacity, they have higher access latencies and lower bandwidth, which can lead to application performance degradation. To minimize performance loss, frequently accessed (hot) data must be placed close to the CPU in faster tiers and infrequently accessed (cold) data can be placed in farther, slower memory tiers. Existing software-managed tiering solutions employ heuristics and pre-configured thresholds to determine hot and cold data, and make placement and migration decisions. Unfortunately, due to their use of static thresholds, these systems fail to adapt to different workloads and the underlying hardware, and hence perform poorly. In this paper, we study the relationship between thresholds and parameters used by tiering systems and application performance. We leverage Bayesian Optimization to tune the parameters of three tiering systems: HeMem, Memtis, and HMSDK. We find that applications are sensitive to these parameter values, and that best parameter values vary widely across workloads, inputs, thread counts, memory ratios, and hardware characteristics. Our experiments reveal that the Bayesian optimizer can learn workload behaviors and set parameter values that result in best performance. With proper parameter tuning, performance can improve by up to 2x compared to default configurations, highlighting the importance of parameter selection for memory tiering systems.Keywords
This publication has 30 references indexed in Scilit:
- Improving Key-Value Cache Performance With Heterogeneous Memory Tiering: A Case Study of Compute-Express-Link-Based Memory ExpansionIEEE Micro, 2024
- Coeus: Clustering (A)like Patterns for Practical Machine Intelligent Hybrid Memory Management2022
- MULTI-CLOCK: Dynamic Tiering for Hybrid Memory Systems2022
- Cori: Dancing to the Right Beat of Periodic Data Movements over Hybrid Memory Systems2021
- AutoTiering: Automatic data placement manager in multi-tier all-flash datacenter2017
- Taking the Human Out of the Loop: A Review of Bayesian OptimizationProceedings of the IEEE, 2015
- Page placement in hybrid memory systems2011
- Sequential Model-Based Optimization for General Algorithm Configuration2011
- The Benchmark Suite2008
- A Simple Synchronous Distributed-Memory Algorithm for the HPCC RandomAccess Benchmark2006