TMO: transparent memory offloading in datacenters
Authors:
Open Access
- 22 February 2022
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 609-621
Abstract
The unrelenting growth of the memory needs of emerging datacenter applications, along with ever increasing cost and volatility of DRAM prices, has led to DRAM being a major infrastructure expense. Alternative technologies, such as NVMe SSDs and upcoming NVM devices, offer higher capacity than DRAM at a fraction of the cost and power. One promising approach is to transparently offload colder memory to cheaper memory technologies via kernel or hypervisor techniques. The key challenge, however, is to develop a datacenter-scale solution that is robust in dealing with diverse workloads and large performance variance of different offload devices such as compressed memory, SSD, and NVM. This paper presents TMO, Meta’s transparent memory offloading solution for heterogeneous datacenter environments. TMO introduces a new Linux kernel mechanism that directly measures in realtime the lost work due to resource shortage across CPU, memory, and I/O. Guided by this information and without any prior application knowledge, TMO automatically adjusts how much memory to offload to heterogeneous devices (e.g., compressed memory or SSD) according to the device’s performance characteristics and the application’s sensitivity to memory-access slowdown. TMO holistically identifies offloading opportunities from not only the application containers but also the sidecar containers that provide infrastructure-level functions. To maximize memory savings, TMO targets both anonymous memory and file cache, and balances the swap-in rate of anonymous memory and the reload rate of file pages that were recently evicted from the file cache. TMO has been running in production for more than a year, and has saved between 20-32% of the total memory across millions of servers in our large datacenter fleet. We have successfully upstreamed TMO into the Linux kernel.Keywords
This publication has 23 references indexed in Scilit:
- Remote memory in the age of fast networks2017
- Utility-Based Hybrid Memory Management2017
- Thermostat2017
- Technology scaling challenges and opportunities of memory devices2016
- Data tiering in heterogeneous memory systems2016
- Evaluating STT-RAM as an energy-efficient main memory alternative2013
- Page placement in hybrid memory systems2011
- Fifty Years of Moore's LawIEEE Transactions on Semiconductor Manufacturing, 2011
- Scalable high performance main memory system using phase-change memory technology2009
- Dynamic tracking of page miss ratio curve for memory managementACM SIGPLAN Notices, 2004