Toasty: Speeding Up Network I/O with Cache-Warm Buffers
Open Access
- 22 March 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 2015-2029
Abstract
Modern NICs DMA packets directly to the LLC using technologies like DDIO, reducing access latencies for networking applications. Prior work has observed several performance issues with DDIO when the working set of packet buffers does not fit into LLC. For example, the leaky DMA problem arises when incoming packets evict older packets that have not yet been processed by the application from LLC, causing them to be fetched again from main memory. While using a smaller pool of packet buffers that fits in cache is an obvious solution, this may result in the NIC running out of buffers to DMA packets into when a burst of packets arrives. This paper proposes Toasty, a system that mitigates this tradeoff between high throughput and resilience to packet loss that arises when sizing the network packet buffer pool. While prior work has proposed hardware-based solutions to this problem, Toasty is a software-only solution that can be deployed on commodity NIC hardware. Toasty manages the packet buffer pool as a LIFO stack instead of a FIFO queue, and adapts the number of buffers populated into the NIC hardware RX ring based on incoming packet load and application processing rate. Together, these changes enable Toasty to recirculate a small working set of cache-warm buffers in steady state, while falling back to a larger pool of buffers during traffic bursts. We implement Toasty over the AF_XDP kernel bypass framework, and our evaluation shows that Toasty improves network throughput for a variety of network functions by up to 78% over the default buffer pool implementation of AF_XDP. We also show that Toasty matches the performance of a small buffer pool that fits in cache, while being more resilient to traffic bursts.Keywords
Funding Information
- Asha Navani Fellowship IIT Bombay (DO\/2024-JANF001-008)
- Microsoft Research India PhD Award (IITBRSP242000100)
This publication has 7 references indexed in Scilit:
- CEIO: A Cache-Efficient Network I/O Architecture for NIC-CPU Data Paths2025
- Comparing User Space and In-Kernel Packet Processing for Edge Data CentersACM SIGCOMM Computer Communication Review, 2023
- The benefits of general-purpose on-NIC memory2022
- Don’t Forget the I/O When Allocating Your LLC2021
- Make the Most out of Last Level Cache in Intel Processors2019
- DMA cache: Using on-chip storage to architecturally separate I/O data from CPU data for improving I/O performance2010
- Software prefetching1991