Redesigning Small Payload Transfer Paths for Emerging NVMe SSD Workloads
Open Access
- 13 November 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Storage
Abstract
As modern NVMe SSDs become increasingly capable of in-device computation, workloads such as SQL filtering and key-value store operations are now being offloaded from the host to the SSD. These tasks often involve small payloads, typically a few dozen to hundreds of bytes, which are inefficiently processed by the standard NVMe interface due to its fixed memory page-granularity DMA transfer. Regardless of how small the payload is, even a few dozen bytes, each request still incurs memory page-sized PCIe traffic, resulting in significant bandwidth waste and latency overhead. Existing approaches either require invasive modifications to the I/O stack and SSD internals, or offer only limited benefits for extremely small payloads. This paper presents ByteExpress, a novel mechanism that efficiently transmits small payloads by embedding them inline within NVMe queue entries, embedding for writes and extracting for reads. ByteExpress maintains PCIe traffic efficiency while delivering high performance, all while preserving full compatibility with existing APIs and SSD architectures. Our implementation and evaluation on the Linux NVMe driver and both real hardware and emulation environments, shows up to 96% reduction in PCIe traffic and 25% higher throughput compared with the standard page-based mechanism, and 79% lower latency compared with the state-of-the-art solution.This publication has 24 references indexed in Scilit:
- KVACCEL: A Novel Write Accelerator for LSM-Tree-Based KV Stores with Host-SSD Collaboration2025
- Maximizing Interconnect Bandwidth and Efficiency in Nonvolatile Memory, Express-Based Key-Value Solid-State Devices With Fine-Grained Value TransferIEEE Micro, 2025
- KV-CSD: A Hardware-Accelerated Key-Value Store for Data-Intensive Applications2023
- OCTOKV: An Agile Network-Based Key-Value Storage System with Robust Load Orchestration2023
- Hvac: Removing I/O Bottleneck for Large-Scale Deep Learning Applications2022
- iLSM-SSD: An Intelligent LSM-Tree Based Key-Value SSD for Data Analytics2019
- Overview of PAN 2019: Bots and Gender Profiling, Celebrity Profiling, Cross-Domain Authorship Attribution and Style Change Detection2019
- 2B-SSD: The Case for Dual, Byte- and Block-Addressable Solid-State Drives2018
- KAML: A Flexible, High-Performance Key-Value SSD2017
- Accelerating File System Metadata Access with Byte-Addressable Nonvolatile MemoryACM Transactions on Storage, 2015