From luna to solar
Authors:
Open Access
- 22 August 2022
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 753-766
Abstract
This paper presents the two generations of storage network stacks that reduced the average I/O latency of Alibaba Cloud's EBS service by 72% in the last five years: Luna, a user-space TCP stack that corresponds the latency of network to the speed of SSD; and Solar, a storage-oriented UDP stack that enables both storage and network hardware accelerations. Luna is our first step towards a high-speed compute-to-storage network in the "storage disaggregation" architecture. Besides the tremendous performance gains and CPU savings compared with the legacy kernel TCP stack, more importantly, it teaches us the necessity of offloading both network and storage into hardware and the importance of recovering instantaneously from network failures. Solar provides a highly reliable and performant storage network running on hardware. For avoiding hardware's resource limitations and offloading storage's entire data path, Solar eliminates the superfluous complexity and the overfull states from the traditional architecture of the storage network. The core design of Solar is unifying the concepts of network packet and storage data block - each network packet is a self-contained storage data block. There are three remarkable advantages to doing so. First, it merges the packet processing and storage virtualization pipelines to bypass the CPU and PCIe; Second, since the storage processes data blocks independently, the packets in Solar become independent. Therefore, the storage (in hardware) does not need to maintain receiving buffers for assembling packets into blocks or handling packet reordering. Finally, due to the low resource requirement and the resilience to packet reordering, Solar inherently supports large-scale multi-path transport for fast failure recovery. Facing the future, Solar demonstrates that we can formalize the storage virtualization procedure into a P4-compatible packet processing pipeline. Hence, SOLAR's design perfectly applies to commodity DPUs (data processing units).Keywords
This publication has 17 references indexed in Scilit:
- Snap2019
- NetCache2017
- ReFlex2017
- IncBricks2017
- Scalable Hierarchical Aggregation Protocol (SHArP): A Hardware Architecture for Efficient Data Reduction2016
- RDMA over Commodity Ethernet at Scale2016
- Paxos Made Switch-yACM SIGCOMM Computer Communication Review, 2016
- Congestion Control for Large-Scale RDMA Deployments2015
- NetPaxos2015
- Improving datacenter performance and robustness with multipath TCPACM SIGCOMM Computer Communication Review, 2011