Full-Stack Architecting to Achieve a Billion-Requests-Per-Second Throughput on a Single Key-Value Store Server Platform
Authors:
Open Access
- 6 April 2016
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computer Systems
- Vol. 34 (2) , 1-30
Abstract
Distributed in-memory key-value stores (KVSs), such as memcached, have become a critical data serving layer in modern Internet-oriented data center infrastructure. Their performance and efficiency directly affect the QoS of web services and the efficiency of data centers. Traditionally, these systems have had significant overheads from inefficient network processing, OS kernel involvement, and concurrency control. Two recent research thrusts have focused on improving key-value performance. Hardware-centric research has started to explore specialized platforms including FPGAs for KVSs; results demonstrated an order of magnitude increase in throughput and energy efficiency over stock memcached. Software-centric research revisited the KVS application to address fundamental software bottlenecks and to exploit the full potential of modern commodity hardware; these efforts also showed orders of magnitude improvement over stock memcached. We aim at architecting high-performance and efficient KVS platforms, and start with a rigorous architectural characterization across system stacks over a collection of representative KVS implementations. Our detailed full-system characterization not only identifies the critical hardware/software ingredients for high-performance KVS systems but also leads to guided optimizations atop a recent design to achieve a record-setting throughput of 120 million requests per second (MRPS) (167MRPS with client-side batching) on a single commodity server. Our system delivers the best performance and energy efficiency (RPS/watt) demonstrated to date over existing KVSs including the best-published FPGA-based and GPU-based claims. We craft a set of design principles for future platform architectures, and via detailed simulations demonstrate the capability of achieving a billion RPS with a single server constructed following our principles.Keywords
Funding Information
- National Science Foundation under award (CNS-1345305)
- Korea government (2014R1A2A1A11052936)
- Intel Science and Technology Center for Cloud Computing
- National Research Foundation of Korea
This publication has 16 references indexed in Scilit:
- Mega-KVProceedings of the VLDB Endowment, 2015
- Integrated 3D-stacked server designs for increasing physical density of key-value stores2014
- Scale-out NUMA2014
- An FPGA-based in-line accelerator for Memcached2013
- An FPGA memcached appliance2013
- Chronos2012
- Scale-out processors2012
- Direct Cache Access for High Bandwidth Network I/OACM SIGARCH Computer Architecture News, 2005
- Latency lags bandwithCommunications of the ACM, 2004
- Cuckoo hashingJournal of Algorithms, 2004