LB Scalability: Achieving the Right Balance Between Being Stateful and Stateless
- 20 September 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Networking
- Vol. 30 (1) , 382-393
Abstract
A high performance Layer-4 load balancer (LB) is one of the most important components of a cloud service infrastructure. Such an LB uses network and transport layer information for deciding how to distribute client requests across a group of servers. A crucial requirement for a stateful LB is per connection consistency (PCC); namely, that all the packets of the same connection will be forwarded to the same server, as long as the server is alive, even if the pool of servers or the assignment function changes. The challenge is in designing a high throughput, low latency solution that is also scalable. This paper proposes a highly scalable LB, called Prism, implemented using a programmable switch ASIC. As far as we know, Prism is the first reported stateful LB that can process millions of connections per second and hundreds of millions connections in total, while ensuring PCC. This is due to the fact that Prism forwards all the packets in hardware, even during server pool changes, while avoiding the need to maintain a hardware state per every active connection. We implemented a prototype of the proposed architecture and showed that Prism can scale to 100 million simultaneous connections, and can accommodate more than one pool update per second.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Inside the Social Network's (Datacenter) NetworkACM SIGCOMM Computer Communication Review, 2015
- Pingmesh2015
- DuetACM SIGCOMM Computer Communication Review, 2014
- Forwarding metamorphosis2013
- Dynamically scaling applications in the cloudACM SIGCOMM Computer Communication Review, 2011
- More Robust Hashing: Cuckoo Hashing with a StashSIAM Journal on Computing, 2010
- A break in the cloudsACM SIGCOMM Computer Communication Review, 2008
- Space Efficient Hash Tables with Worst Case Constant Access TimeTheory of Computing Systems, 2004
- Cuckoo hashingJournal of Algorithms, 2004
- A Generalized Birthday Problem2002