Dynamic Adaptation for Elastic System Services Using Virtual Servers
Authors:
- Abhishek KulkarniAKAbhishek Kulkarni
- Hugh GreenbergHGHugh Greenberg
- Michael LangMLMichael Lang
- Andrew LumsdaineALAndrew Lumsdaine
- 1 December 2015
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 125-134
Abstract
A vast majority of legacy runtime systems and middleware prevalent in cluster and supercomputing environments are static in nature. Due to the rising scale and complexity of high-performance computing systems, the static nature of systems software would prospectively impede its scalability and resilience. Traditionally, the mobility of servers is further limited since services are statically bound to specific communication endpoints. To address these challenges imminent for exascale-class systems, distributed middleware needs to support dynamic reconfiguration, redundant and replicated state, and adaptation where the number of servers can vary according to the load in the system. We identify the key features necessary from the underlying network infrastructure to support dynamic adaptation and elasticity in distributed system software, and describe the implementation of a high-performance middleware library that implements the proposed interface. We discuss several novel approaches for dynamic resolution using range computations performed by hosts (in software) and by switches (in hardware), and compare the performance on contemporary Ethernet networks. Finally, we validate the benefits offered by our library with two different applications -- a scalable DHCP server and an elastic key-value store.Keywords
This publication has 10 references indexed in Scilit:
- Using simulation to explore distributed key-value stores for extreme-scale system services2013
- A Framework for Elastic Execution of Existing MPI Programs2011
- PortLand2009
- Virtual routers on the moveACM SIGCOMM Computer Communication Review, 2008
- Floodless in seattle2008
- OpenFlowACM SIGCOMM Computer Communication Review, 2008
- Fast PaxosDistributed Computing, 2006
- Chord2001
- JXTA: a network programming environmentIEEE Internet Computing, 2001
- Consistent hashing and random trees1997