Design and implementation trade-offs for wide-area resource discovery
Authors:
Open Access
- 1 September 2008
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Internet Technology
- Vol. 8 (4) , 1-44
Abstract
We describe the design and implementation of SWORD, a scalable resource discovery service for wide-area distributed systems. In contrast to previous systems, SWORD allows users to describe desired resources as a topology of interconnected groups with required intragroup, intergroup, and per-node characteristics, along with the utility that the application derives from specified ranges of metric values. This design gives users the flexibility to find geographically distributed resources for applications that are sensitive to both node and network characteristics, and allows the system to rank acceptable configurations based on their quality for that application. Rather than evaluating a single implementation of SWORD, we explore a variety of architectural designs that deliver the required functionality in a scalable and highly available manner. We discuss the trade-offs of using a centralized architecture as compared to a fully decentralized design to perform wide-area resource discovery. To summarize our results, we found that a centralized architecture based on 4-node server cluster sites at network-peering facilities outperforms a decentralized DHT-based resource discovery infrastructure with respect to query latency for all but the smallest number of sites. However, although a centralized architecture shows significant promise in stable environments, we find that our decentralized implementation has acceptable performance and also benefits from the DHT's self-healing properties in more volatile environments. We evaluate the advantages and disadvantages of centralized and distributed resource discovery architectures on 1000 hosts in emulation and on approximately 200 PlanetLab nodes spread across the Internet.Keywords
This publication has 38 references indexed in Scilit:
- OpenDHT2005
- Simple Efficient Load Balancing Algorithms for Peer-to-Peer Systems2005
- An algebraic approach to practical and scalable overlay network monitoring2004
- The ganglia distributed monitoring system: design, implementation, and experienceParallel Computing, 2004
- SHARP2003
- AstrolabeACM Transactions on Computer Systems, 2003
- pSearchACM SIGCOMM Computer Communication Review, 2003
- Towards capturing representative AS-level Internet topologies2002
- The Anatomy of the Grid: Enabling Scalable Virtual OrganizationsThe International Journal of High Performance Computing Applications, 2001
- ECONOMIC MODELS FOR ALLOCATING RESOURCES IN COMPUTER SYSTEMS1996