Data Replication in Data Intensive Scientific Applications with Performance Guarantee
Authors:
- 10 December 2010
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 22 (8) , 1299-1306
Abstract
Data replication has been well adopted in data intensive scientific applications to reduce data file transfer time and bandwidth consumption. However, the problem of data replication in Data Grids, an enabling technology for data intensive applications, has proven to be NP-hard and even non approximable, making this problem difficult to solve. Meanwhile, most of the previous research in this field is either theoretical investigation without practical consideration, or heuristics-based with little or no theoretical performance guarantee. In this paper, we propose a data replication algorithm that not only has a provable theoretical performance guarantee, but also can be implemented in a distributed and practical manner. Specifically, we design a polynomial time centralized replication algorithm that reduces the total data file access delay by at least half of that reduced by the optimal replication solution. Based on this centralized algorithm, we also design a distributed caching algorithm, which can be easily adopted in a distributed environment such as Data Grids. Extensive simulations are performed to validate the efficiency of our proposed algorithms. Using our own simulator, we show that our centralized replication algorithm performs comparably to the optimal algorithm and other intuitive heuristics under different network parameters. Using GridSim, a popular distributed Grid simulator, we demonstrate that the distributed caching technique significantly outperforms an existing popular file caching technique in Data Grids, and it is more scalable and adaptive to the dynamic change of file access patterns in Data Grids.Keywords
This publication has 37 references indexed in Scilit:
- A Reference Architecture for Scientific Workflow Management Systems and the VIEW SOA SolutionIEEE Transactions on Services Computing, 2009
- Cloud Computing and Grid Computing 360-Degree Compared2008
- An SCP-based heuristic approach for scheduling distributed data-intensive applications on global gridsJournal of Parallel and Distributed Computing, 2008
- An on-line replication strategy to increase availability in Data GridsFuture Generation Computer Systems, 2008
- A taxonomy of Data Grids for distributed data sharing, management, and processingACM Computing Surveys, 2006
- Utility-Based Replication Strategies in Data Grids2006
- Content and service replication strategies in multi-hop wireless mesh networks2005
- Analysis of Scheduling and Replica Optimisation Strategies for Data Grids Using OptorSimJournal of Grid Computing, 2004
- Efficient distributed disk caching in data grid management2003
- Directed diffusion2000