I/O Deduplication
Authors:
Open Access
- 28 September 2010
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Storage
- Vol. 6 (3) , 1-26
Abstract
Duplication of data in storage systems is becoming increasingly common. We introduce I/O Deduplication, a storage optimization that utilizes content similarity for improving I/O performance by eliminating I/O operations and reducing the mechanical delays during I/O operations. I/O Deduplication consists of three main techniques: content-based caching, dynamic replica retrieval , and selective duplication . Each of these techniques is motivated by our observations with I/O workload traces obtained from actively-used production storage systems, all of which revealed surprisingly high levels of content similarity for both stored and accessed data. Evaluation of a prototype implementation using these workloads showed an overall improvement in disk I/O performance of 28 to 47% across these workloads. Further breakdown also showed that each of the three techniques contributed significantly to the overall performance improvement.Keywords
Funding Information
- Division of Computer and Network Systems (CNS-0747038CNS-0821345IIS-0837716)
- Division of Information and Intelligent Systems (CNS-0747038CNS-0821345IIS-0837716)
This publication has 11 references indexed in Scilit:
- The case for active block layer extensionsACM SIGOPS Operating Systems Review, 2008
- The automatic improvement of locality in storage systemsACM Transactions on Computer Systems, 2005
- FS22005
- A low-bandwidth network file system2001
- Adaptive block rearrangementACM Transactions on Computer Systems, 1995
- Copy detection mechanisms for digital documents1995
- Doubly distorted mirrors1993
- A system for adaptive disk rearrangementSoftware: Practice and Experience, 1990
- Minimizing Expected Head Movement in One-Dimensional and Two-Dimensional Mass Storage SystemsACM Computing Surveys, 1980
- Space/time trade-offs in hash coding with allowable errorsCommunications of the ACM, 1970