CORE: Augmenting regenerating-coding-based recovery for single and concurrent failures in distributed storage systems
- 1 May 2013
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 2160195X,p. 1-6
Abstract
Data availability is critical in distributed storage systems, especially when node failures are prevalent in real life. A key requirement is to minimize the amount of data transferred among nodes when recovering the lost or unavailable data of failed nodes. This paper explores recovery solutions based on regenerating codes, which are shown to provide fault-tolerant storage and minimum recovery bandwidth. Existing optimal regenerating codes are designed for single node failures. We build a system called CORE, which augments existing optimal regenerating codes to support a general number of failures including single and concurrent failures. We theoretically show that CORE achieves the minimum possible recovery bandwidth for most cases. We implement CORE and evaluate our prototype atop a Hadoop HDFS cluster testbed with up to 20 storage nodes. We demonstrate that our CORE prototype conforms to our theoretical findings and achieves recovery bandwidth saving when compared to the conventional recovery approach based on erasure codes.Keywords
All Related Versions
This publication has 23 references indexed in Scilit:
- Single Disk Failure Recovery for X-Code-Based Parallel Storage SystemsIEEE Transactions on Computers, 2014
- CORE: Augmenting regenerating-coding-based recovery for single and concurrent failures in distributed storage systems2013
- A cost-based heterogeneous recovery scheme for distributed storage systems with RAID-6 codes2012
- On the speedup of single-disk failure recovery in XOR-coded storage systems: Theory and practice2012
- Simple regenerating codes: Network coding for cloud storage2012
- Optimal Exact-Regenerating Codes for Distributed Storage at the MSR and MBR Points via a Product-Matrix ConstructionIEEE Transactions on Information Theory, 2011
- Repairing Multiple Failures with Coordinated and Adaptive Regenerating Codes2011
- Cooperative Recovery of Distributed Storage Systems from Multiple Losses with Network CodingIEEE Journal on Selected Areas in Communications, 2010
- Explicit construction of optimal exact regenerating codes for distributed storage2009
- OceanStore2000