Hierarchical Codes: How to Make Erasure Codes Attractive for Peer-to-Peer Storage Systems
Authors:
- Alessandro DuminucoADAlessandro Duminuco
- Ernst BiersackEBErnst Biersack
- 1 September 2008
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 89-98
Abstract
Redundancy is the basic technique to provide reliability in storage systems consisting of multiple components. A redundancy scheme defines how the redundant data are produced and maintained. The simplest redundancy scheme is replication, which however suffers from storage inefficiency. Another approach is erasure coding, which provides the same level of reliability as replication using a significantly smaller amount of storage. When redundant data are lost, they need to be replaced. While replacing replicated data consists in a simple copy, it becomes a complex operation with erasure codes: new data are produced performing a coding over some other available data. The amount of data to be read and coded is d times larger than the amount of data produced. This implies that coding has a larger computational and I/O cost, which, for distributed storage systems, translates into increased network traffic. Participants of peer-to-peer systems have ample storage and CPU power, but their network bandwidth may be limited. For these reasons existing coding techniques are not suitable for P2P storage. This work explores the design space between replication and the existing erasure codes. We propose and evaluate a new class of erasure codes, called hierarchical codes, which aims at finding a flexible trade-off that allows the reduction of the network traffic due to maintenance without losing the benefits given by traditional codes.Keywords
This publication has 6 references indexed in Scilit:
- Network Coding for Distributed Storage Systems2007
- Linear network codingIEEE Transactions on Information Theory, 2003
- Farsite2002
- Wide-area cooperative storage with CFS2001
- Network information flowIEEE Transactions on Information Theory, 2000
- A tutorial on Reed–Solomon coding for fault‐tolerance in RAID-like systemsSoftware: Practice and Experience, 1997