RAPID: A Fast Data Update Protocol in Erasure Coded Storage Systems for Big Data
Authors:
- G. J. AkashGAG. J. Akash
- Ojus Thomas LeeOLOjus Thomas Lee
- S. D. Madhu KumarSKS. D. Madhu Kumar
- Priya ChandranPCPriya Chandran
- Alfredo CuzzocreaACAlfredo Cuzzocrea
- 1 May 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 890-897
Abstract
Erasure codes are nowadays used extensively in distributed storage systems that handle big data, since they offer significant fault tolerance with low storage overhead. Even though erasure coded systems are space efficient, these involve higher network bandwidth and computational complexity in their operations. In this paper, we present RAPID, a protocol for fast data updates, which works by choosing a subset of code blocks for updates and adapts the strength of the subset based on the predicted number of failures. The proposal uses a prediction based heuristic in which the set of failures that may happen in the near future is represented as a function of past failures. A hybrid protocol that uses both locking and buffering mechanisms is adopted in the solution to maintain the consistency on the data and code blocks updates. Our experimental results demonstrate improvement in the performance of data updates by 30% and the failure prediction mechanism proposed shows an accuracy of 80%.Keywords
This publication has 14 references indexed in Scilit:
- Erasure coded storage systems for cloud storage — challenges and opportunities2016
- Privacy and Security of Big Data2014
- A "hitchhiker's" guide to fast and efficient data reconstruction in erasure-coded data centers2014
- Efficient updates in cross-object erasure-coded storage systems2013
- Consistency and fault tolerance for erasure-coded distributed storage systems2012
- Two Efficient Partial-Updating Schemes for Erasure-Coded Storage Clusters2012
- Using Erasure Codes Efficiently for Storage in a Distributed System2005
- A decentralized algorithm for erasure-coded virtual disks2004
- RAID-x: a new distributed disk array for I/O-centric cluster computing2000
- A comprehensive review of hard-disk drive reliability1999