An Adaptive Erasure Code for JointCloud Storage of Internet of Things Big Data
Authors:
- 16 October 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 7 (3) , 1613-1624
Abstract
JointCloud is a cross-cloud cooperation architecture for integrated internet service customization. The customized cross-cloud storage service based on this architecture is called JointCloud storage. Storing the Internet of Things (IoT) big data in erasure-coded JointCloud storage systems ensures that data can be accessed when several cloud services interrupt. However, because existing erasure codes cannot adapt generator matrix and data placement scheme to different network environments and encoding parameters, they usually incur a large network resource consumption for repairing data in JointCloud storage systems. As a result, the availability of IoT applications running on JointCloud storage systems is impaired. In this paper, to minimizes the network resource consumption of repairing data, we propose an adaptive erasure code for JointCloud storage of IoT big data called ACIoT. Specifically, we first propose the concept of average weighted locality of a stripe of erasure-coded data, which is proportional to the average network resource consumption of repairing this stripe in JointCloud storage systems. Then, we propose an active parallel trial-and-error algorithm to calculate the optimal generator matrix and data placement scheme to achieve the lowest average weighted locality, under different network environments and encoding parameters. By encoding and placing each stripe of data with the optimal generator matrix and data placement scheme, ACIoT can achieve the minimum network resource consumption. Experiments show that: compared with several state-of-the-art erasure codes, ACIoT reduces the network resource consumption by 26.4%-44.7%.Keywords
Funding Information
- National Basic Research Program of China (2016YFB1000101)
- National Natural Science Foundation of China (61379052)
- Science Foundation of Ministry of Education of China (2018A02002)
- Science Fund for Distinguished Young Scholars of Hunan Province (14JJ1026)
This publication has 20 references indexed in Scilit:
- Local erasure correction codes with unequal locality profile2016
- Codes with unequal locality2016
- T-Update: A tree-structured update scheme with top-down transmission in erasure-coded systems2016
- On improving recovery performance in erasure code based geo-diverse storage clusters2016
- On the locality of codeword symbols in non-linear codesDiscrete Mathematics, 2014
- Optimal locally repairable codes via rank-metric codes2013
- XORing elephantsProceedings of the VLDB Endowment, 2013
- Optimal linear codes with a local-error-correction property2012
- Simple regenerating codes: Network coding for cloud storage2012
- Research and performance evaluation of data replication technology in distributed storage systemsComputers & Mathematics with Applications, 2006