Achieving Tunable Erasure Coding with Cluster-Aware Redundancy Transitioning
Open Access
- 14 September 2024
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Architecture and Code Optimization
- Vol. 21 (3) , 1-24
Abstract
Erasure coding has been demonstrated as a storage-efficient means against failures, yet its tunability remains a challenging issue in data centers, which is prone to induce substantial cross-cluster traffic. In this article, we presentClusterRT, a cluster-aware redundancy transitioning approach that can dynamically tailor the redundancy degree of erasure coding in data centers.ClusterRTformulates the data relocation as the maximum flow problem to reduce cross-cluster data transfers. It then designs a parity-coordinated update algorithm, which gathers the parity chunks within the same cluster and leverages encoding dependency to further decrease the cross-cluster update traffic.ClusterRTfinally rotates the parity chunks to balance the cross-cluster transitioning traffic across the data center. Large-scale simulation and Alibaba Cloud ECS experiments show thatClusterRTreduces 94.0% to 96.2% of transitioning traffic and reduces 70.4% to 88.4% of transitioning time.Keywords
Funding Information
- National Key R&D Program of China (2021YFF0704001)
- Major Research Plan of the National Natural Science Foundation of China (92373114)
- Natural Science Foundation of China (62072381)
- Natural Science Foundation of Fujian Province of China (2023J06001)
This publication has 29 references indexed in Scilit:
- Optimal Rack-Coordinated Updates in Erasure-Coded Data Centers2021
- On the Optimal Repair-Scaling Trade-off in Locally Repairable Codes2020
- ClusterSR: Cluster-Aware Scattered Repair in Erasure-Coded Storage2020
- Reconsidering Single Failure Recovery in Clustered File Systems2016
- Scale-RS: An Efficient Scaling Scheme for RS-Coded Storage ClustersIEEE Transactions on Parallel and Distributed Systems, 2014
- Locally Repairable CodesIEEE Transactions on Information Theory, 2014
- ALV: A New Data Redistribution Approach to RAID-5 ScalingIEEE Transactions on Computers, 2009
- Systematic MDS Erasure Codes Based on Vandermonde MatricesIEEE Communications Letters, 2004
- The complexity of finding maximum disjoint paths with length constraintsNetworks, 1982
- Polynomial Codes Over Certain Finite FieldsJournal of the Society for Industrial and Applied Mathematics, 1960