Optimal Rack-Coordinated Updates in Erasure-Coded Data Centers
Authors:
- 10 May 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-10
Abstract
Erasure coding has been extensively deployed in today’s data centers to tackle prevalent failures, yet it is prone to give rise to substantial cross-rack traffic for parity update. In this paper, we propose a new rack-coordinated update mechanism to suppress the cross-rack update traffic, which comprises two successive phases: a delta-collecting phase that collects data delta chunks, and another selective parity update phase that renews the parity chunks based on the update pattern and parity layout. We further design RackCU, an optimal rack-coordinated update solution that achieves the theoretical lower bound of the cross-rack update traffic. We finally conduct extensive evaluations, in terms of large-scale simulation and real-world data center experiments, showing that RackCU can reduce 22.1%-75.1% of the cross-rack update traffic and hence improve 34.2%-292.6% of the update throughput.Keywords
Funding Information
- Tencent
- Natural Science Foundation of Fujian Province
- Innovation and Technology Fund
This publication has 23 references indexed in Scilit:
- Partial-parallel-repair (PPR)2016
- T-Update: A tree-structured update scheme with top-down transmission in erasure-coded systems2016
- Parity-Switched Data Placement: Optimizing Partial Stripe Writes in XOR-Coded Storage SystemsIEEE Transactions on Parallel and Distributed Systems, 2016
- Locally Repairable CodesIEEE Transactions on Information Theory, 2014
- The quantcast file systemProceedings of the VLDB Endowment, 2013
- Understanding network failures in data centers2011
- Write off-loadingACM Transactions on Storage, 2008
- Erasure Coding Vs. Replication: A Quantitative Comparison2002
- Parity logging overcoming the small write problem in redundant disk arraysACM SIGARCH Computer Architecture News, 1993
- Polynomial Codes Over Certain Finite FieldsJournal of the Society for Industrial and Applied Mathematics, 1960