An Intermediate Data Partition Algorithm for Skew Mitigation in Spark Computing Environment
Authors:
- Zhuo TangZTZhuo Tang
- Wei LvWLWei Lv
- Kenli LiKLKenli Li
- Keqin LiKeqin Li
- 31 October 2018
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cloud Computing
- Vol. 9 (2) , 461-474
Abstract
No abstract availableKeywords
Funding Information
- National Natural Science Foundation of China (61572176, L1624040, L182400035, 61873090)
- National Key Research and Development Program of China (2017YFB0202201)
This publication has 25 references indexed in Scilit:
- LIBRA: Lightweight Data Skew Mitigation in MapReduceIEEE Transactions on Parallel and Distributed Systems, 2014
- SkewTune2012
- Parallel data processing with MapReduceACM SIGMOD Record, 2012
- Purlieus2011
- Resource-Aware Adaptive Scheduling for MapReduce Clusters2011
- Efficient outer join data skew handling in parallel DBMSProceedings of the VLDB Endowment, 2009
- Handling data skew in parallel joins in shared-nothing systems2008
- Optimal Skewed Data Allocation on Multiple Channels with Flat Broadcast per ChannelIEEE Transactions on Computers, 2005
- A symmetric fragment and replicate algorithm for distributed joinsIEEE Transactions on Parallel and Distributed Systems, 1993
- Faster methods for random samplingCommunications of the ACM, 1984