“Saksham Model” Performance Improvisation Using Node Capability Evaluation in Apache Hadoop
- 1 January 2021
- book chapter
- Published by IGI Global Scientific Publishing
- p. 1282-1302
Abstract
Big Data processing and analysis requires tremendous processing capability. Distributed computing brings many commodity systems under the common platform to answer the need for Big Data processing and analysis. Apache Hadoop is the most suitable set of tools for Big Data storage, processing, and analysis. But Hadoop found to be inefficient when it comes to heterogeneous set computers which have different processing capabilities. In this research, we propose the Saksham model which optimizes the processing time by efficient use of node processing capability and file management. The proposed model shows the performance improvement for Big Data processing. To achieve better performance, Saksham model uses two vital aspects of heterogeneous distributed computing: Effective block rearrangement policy and use of node processing capability. The results demonstrate that the proposed model successfully achieves better job execution time and improves data locality.Keywords
This publication has 13 references indexed in Scilit:
- Load Balancing through Block Rearrangement Policy for Hadoop Heterogeneous Cluster2018
- Tula: A disk latency aware balancing and block placement strategy for Hadoop2017
- An Improved Replica Placement Policy for Hadoop Distributed File System Running on Cloud Platforms2017
- A dynamic replica strategy based on Markov model for hadoop distributed file system (HDFS)2016
- CoS-HDFS2016
- A speculative approach to spatial‐temporal efficiency with multi‐objective optimization in a heterogeneous cloud environmentSecurity and Communication Networks, 2016
- An efficient Hadoop data replication method design for heterogeneous clusters2016
- A sliding window‐based dynamic load balancing for heterogeneous Hadoop clustersConcurrency and Computation: Practice and Experience, 2016
- Load Rebalancing for Distributed File Systems in CloudsIEEE Transactions on Parallel and Distributed Systems, 2012
- MapReduceCommunications of the ACM, 2008