An Evaluation of Power-Proportional Data Placement for Hadoop Distributed File Systems
Authors:
- Hieu Hanh LeHLHieu Hanh Le
- Satoshi HikidaSHSatoshi Hikida
- Haruo YokotaHYHaruo Yokota
- 1 December 2011
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 752-759
Abstract
Power-saving storage in data centers is now gaining much interest because of the increase in power requirements. In particular, with the expansion of services requiring distributed-processing frameworks, power proportionality in power-aware file systems is attracting great attention from academia and industry. The concept of power proportionality is that a system should perform work in proportion to the energy it consumes. To provide this important characteristic, data placement methods that enable a system to operate in multiple gears, each containing a different number of active nodes, have been proposed. Of these, RABBIT, with its power proportional data placement, is a novel method that is implemented over a Hadoop Distributed File System (HDFS) and has been shown to be successful in guaranteeing power proportionality for read only tasks. However, the data layout used in RABBIT does not consider write access occurring when the system operates in low gear and there are inactive nodes. In this paper, to discover an appropriate approach for power-efficient distributed file systems, we evaluate the performance of the RABBIT and PARAID methods on read-only tasks and the cost to system performance of write access in low gear. The skewed data placement in PARAID, which was one of the first methods to suggest power proportionality in disk-based storage systems, is supposed to offer good performance when dealing with a frequently updated dataset.Keywords
This publication has 9 references indexed in Scilit:
- Sierra2011
- Dynamic right-sizing for power-proportional data centers2011
- Energy proportionality for disk storage using replication2011
- On the energy (in)efficiency of Hadoop clustersACM SIGOPS Operating Systems Review, 2010
- Write off-loadingACM Transactions on Storage, 2008
- GRAID: A Green RAID Storage Architecture with Improved Energy Efficiency and Reliability2008
- MapReduceCommunications of the ACM, 2008
- The Case for Energy-Proportional ComputingComputer, 2007
- The Google file system2003