A Front-end, Hadoop-based Data Management Service for Efficient Federated Clouds
Authors:
- George KousiourisGKGeorge Kousiouris
- George VafiadisGVGeorge Vafiadis
- Theodora VarvarigouTVTheodora Varvarigou
- 1 November 2011
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 511-516
Abstract
In the recent years, cloud computing has emerged as the new IT paradigm that promises elastic resources on a pay-per-use basis. The challenges of cloud computing are focused around massive data storage and efficient large scale distributed computation. Hadoop, a community driven Apache project has provided an efficient and cost effective platform for large scale computation using the map-reduce methodology, pioneered by Google. In this paper, the design of a Hadoop-based data management system as the front-end service for Cloud data management is investigated. This framework is enriched with Restful APIs in front of Hadoop and a series of components that aim to extend Hadoop's functionality beyond its well known back-end, heavy data processing scope. These components are used to enrich security, logging and data analysis features and also data access compatibility between different but interconnected Cloud providers (federated Clouds). Hadoop capabilities are also extended in a quest for intelligent decision making regarding the choice of the fittest services for federation in a federated cloud scenario, in addition to legally compliant behaviour regarding the geographical location of data storage.Keywords
This publication has 14 references indexed in Scilit:
- OPTIMIS: A holistic approach to cloud service provisioningFuture Generation Computer Systems, 2012
- The effects of scheduling, workload type and consolidation scenarios on virtual machine performance and their prediction through optimized artificial neural networksJournal of Systems and Software, 2011
- Benchmarking cloud-based data management systems2010
- Enhancing UNICORE Storage Management Using Hadoop Distributed File System2010
- A Peer-to-Peer Framework for Supporting MapReduce Applications in Dynamic Cloud Environments2010
- Cloud computing: state-of-the-art and research challengesJournal of Internet Services and Applications, 2010
- Design a cloud storage platform for pervasive computing environmentsCluster Computing, 2009
- Using Hadoop as a grid storage elementJournal of Physics: Conference Series, 2009
- Write off-loadingACM Transactions on Storage, 2008
- Long-term time series prediction with the NARX network: An empirical evaluationNeurocomputing, 2008