Big Data Real-Time Processing Based on Storm
Authors:
- Wenjie YangWYWenjie Yang
- Xingang LiuXLXingang Liu
- Lan ZhangLZLan Zhang
- Laurence T. YangLYLaurence T. Yang
- 1 July 2013
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 2324898X,p. 1784-1787
Abstract
As the growth of Internet, Cloud Computing, Mobile Network and Internet of Things is increasing rapidly, Big Data is becoming a hot-spot in recent years. Big Data Processing is involved in our daily life such as mobile devices, RFID and wireless sensors, which aims at dealing with billions of users' interactive data. At the same time, real-time processing is eagerly needed in integrated system. In this paper, several technologies associated with real-time big data processing are introduced, among which the core technology called Storm is emphasized. An entire system is built based on Storm, associated with RabbitMQ, NoSQL and JSP. To ensure the practical applicability and high efficiency, a simulation system is established and shows acceptable performance in various expressions using data sheet and Ganglia. It is proved that the big data real-time processing based on Storm can be widely used in various computing environment.Keywords
This publication has 6 references indexed in Scilit:
- Big Data Processing in Cloud Computing Environments2012
- Driving big data with big compute2012
- Sedna: A Memory Based Key-Value Storage System for Realtime Processing in Cloud2012
- Active Status Certificate Publish and Subscribe Based on AMQP2011
- Real-time data processing for an advanced imaging system using the Xilinx Virtex-5 FPGA2010
- Design and evaluation of benchmarks for financial applications using Advanced Message Queuing Protocol (AMQP) over InfiniBand2008