Distributed Data Stream Processing and Edge Computing: A Survey on Resource Elasticity and Future Directions
Authors:
- Marcos Dias de AssuncaoMAMarcos Dias de Assuncao
- Alexandre da Silva VeithAVAlexandre da Silva Veith
Preprint
Abstract
Under several emerging application scenarios, such as in smart cities, operational monitoring of large infrastructure, wearable assistance, and Internet of Things, continuous data streams must be processed under very short delays. Several solutions, including multiple software engines, have been developed for processing unbounded data streams in a scalable and efficient manner. More recently, architecture has been proposed to use edge computing for data stream processing. This paper surveys state of the art on stream processing engines and mechanisms for exploiting resource elasticity features of cloud computing in stream processing. Resource elasticity allows for an application or service to scale out/in according to fluctuating demands. Although such features have been extensively investigated for enterprise applications, stream processing poses challenges on achieving elastic systems that can make efficient resource management decisions based on current load. Elasticity becomes even more challenging in highly distributed environments comprising edge and cloud computing resources. This work examines some of these challenges and discusses solutions proposed in the literature to address them.Keywords
All Related Versions
This publication has 37 references indexed in Scilit:
- Joint flow and virtual machine placement in hybrid cloud data centersJournal of Network and Computer Applications, 2017
- The design of the gateway for the Cloud of ThingsAnnals of Telecommunications, 2016
- Resource management for bursty streams on multi-tenancy cloud environmentsFuture Generation Computer Systems, 2016
- Dynamic energy-aware cloudlet-based mobile cloud computing model for green computingJournal of Network and Computer Applications, 2016
- JetStream: Enabling high throughput live event streaming on multi-site cloudsFuture Generation Computer Systems, 2016
- Improving the predictability of distributed stream processorsFuture Generation Computer Systems, 2015
- A Review of Auto-scaling Techniques for Elastic Applications in Cloud EnvironmentsJournal of Grid Computing, 2014
- Scheduling linear chain streaming applications on heterogeneous systems with failuresFuture Generation Computer Systems, 2013
- Efficient Dynamic Operator Placement in a Locally Distributed Continuous Query System2006
- Aurora: a new model and architecture for data stream managementThe VLDB Journal, 2003