A Metadata and Z Score-based Load-Shedding Technique in IoT-based Data Collection Systems
- 1 February 2021
- journal article
- research article
- Published by Ram Arti Publishers in International Journal of Mathematical, Engineering and Management Sciences
- Vol. 6 (1) , 363-382
Abstract
The Internet-of-Things (IoT) has emerged as an alternative to communicate different pieces of technology to foster the distributed data collection. The measurement projects and the Real-time data processing are articulated to take advantage of this environment, fostering a sustainable data-driven decision making. The Data Stream Processing Strategy (DSPS) is a Stream Processing Engine focused on measurement projects, where each concept is previously agreed through a measurement framework. The Measurement Adapter (MA) is a component whose responsibility is to pair each metric's definition from the measurement project with data sensors to transmit data (i.e., measures) with metadata (i. e., tags indicating the data meaning) together. The Gathering Function (GF) receives and derivates data for its processing from each MA, while it implements load-shedding (LS) techniques based on Metadata to avoid a processing collapse when all MAs informs jointly and frequently. Here, a Metadata and Z-score based load-shedding technique implemented locally in the MA is proposed. Thus, the load-shedding is located at the same data source to avoid data transmission and saving resources. Also, an incremental estimation of average, deviations, covariance, and correlations are implemented and employed to calculate the Z-scores and to data retain/discard selectively. Four simulations discrete were designed and performed to analyze the proposal. Results indicate that the local LS required only 24% of the original data transmissions, a minimum of 18.61 ms as the data lifespan, while it consumes 890.26 KB. As future work, other kinds of dependencies analysis will be analyzed to provide local alternatives to LS.Keywords
This publication has 16 references indexed in Scilit:
- New Landscapes of the Data Stream Processing in the era of Fog ComputingFuture Generation Computer Systems, 2019
- Resource Management Approaches in Fog Computing: a Comprehensive ReviewJournal of Grid Computing, 2019
- An Architecture for the Real-Time Data Stream Monitoring in IoT2019
- Survey of cross‐technology communication for IoT heterogeneous devicesIET Communications, 2019
- A survey of Internet of Things communication using ICN: A use case perspectiveComputer Communications, 2019
- Production Application Performance Data Streaming for System MonitoringACM Transactions on Modeling and Performance Evaluation of Computing Systems, 2019
- A Kind of Distributed Fusion Incremental Kalman Filter2018
- Benchmarking real-time vehicle data streaming models for a smart cityInformation Systems, 2017
- Optimized on-demand data streaming from sensor nodes2017
- A self-tuning filterJournal of Process Control, 1997