Efficient Edge Data Management Framework for IIoT via Prediction-Based Data Reduction
- 26 October 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 34 (12) , 3309-3322
Abstract
Large amounts of time series data are required to support data analysis at the edge in the end-edge-cloud Industrial Internet of Things (IIoT) architecture. Reducing the storage cost is one of the main challenges in edge data management due to the limited storage resource of edge nodes. The state-of-the-art data reduction method has a high time overhead and poor reduction efficiency for unstable data sets. To solve this problem, this study proposes a time-series data management framework that combines data partition and data compression techniques. For the data partition technique, we propose an adaptive selection strategy to integrate the access pattern of the application and the characteristics of the time series data, thereby improving the partition accuracy. For the data compression technique, we propose a compression scheme based on time series data segmentation by using the idea of divide and conquer; we further introduce a change point detection technique to improve the compression efficiency for unstable data sets. Experimental results obtained with three types of real industrial data sets show that our framework is significantly better than the state-of-the-art method in terms of compression ratio and time overhead.Keywords
Funding Information
- National Natural Science Foundation of China (62372167, 61972139, 62141212, 62133014)
- Natural Science Foundation of Hunan Province (2021JJ30153, 2022JJ10021)
- Natural Science Foundation of Chongqing Municipality (cstc2021jcyj-msxmX0817, cstc2021jcyj-msxmX0461, CSTB2022NSCQ-MSX1393)
- chongqing Science and Technology Innovation Project (CQYC20220511500)
This publication has 34 references indexed in Scilit:
- QoE-driven big data management in pervasive edge computing environmentBig Data Mining and Analytics, 2018
- Changepoint Detection in the Presence of OutliersJournal of the American Statistical Association, 2018
- Secure Data Storage and Searching for Industrial IoT by Integrating Fog Computing and Cloud ComputingIEEE Transactions on Industrial Informatics, 2018
- Hybrid compression technique with data segmentation for electroencephalography data2017
- Battery Status Sensing Software-Defined Multicast for V2G Regulation in Smart GridIEEE Sensors Journal, 2017
- Efficient Edge Storage Management Based on Near Real-Time Forecasts2017
- A New Deep Learning-Based Food Recognition System for Dietary Assessment on An Edge Computing Service InfrastructureIEEE Transactions on Services Computing, 2017
- Data Reduction in Wireless Sensor Networks: A Hierarchical LMS Prediction ApproachIEEE Sensors Journal, 2015
- IoT Data Compression: Sensor-Agnostic Approach2015
- A time-series compression technique and its application to the smart gridThe VLDB Journal, 2014