IoT-Based Energy Consumption Prediction Using Transformers
Open Access
- 29 June 2024
- journal article
- research article
- Published by Gazi University in Gazi University Journal of Science Part A: Engineering and Innovation
- Vol. 11 (2) , 304-323
Abstract
With the advancement of various IoT-based systems, the amount of data is steadily increasing. The increase of data on a daily basis is essential for decision-makers to assess current situations and formulate future policies. Among the various types of data, time-series data presents a challenging relationship between current and future dependencies. Time-series prediction aims to forecast future values of target variables by leveraging insights gained from past data points. Recent advancements in deep learning-based algorithms have surpassed traditional machine learning-based algorithms for time-series in IoT systems. In this study, we employ Enc & Dec Transformer, the latest advancements in neural networks for time-series prediction problems. The obtained results were compared with Encoder-only and Decoder-only Transformer blocks as well as well-known recurrent based algorithms, including 1D-CNN, RNN, LSTM, and GRU. To validate our approach, we utilize three different univariate time-series datasets collected on an hourly basis, focusing on energy consumption within IoT systems. Our results demonstrate that our proposed Transformer model outperforms its counterparts, achieving a minimum Mean Squared Error (MSE) of 0.020 on small, 0.008 on medium, and 0.006 on large-sized datasets.Keywords
This publication has 25 references indexed in Scilit:
- Persistence Initialization: a novel adaptation of the Transformer architecture for time series forecastingApplied Intelligence, 2023
- Temporal Fusion Transformers for interpretable multi-horizon time series forecastingInternational Journal of Forecasting, 2021
- Evaluation of the Transformer Architecture for Univariate Time Series Forecasting2021
- Remaining useful life estimation via transformer encoder enhanced by a gated convolutional unitJournal of Intelligent Manufacturing, 2021
- Energy consumption prediction by using machine learning for smart building: Case study in MalaysiaDevelopments in the Built Environment, 2021
- End-to-End Object Detection with Transformers2020
- Long short-term memory (LSTM) recurrent neural network for low-flow hydrological time series forecastingActa Geophysica, 2019
- Time series forecasting using artificial neural networks methodologies: A systematic reviewFuture Computing and Informatics Journal, 2018
- Multi-step Time Series Forecasting of Electric Load Using Machine Learning Models2018
- Hybrid of ARIMA and SVMs for Short-Term Load ForecastingEnergy Procedia, 2012