Optimization of the Use of Cloud Computing Resources Using Exploratory Data Analysis and Machine Learning
Open Access
- 1 July 2024
- journal article
- research article
- Published by Walter de Gruyter GmbH in Journal of Artificial Intelligence and Soft Computing Research
- Vol. 14 (4) , 287-308
Abstract
Rapid growth in the popularity of cloud computing has been largely caused by increasing demand for scalable IT solutions, which could provide a cost-effective way to manage the software development process and meet business objectives. Optimization of cloud resource usage remains a key issue given its potential to significantly increase efficiency and flexibility, minimize costs, ensure security, and maintain high availability of services. This paper presents a novel concept of a Cloud Computing Resource Prediction and Optimization System, which is based on exploratory data analysis that acknowledges, among others, the information value of outliers and dynamic feature selection. The optimization of cloud resource usage relies on long-term forecasting, which is considered a dynamic and proactive optimization category. The analysis presented here focuses on the applicability of classical statistical models, XGBoost, neural networks and Transformer. Experimental results reveal that machine learning methods are highly effective in long-term forecasting. Particularly promising results – in the context of potential prediction-based dynamic resource reservations – have been yielded by prediction methods based on the BiGRU neural network and the Temporal Fusion Transformer.Keywords
This publication has 32 references indexed in Scilit:
- Interpretable Anomaly Detection with DIFFI: Depth-based feature importance of Isolation ForestEngineering Applications of Artificial Intelligence, 2022
- Movement forecasting of financial time series based on adaptive LSTM-BN networkExpert Systems with Applications, 2022
- Anomaly detection in the context of long-term cloud resource usage planningKnowledge and Information Systems, 2022
- Machine Learning Based Adaptive Auto-scaling Policy for Resource Orchestration in Kubernetes Clusters2022
- XGBoost and Deep Neural Network Comparison: The Case of Teams’ Performance2021
- VM Reservation Plan Adaptation Using Machine Learning in Cloud ComputingJournal of Grid Computing, 2019
- Holt-Winters Forecasting2019
- Workload prediction in cloud using artificial neural network and adaptive differential evolutionFuture Generation Computer Systems, 2018
- A note on the validity of cross-validation for evaluating autoregressive time series predictionComputational Statistics & Data Analysis, 2018
- A review and comparison of strategies for multi-step ahead time series forecasting based on the NN5 forecasting competitionExpert Systems with Applications, 2012