Modeling and Evaluation of Attention Mechanism Neural Network Based on Industrial Time Series Data
Open Access
Abstract
Chemical process control systems are complex, and modeling the controlled object is the first task in automatic control and optimal design. Most chemical process modeling experiments require test signals to be applied to the process, which may lead to production interruptions or cause safety accidents. Therefore, this paper proposes an improved transformer model based on a self-attention mechanism for modeling industrial processes. Then, an evaluation mechanism based on root mean square error (RMSE) and Kullback–Leibler divergence (KLD) metrics is designed to obtain more appropriate model parameters. The Variational Auto-Encoder (VAE) network is used to compute the associated KLD. Finally, a real nonlinear dynamic process in the petrochemical industry is modeled and evaluated using the proposed methodology to predict the time series data of the process. This study demonstrates the validity of the proposed transformer model and illustrates the versatility of using an integrated modeling, evaluation, and prediction scheme for nonlinear dynamic processes in process industries. The scheme is of great importance for the field of industrial soft measurements as well as for deep learning-based time series prediction. In addition, the issue of a suitable time domain for the prediction is discussed.Keywords
Funding Information
- National Natural Science Foundation of China (61703434, 2462020YXZZ023)
- Science Foundation of China University of Petroleum, Beijing (61703434, 2462020YXZZ023)
This publication has 37 references indexed in Scilit:
- Multimodal imputation-based stacked ensemble for prediction and classification of air quality index in Indian citiesComputers and Electrical Engineering, 2024
- Intelligent identification method of chemical processes based on maximum mean discrepancy domain generalizationJournal of the Taiwan Institute of Chemical Engineers, 2023
- A multi-head attention-based transformer model for traffic flow forecasting with a comparative analysis to recurrent neural networksExpert Systems with Applications, 2022
- Dynamic historical information incorporated attention deep learning model for industrial soft sensor modelingAdvanced Engineering Informatics, 2022
- Wiener structure based adaptive control for dynamic processes with approximate monotonic nonlinearitiesJournal of the Franklin Institute, 2020
- Recognition of geochemical anomalies using a deep variational autoencoder networkApplied Geochemistry, 2020
- Toward a low cost and high performance MPC: The role of system identificationComputers & Chemical Engineering, 2013
- Restoration of particle size distributions from fiber-optical in-line measurements in fluidized bed processesChemical Engineering Science, 2011
- Data-driven Soft Sensors in the process industryComputers & Chemical Engineering, 2009
- Industrial applications of model based predictive controlAutomatica, 1993