Controllable Mixture-of-Experts for Multivariate Soft Sensors
- 11 August 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 22 (15455955) , 19789-19800
Abstract
Multivariate soft sensors are critical in industrial manufacturing for providing precise estimations of multiple quality variables and ensuring data reliability and completeness. Existing methods predominantly focus on parameter-sharing architectures while overlooking two fundamental issues: 1) catastrophic interference, where sharing parameters across all tasks leads to performance degradation; 2) uncontrollable optimization, where the optimizer lacks controllability over task priorities, misleading specific tasks converge to unexpected values. To handle these issues and enhance multivariate modeling, we propose a controllable Mixture-of-Experts (ControlMoE) framework for industrial soft sensors, consisting of a Mixture-of-Sequential-Experts (MoSE) module and a Proportional-Integral-Derivative Calibrating (PIDC) module. The MoSE module integrates sequential expert networks with task-specific gating networks, enabling parameter sharing while preserving task distinctions to mitigate catastrophic interference. The PIDC module employs an innovative PID controller to dynamically calibrate task learning weights, ensuring expected outcomes through feedback control and enhancing optimization controllability. Evaluations on two industrial datasets from Chinese nuclear monitoring stations demonstrate that ControlMoE achieves superior accuracy and controllability in multivariate soft sensor modeling. Note to Practitioners—This paper tackles the challenges in applying multivariate soft sensors within industrial manufacturing processes, particularly in environments such as nuclear monitoring where precision and control over multiple quality variables are critical. Traditional methods often suffer from performance degradation due to parameter sharing across all tasks-known as catastrophic interference–and lack control in optimizing multiple tasks simultaneously, leading to suboptimal outcomes. Our proposed framework, ControlMoE, introduces a novel architecture that combines sequential expert networks with a dynamic weight adjustment mechanism using PID controllers. This enables precise control over task-specific optimizations, ensuring that the desired importance and sequence of tasks are maintained, thereby enhancing both accuracy and operational control. We believe this framework could be widely applicable in various industrial settings where multivariate monitoring and control are necessary, improving not only performance but also the reliability of the data produced for critical decision-making.Keywords
Funding Information
- National Natural Science Foundation of China (62073288, 12075212)
This publication has 59 references indexed in Scilit:
- Co-attentive multi-task convolutional neural network for facial expression recognitionPattern Recognition, 2022
- Robust probabilistic principal component regression with switching mixture Gaussian noise for soft sensingChemometrics and Intelligent Laboratory Systems, 2022
- An ensemble method to forecast 24-h ahead solar irradiance using wavelet decomposition and BiLSTM deep learning networkEarth Science Informatics, 2021
- Extreme gradient boosting and deep neural network based ensemble learning approach to forecast hourly solar irradianceJournal of Cleaner Production, 2021
- Temporal convolutional neural (TCN) network for an effective weather forecasting using time-series data from the local weather stationSoft Computing, 2020
- Deep learning for quality prediction of nonlinear dynamic processes with variable attention‐based long short‐term memory networkThe Canadian Journal of Chemical Engineering, 2019
- Soft sensor based on stacked auto-encoder deep neural network for air preheater rotor deformation predictionAdvanced Engineering Informatics, 2018
- A reduced order soft sensor approach and its application to a continuous digesterJournal of Process Control, 2011
- Precision Manufacturing2008
- Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem1989