Enhancing Control in Chemical Processes using Reinforcement from Human Feedback
- 19 June 2026
- conference paper
- Published by The CACHE Foundation, Inc.
- Vol. 6, 2457-2465
Abstract
Reinforcement learning (RL) presents a promising alternative to model-based advanced control schemes, such as model predictive control (MPC), whose application can be limited by highly complex system models. However, incorporating constraints in RL remains challenging and formulating a suitable optimization objective is not straightforward. Reinforcement learning from human feedback (RLHF) offers an approach to derive the RL reward function from human expert preferences, enabling the incorporation of process knowledge. In this work, we present the application of RLHF to fine-tune an approximate MPC controller with suboptimal performance. We demonstrate that combining conventional reward formulations with RLHF, along with varying trajectory segment lengths for collecting human feedback, improves the control methodology for a batch bioreactor by enhancing safety and accounting for long-term effects. Furthermore, direct-preference based policy optimization (DPPO) represents a promising alternative for directly fine-tuning learning-based controllers while circumventing explicit reward model design.Keywords
This publication has 17 references indexed in Scilit:
- A Survey of Reinforcement Learning from Human FeedbackarXiv, 2025
- Improving Reinforcement Learning from Human Feedback with Efficient Reward Model EnsemblearXiv, 2024
- Reward Model Ensembles Help Mitigate OveroptimizationarXiv, 2024
- The Alignment Ceiling: Objective Mismatch in Reinforcement Learning from Human FeedbackarXiv, 2024
- Open Problems and Fundamental Limitations of Reinforcement Learning from Human FeedbackarXiv, 2023
- Efficient Representation and Approximation of Model Predictive Control Laws via Deep LearningIEEE Transactions on Cybernetics, 2020
- Continuous control with deep reinforcement learningarXiv, 2019
- Learning Reward Functions by Integrating Human Demonstrations and Preferences2019
- Deep Reinforcement Learning That MattersProceedings of the AAAI Conference on Artificial Intelligence, 2018
- Dynamic optimization of batch processesComputers & Chemical Engineering, 2003