Wenqi Cai
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Wenqi Cai has a total of 14 co-authors. Here are the top co-authors by number of shared publications:
- Ibrahima Ndoye
Environmental Science and Engineering Program, Biological and Environmental Science and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia
3 shared publications - Taous-Meriem Laleg-Kirati
The National Institute for Research in Digital Science and Technology, Paris-Saclay, France
3 shared publications - Boon S. Ooi
King Abdullah University of Science and Technology (KAUST)
2 shared publications - Mohamed-Slim Alouini
King Abdullah University of Science and Technology
2 shared publications - Sebastien Gros
NTNU
2 shared publications Abdullah N. Alhashim
King Abdullah University of Science and Technology (KAUST),Thuwal 23955-6900, Saudi Arabia
1 shared publication
Latest Publications
- Open Access
- 22 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 145264-145280
A Learning-Based Model Predictive Control Strategy for Home Energy Management Systems
This paper presents a model predictive control (MPC)-based reinforcement learning (RL) approach for a home energy management system (HEMS). The house consists of an air-to-water heat pump connected to a hot water tank that supplies thermal energy to a water-based floor heating system. Additionally, it includes a photovoltaic (PV) array and a battery storage system. The HEMS is supposed to exploit the house thermal inertia and battery storage to shift demand from peak hours to off-peak periods and earn benefits by selling excess energy to the utility grid during periods of high electricity prices. However, designing such a HEMS is challenging because the discrepancies due to model mismatch make erroneous predictions of the system dynamics, leading to a non-optimal decision making. Besides, uncertainties in the house thermodynamics, misprediction in the forecasting of PV generation, outdoor temperature, and user load demand make the problem more challenging. We solve this issue by approximating the optimal policy by a parameterized MPC scheme and updating the parameters via a compatible delayed deterministic actor-critic (with gradient Q-learning critic, i.e., CDDAC-GQ) algorithm. Simulation results show that the proposed MPC-based RL HEMS can effectively deliver a policy that satisfies both indoor thermal comfort and economic costs even in the case of inaccurate model and system uncertainties. Furthermore, we conduct a thorough comparison between the CDDAC-GQ algorithm and the conventional twin delayed deep deterministic policy gradient (TD3) algorithm, the results of which affirm the efficacy of our proposed method in addressing complex HEMS problems.
- 22 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 145264-145280
Open AccessA Learning-Based Model Predictive Control Strategy for Home Energy Management Systems
This paper presents a model predictive control (MPC)-based reinforcement learning (RL) approach for a home energy management system (HEMS). The house consists of an air-to-water heat pump connected to a hot water tank that supplies thermal energy to a water-based floor heating system. Additionally, it includes a photovoltaic (PV) array and a battery storage system. The HEMS is supposed to exploit the house thermal inertia and battery storage to shift demand from peak hours to off-peak periods and earn benefits by selling excess energy to the utility grid during periods of high electricity prices. However, designing such a HEMS is challenging because the discrepancies due to model mismatch make erroneous predictions of the system dynamics, leading to a non-optimal decision making. Besides, uncertainties in the house thermodynamics, misprediction in the forecasting of PV generation, outdoor temperature, and user load demand make the problem more challenging. We solve this issue by approximating the optimal policy by a parameterized MPC scheme and updating the parameters via a compatible delayed deterministic actor-critic (with gradient Q-learning critic, i.e., CDDAC-GQ) algorithm. Simulation results show that the proposed MPC-based RL HEMS can effectively deliver a policy that satisfies both indoor thermal comfort and economic costs even in the case of inaccurate model and system uncertainties. Furthermore, we conduct a thorough comparison between the CDDAC-GQ algorithm and the conventional twin delayed deep deterministic policy gradient (TD3) algorithm, the results of which affirm the efficacy of our proposed method in addressing complex HEMS problems.
- Open Access
Quasi-Newton Iteration in Deterministic Policy Gradient
This paper presents a model-free approximation for the Hessian of the performance of deterministic policies to use in the context of Reinforcement Learning based on Quasi-Newton steps in the policy parameters. We show that the approximate Hessian converges to the exact Hessian at the optimal policy, and allows for a superlinear convergence in the learning, provided that the policy parametrization is rich. The natural policy gradient method can be interpreted as a particular case of the proposed method. We analytically verify the formulation in a simple linear case and compare the convergence of the proposed method with the natural policy gradient in a nonlinear example.
Open AccessQuasi-Newton Iteration in Deterministic Policy Gradient
This paper presents a model-free approximation for the Hessian of the performance of deterministic policies to use in the context of Reinforcement Learning based on Quasi-Newton steps in the policy parameters. We show that the approximate Hessian converges to the exact Hessian at the optimal policy, and allows for a superlinear convergence in the learning, provided that the policy parametrization is rich. The natural policy gradient method can be interpreted as a particular case of the proposed method. We analytically verify the formulation in a simple linear case and compare the convergence of the proposed method with the natural policy gradient in a nonlinear example.
- 1 June 2021
- Published by Elsevier in IFAC Journal of Systems and Control
- Vol. 16
Reduction of the beam pointing error for improved free-space optical communication link performance
- 1 June 2021
- Published by Elsevier in IFAC Journal of Systems and Control
- Vol. 16
Reduction of the beam pointing error for improved free-space optical communication link performance
- Open Access
- 1 January 2020
- Published by Elsevier in IFAC-PapersOnLine
- Vol. 53 (2) , 5368-5374
An Extremum Seeking Control Based Approach for Alignment Problem of Mobile Optical Communication Systems
- 1 January 2020
- Published by Elsevier in IFAC-PapersOnLine
- Vol. 53 (2) , 5368-5374
Open AccessAn Extremum Seeking Control Based Approach for Alignment Problem of Mobile Optical Communication Systems
- Open Access
- 4 October 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Photonics Journal
- Vol. 11 (6) , 1-13
Modeling and Experimental Study of The Vibration Effects in Urban Free-Space Optical Communication Systems
Free-space optical (FSO) communication, considered as a last-mile technology, is widely used in many urban scenarios. However, the performance of urban free-space optical (UFSO) communication systems fades in the presence of system vibration caused by many factors in the chaotic urban environment. In this paper, we develop a dedicated indoor vibration platform and atmospheric turbulence to estimate the Bifurcated-Gaussian (B-G) distribution model of the receiver optical power under different vibration levels and link distances using nonlinear iteration method. Mean square error (MSE) and coefficient of determination ( $R^2$ ) metrics have been used to show a good agreement between the PDFs of the experimental data with the resulting B-G distribution model. Besides, the UFSO channel under the effects of both vibration and atmospheric turbulence is also explored under three atmospheric turbulence conditions. Our proposed B-G distribution model describes the vibrating UFSO channels properly and can easily help to perform and evaluate the link performance of UFSO systems, e.g., bit-error-rate (BER), outage probability. Furthermore, this work paves the way for constructing completed auxiliary control subsystems for robust UFSO links and contributes to more extensive optical communication scenarios, such as underwater optical communication, etc.- 4 October 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Photonics Journal
- Vol. 11 (6) , 1-13
Open AccessModeling and Experimental Study of The Vibration Effects in Urban Free-Space Optical Communication Systems
Free-space optical (FSO) communication, considered as a last-mile technology, is widely used in many urban scenarios. However, the performance of urban free-space optical (UFSO) communication systems fades in the presence of system vibration caused by many factors in the chaotic urban environment. In this paper, we develop a dedicated indoor vibration platform and atmospheric turbulence to estimate the Bifurcated-Gaussian (B-G) distribution model of the receiver optical power under different vibration levels and link distances using nonlinear iteration method. Mean square error (MSE) and coefficient of determination ( $R^2$ ) metrics have been used to show a good agreement between the PDFs of the experimental data with the resulting B-G distribution model. Besides, the UFSO channel under the effects of both vibration and atmospheric turbulence is also explored under three atmospheric turbulence conditions. Our proposed B-G distribution model describes the vibrating UFSO channels properly and can easily help to perform and evaluate the link performance of UFSO systems, e.g., bit-error-rate (BER), outage probability. Furthermore, this work paves the way for constructing completed auxiliary control subsystems for robust UFSO links and contributes to more extensive optical communication scenarios, such as underwater optical communication, etc.