WCWenqi Cai

Wenqi Cai

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Wenqi Cai has a total of 14 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    A Learning-Based Model Predictive Control Strategy for Home Energy Management Systems

    by Wenqi Cai, Shambhuraj Sawant, Dirk Reinhardt, Soroush Rastegarpour, Sébastien Gros

    This paper presents a model predictive control (MPC)-based reinforcement learning (RL) approach for a home energy management system (HEMS). The house consists of an air-to-water heat pump connected to a hot water tank that supplies thermal energy to a water-based floor heating system. Additionally, it includes a photovoltaic (PV) array and a battery storage system. The HEMS is supposed to exploit the house thermal inertia and battery storage to shift demand from peak hours to off-peak periods and earn benefits by selling excess energy to the utility grid during periods of high electricity prices. However, designing such a HEMS is challenging because the discrepancies due to model mismatch make erroneous predictions of the system dynamics, leading to a non-optimal decision making. Besides, uncertainties in the house thermodynamics, misprediction in the forecasting of PV generation, outdoor temperature, and user load demand make the problem more challenging. We solve this issue by approximating the optimal policy by a parameterized MPC scheme and updating the parameters via a compatible delayed deterministic actor-critic (with gradient Q-learning critic, i.e., CDDAC-GQ) algorithm. Simulation results show that the proposed MPC-based RL HEMS can effectively deliver a policy that satisfies both indoor thermal comfort and economic costs even in the case of inaccurate model and system uncertainties. Furthermore, we conduct a thorough comparison between the CDDAC-GQ algorithm and the conventional twin delayed deep deterministic policy gradient (TD3) algorithm, the results of which affirm the efficacy of our proposed method in addressing complex HEMS problems.


  • Open Access

    Quasi-Newton Iteration in Deterministic Policy Gradient

    by Arash Bahari Kordabad, Hossein Nejatbakhsh Esfahani, Wenqi Cai, Sebastien Gros

    This paper presents a model-free approximation for the Hessian of the performance of deterministic policies to use in the context of Reinforcement Learning based on Quasi-Newton steps in the policy parameters. We show that the approximate Hessian converges to the exact Hessian at the optimal policy, and allows for a superlinear convergence in the learning, provided that the policy parametrization is rich. The natural policy gradient method can be interpreted as a particular case of the proposed method. We analytically verify the formulation in a simple linear case and compare the convergence of the proposed method with the natural policy gradient in a nonlinear example.

  • Reduction of the beam pointing error for improved free-space optical communication link performance

    by Ibrahima N’dOye, Wenqi Cai, Asem Alalwan, Xiaobin Sun, Wael Ghazy Headary, Mohamed-Slim Alouini, Boon S. Ooi, Taous-Meriem Laleg-Kirati


  • Open Access

    An Extremum Seeking Control Based Approach for Alignment Problem of Mobile Optical Communication Systems

    by Wenqi Cai, Abdullah N. Alhashim, Ibrahima N’dOye, Taous Meriem Laleg-Kirati


  • Open Access

    Modeling and Experimental Study of The Vibration Effects in Urban Free-Space Optical Communication Systems

    by Wenqi Cai, Ibrahima N'DOye, Boon S. Ooi, Mohamed-Slim Alouini, Taous Meriem Laleg-Kirati

    Free-space optical (FSO) communication, considered as a last-mile technology, is widely used in many urban scenarios. However, the performance of urban free-space optical (UFSO) communication systems fades in the presence of system vibration caused by many factors in the chaotic urban environment. In this paper, we develop a dedicated indoor vibration platform and atmospheric turbulence to estimate the Bifurcated-Gaussian (B-G) distribution model of the receiver optical power under different vibration levels and link distances using nonlinear iteration method. Mean square error (MSE) and coefficient of determination ( $R^2$ ) metrics have been used to show a good agreement between the PDFs of the experimental data with the resulting B-G distribution model. Besides, the UFSO channel under the effects of both vibration and atmospheric turbulence is also explored under three atmospheric turbulence conditions. Our proposed B-G distribution model describes the vibrating UFSO channels properly and can easily help to perform and evaluate the link performance of UFSO systems, e.g., bit-error-rate (BER), outage probability. Furthermore, this work paves the way for constructing completed auxiliary control subsystems for robust UFSO links and contributes to more extensive optical communication scenarios, such as underwater optical communication, etc.