Jianhong Wang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Jianhong Wang has a total of 28 co-authors. Here are the top co-authors by number of shared publications:
- Dawei Qiu
Nanyang Technological University
2 shared publications Goran Strbac
Department of Electrical and Electronic Engineering, Imperial College London, London, U.K.
2 shared publications- Chongqing Kang
Department of Electrical Engineering, Tsinghua University, Beijing, China
1 shared publication Congzhen Qiao
College of Chemistry and Chemical Engineering, Institute of Fine Chemistry and Engineering, Henan University, Kaifeng 475004, China
1 shared publicationCuiping Zhai
Henan University
1 shared publication- Dian Huang
College of Management and Economics, Tianjin University, Tianjin, China
1 shared publication
Latest Publications
- 20 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Power Systems
- Vol. 40 (3) , 2750-2763
Robust Reinforcement Learning for Decision Making Under Uncertainty in Electricity Markets
Reinforcement learning (RL) is a powerful tool for market agents solving decision-making problems in electricity markets. Vanilla RL enables agents to learn optimal policies in dynamic and uncertain market environments via trial and error. However, uncertainties in state transitions are often treated as exogenous state features with statistical errors. This approach can result in policies that are sensitive to perturbations of these uncertainties, potentially leading to performance degradation. This sensitivity is particularly critical in electricity markets, where the penetration of renewable energy and demand variability are increasing. To address this issue, this paper proposes a robust adversarial RL algorithm aimed at learning a robust optimal policy that accounts for market uncertainties in state transitions to systematically mitigate sensitivity to perturbations in uncertain environments. Specifically, we leverage the uncertainty set regularizer technique to define uncertainty sets within the parametric space of state transitions. Furthermore, we introduce a novel adversarial approach to generate unknown uncertainty sets using the value function as a basis. We finally conduct a comprehensive assessment of the robust adversarial RL algorithm across three electricity market applications: strategic bidding, retail pricing, and peer-to-peer energy trading, demonstrating significant improvements in robustness performance against various uncertainties.
- 20 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Power Systems
- Vol. 40 (3) , 2750-2763
Robust Reinforcement Learning for Decision Making Under Uncertainty in Electricity Markets
Reinforcement learning (RL) is a powerful tool for market agents solving decision-making problems in electricity markets. Vanilla RL enables agents to learn optimal policies in dynamic and uncertain market environments via trial and error. However, uncertainties in state transitions are often treated as exogenous state features with statistical errors. This approach can result in policies that are sensitive to perturbations of these uncertainties, potentially leading to performance degradation. This sensitivity is particularly critical in electricity markets, where the penetration of renewable energy and demand variability are increasing. To address this issue, this paper proposes a robust adversarial RL algorithm aimed at learning a robust optimal policy that accounts for market uncertainties in state transitions to systematically mitigate sensitivity to perturbations in uncertain environments. Specifically, we leverage the uncertainty set regularizer technique to define uncertainty sets within the parametric space of state transitions. Furthermore, we introduce a novel adversarial approach to generate unknown uncertainty sets using the value function as a basis. We finally conduct a comprehensive assessment of the robust adversarial RL algorithm across three electricity market applications: strategic bidding, retail pricing, and peer-to-peer energy trading, demonstrating significant improvements in robustness performance against various uncertainties.
- 16 May 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Power Systems
- Vol. 39 (2) , 3401-3416
Resilience-Oriented Coordination of Networked Microgrids: A Shapley Q-Value Learning Approach
High-impact and low-probability extreme events have occurred more frequently than before because of rapid climate change, which can seriously damage distribution systems. However, conventional distribution management can be dysfunctional after an event, destroying its centralized supervision towards resilience enhancement. In this context, networked microgrids (NMGs) with distributed energy resources provide a viable solution for the resilience enhancement of distribution systems. Existing literature tends to employ model-based optimization approaches for resilient operations of NMGs, which require complete system models and can be time-consuming. To address these challenges, this article suggests a decentralized framework for resilience-oriented coordination of NMGs and proposes a novel multi-agent reinforcement learning (MARL) method to solve it. Specifically, the proposed MARL method develops an efficient credit assignment scheme for NMGs to learn their contributions to the distribution system resilience via the Shapley Q-value technique with more efficient resilience enhancement. Case studies based on two modified IEEE 15- and 69-bus distribution networks are conducted to validate the effectiveness of the proposed MARL method in enabling effective coordination among NMGs and providing a high resilience level.
- 16 May 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Power Systems
- Vol. 39 (2) , 3401-3416
Resilience-Oriented Coordination of Networked Microgrids: A Shapley Q-Value Learning Approach
High-impact and low-probability extreme events have occurred more frequently than before because of rapid climate change, which can seriously damage distribution systems. However, conventional distribution management can be dysfunctional after an event, destroying its centralized supervision towards resilience enhancement. In this context, networked microgrids (NMGs) with distributed energy resources provide a viable solution for the resilience enhancement of distribution systems. Existing literature tends to employ model-based optimization approaches for resilient operations of NMGs, which require complete system models and can be time-consuming. To address these challenges, this article suggests a decentralized framework for resilience-oriented coordination of NMGs and proposes a novel multi-agent reinforcement learning (MARL) method to solve it. Specifically, the proposed MARL method develops an efficient credit assignment scheme for NMGs to learn their contributions to the distribution system resilience via the Shapley Q-value technique with more efficient resilience enhancement. Case studies based on two modified IEEE 15- and 69-bus distribution networks are conducted to validate the effectiveness of the proposed MARL method in enabling effective coordination among NMGs and providing a high resilience level.
- Open Access
- 1 January 2023
- Published by Wiley in International Journal of Intelligent Systems
- Vol. 2023 (1) , 1-14
Cooperative Game of Energy‐Constrained Agents in Wireless Communication Systems through Reinforcement Learning
Security issues are always considered in systems with wireless networks. However, few of them investigated covert signals existing on different communication channels to confuse advisories. In this paper, we consider the cooperation between two energy‐constrained agents, who could inject covert signals. First, the system performance is measured by Kullback–Leibler divergence (KLD) to avoid much deviation. Then, the cooperative game between two agents is considered, in which two agents share the common goal at confusing advisories. More formally, this cooperative game is formulated as a Markov decision process (MDP) and the most economic strategies are obtained through reinforcement learning (RL) under the imperfect information. Finally, the feasibility of theoretical results is demonstrated on the interconnected New England test system (NETS) as well as its reduced system.
- 1 January 2023
- Published by Wiley in International Journal of Intelligent Systems
- Vol. 2023 (1) , 1-14
Open AccessCooperative Game of Energy‐Constrained Agents in Wireless Communication Systems through Reinforcement Learning
Security issues are always considered in systems with wireless networks. However, few of them investigated covert signals existing on different communication channels to confuse advisories. In this paper, we consider the cooperation between two energy‐constrained agents, who could inject covert signals. First, the system performance is measured by Kullback–Leibler divergence (KLD) to avoid much deviation. Then, the cooperative game between two agents is considered, in which two agents share the common goal at confusing advisories. More formally, this cooperative game is formulated as a Markov decision process (MDP) and the most economic strategies are obtained through reinforcement learning (RL) under the imperfect information. Finally, the feasibility of theoretical results is demonstrated on the interconnected New England test system (NETS) as well as its reduced system.
- 23 December 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Smart Grid
- Vol. 14 (4) , 3176-3188
Blending Data and Physics Against False Data Injection Attack: An Event-Triggered Moving Target Defence Approach
Fast and accurate detection of cyberattacks is a key element for a cyber-resilient power system. Recently, data-driven detectors and physics-based Moving Target Defences (MTD) have been proposed to detect false data injection (FDI) attacks on state estimation. However, the uncontrollable false positive rate of the data-driven detector and the extra cost of frequent MTD usage limit their wide applications. Few works have explored the overlap between these two areas. To fill this gap, this paper proposes blending data-driven and physics-based approaches to enhance the detection performance. To start, a physics-informed data-driven attack detection and identification algorithm is proposed. Then, an MTD protocol is triggered by the positive alarm from the data-driven detector. The MTD is formulated as a bilevel optimisation to robustly guarantee its effectiveness against the worst-case attack around the identified attack vector. Meanwhile, MTD hiddenness is also improved so that the defence cannot be detected by the attacker. To guarantee feasibility and convergence, the convex two-stage reformulation is derived through duality and linear matrix inequality. The simulation results verify that blending data and physics can achieve extremely high detection rate while simultaneously reducing the false positive rate of the data-driven detector and the extra cost of MTD. All codes are available at https://github.com/xuwkk/DDET-MTD .
- 23 December 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Smart Grid
- Vol. 14 (4) , 3176-3188
Blending Data and Physics Against False Data Injection Attack: An Event-Triggered Moving Target Defence Approach
Fast and accurate detection of cyberattacks is a key element for a cyber-resilient power system. Recently, data-driven detectors and physics-based Moving Target Defences (MTD) have been proposed to detect false data injection (FDI) attacks on state estimation. However, the uncontrollable false positive rate of the data-driven detector and the extra cost of frequent MTD usage limit their wide applications. Few works have explored the overlap between these two areas. To fill this gap, this paper proposes blending data-driven and physics-based approaches to enhance the detection performance. To start, a physics-informed data-driven attack detection and identification algorithm is proposed. Then, an MTD protocol is triggered by the positive alarm from the data-driven detector. The MTD is formulated as a bilevel optimisation to robustly guarantee its effectiveness against the worst-case attack around the identified attack vector. Meanwhile, MTD hiddenness is also improved so that the defence cannot be detected by the attacker. To guarantee feasibility and convergence, the convex two-stage reformulation is derived through duality and linear matrix inequality. The simulation results verify that blending data and physics can achieve extremely high detection rate while simultaneously reducing the false positive rate of the data-driven detector and the extra cost of MTD. All codes are available at https://github.com/xuwkk/DDET-MTD .
- 30 October 2017
- Published by Springer Nature in Journal of Solid State Electrochemistry
- Vol. 22 (3) , 817-824
Electrocatalytic oxidation of methanol on Pt-Pd nanoparticles supported on honeycomb-like porous carbons in alkaline media
- 30 October 2017
- Published by Springer Nature in Journal of Solid State Electrochemistry
- Vol. 22 (3) , 817-824
Electrocatalytic oxidation of methanol on Pt-Pd nanoparticles supported on honeycomb-like porous carbons in alkaline media
- 22 August 2016
- Published by American Chemical Society (ACS) in Journal of Chemical & Engineering Data
- Vol. 61 (10) , 3533-3541
Competition Adsorption, Equilibrium, Kinetic, and Thermodynamic Studied over La(III)-loaded Active Carbons for Dibenzothiophene Removal
- 22 August 2016
- Published by American Chemical Society (ACS) in Journal of Chemical & Engineering Data
- Vol. 61 (10) , 3533-3541
Competition Adsorption, Equilibrium, Kinetic, and Thermodynamic Studied over La(III)-loaded Active Carbons for Dibenzothiophene Removal