Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Tianyi Chen has a total of 10 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Coordination for Multienergy Microgrids Using Multiagent Reinforcement Learning

    by Dawei Qiu, Tianyi Chen, Goran Strbac, Shengrong Bu

    Multi-energy microgrids (MEMGs) have significant potential to offer high energy utilization efficiency and system flexibility. The coordination of these MEMGs poses challenges due to the various system dynamics and uncertainties and the need to preserve privacy. This paper proposes a double auction (DA) market-based coordination framework. As such, MEMGs can not only schedule their own energy components but also trade energy with others in DA market. After that, we formulate this problem as Markov games and propose a multi-agent reinforcement learning method by making use of the DA market public information to enhance the stability with privacy perseverance. Case studies involving a real-world scenario validate the superior performance of the proposed method in reducing both energy costs and carbon emissions.


  • Open Access

    Co-Optimizing Battery Storage for Energy Arbitrage and Frequency Regulation in Real-Time Markets Using Deep Reinforcement Learning

    by Yushen Miao, Tianyi Chen, Shengrong Bu, Hao Liang, Zhu Han

    Battery energy storage systems (BESSs) play a critical role in eliminating uncertainties associated with renewable energy generation, to maintain stability and improve flexibility of power networks. In this paper, a BESS is used to provide energy arbitrage (EA) and frequency regulation (FR) services simultaneously to maximize its total revenue within the physical constraints. The EA and FR actions are taken at different timescales. The multitimescale problem is formulated as two nested Markov decision process (MDP) submodels. The problem is a complex decision-making problem with enormous high-dimensional data and uncertainty (e.g., the price of the electricity). Therefore, a novel co-optimization scheme is proposed to handle the multitimescale problem, and also coordinate EA and FR services. A triplet deep deterministic policy gradient with exploration noise decay (TDD–ND) approach is used to obtain the optimal policy at each timescale. Simulations are conducted with real-time electricity prices and regulation signals data from the American PJM regulation market. The simulation results show that the proposed approach performs better than other studied policies in literature.


  • Top Cited Papers

    Peer-to-Peer Energy Trading and Energy Conversion in Interconnected Multi-Energy Microgrids Using Multi-Agent Deep Reinforcement Learning

    by Tianyi Chen, Shengrong Bu, Xue Liu, Jikun Kang, F. Richard Yu, Zhu Han

    A key aspect of multi-energy microgrids (MEMGs) is the capability to efficiently convert and store energy in order to reduce the costs and environmental impact. Peer-to-peer (P2P) energy trading is a novel paradigm for decentralised energy market designs. In this paper, we investigate the external P2P energy trading problem and internal energy conversion problem within interconnected residential, commercial and industrial MEMGs. These two problems are complex decision-making problems with enormous high-dimensional data and uncertainty, so a multi-agent deep reinforcement learning approach combining the multi-agent actor-critic algorithm with the twin delayed deep deterministic policy gradient algorithm is proposed. The proposed approach can handle the high-dimensional continuous action space and aligns with the nature of P2P energy trading with multiple MEMGs. Simulation results based on three real-world MG datasets show that the proposed approach significantly reduces each MG’s average hourly operation cost. The impact of carbon tax pricing is also considered.


  • Realistic Peer-to-Peer Energy Trading Model for Microgrids using Deep Reinforcement Learning

    by Tianyi Chen, Shengrong Bu

    In this paper, we integrate deep reinforcement learning with our realistic peer-to-peer (P2P) energy trading model to address a decision-making problem for microgrids (MGs) in the local energy market. First, an hour-ahead P2P energy trading model with a set of critical physical constraints is formed. Then, the decision-making process of energy trading is built as a Markov decision process, which is used to find the optimal strategies for MGs using a deep reinforcement learning (DRL) algorithm. Specifically, a modified deep Q-network (DQN) algorithm helps the MGs to utilise their resources and make better strategies. Finally, we choose several real-world electricity data sets to perform the simulations. The DQN-based energy trading strategies improve the utilities of the MGs and significantly reduce the power plant schedule with a virtual penalty function. Moreover, the model can determine the best battery for the selected MG. The results show that this P2P energy trading model can be applied to real-world situations.