Learning Multi-Agent Reservoir Cooperative Operations Over Multi-Relational Directed Acyclic Graph
- 9 December 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 23 (15455955) , 846-858
Abstract
Operating large multi-reservoir systems is critical for effective water allocation, hydropower generation, and economic development. However, conventional robust planning-based methods scale poorly beyond single-reservoir or cascaded systems due to computational intractability. Addressing inter-reservoir relation extraction and coordination under conflicting objectives, uncertain inflows, and complex network couplings therefore remains an open challenge. To tackle this problem, we introduce the multi-agent reservoir cooperative operation (MARCO) environment, which integrates multiple objectives, heterogeneous topology, and stochastic inflows in a unified framework. An algorithm is then designed to construct a multi-relational directed acyclic graph (MR-DAG) that encodes the underlying topology through coupled objectives and entity relations. Building on this representation, we propose the multi-agent relational directed acyclic graph transformer (MAR-DAGT), a reinforcement learning algorithm that performs typed message passing for efficient feature extraction and employs acyclic decision-making to exploit causal structure for improved credit assignment. Extensive experiments on MARCO show that MAR-DAGT consistently outperforms other MARL and optimization-based baselines in terms of objective satisfaction and robustness to inflow uncertainty.Keywords
Funding Information
- National Science and Technology Major Project of China (2022ZD0119702)
- National Natural Science Foundation of China (62473292)
- National Natural Science Foundation of China (62088101)
- Shanghai Municipal Science and Technology Major Project (2021SHZDZX0100)
This publication has 37 references indexed in Scilit:
- Parallelized robust distributed model predictive control in the presence of coupled state constraintsAutomatica, 2024
- do-mpc: Towards FAIR nonlinear and robust model predictive controlControl Engineering Practice, 2023
- Event-triggered distributed predictive cooperation control for multi-agent systems subject to bounded disturbancesAutomatica, 2023
- Model Predictive Control of water resources systems: A review and research agendaAnnual Reviews in Control, 2023
- Stochastic MPC For Optimal Operation of Hydropower Station Under UncertaintyIFAC-PapersOnLine, 2022
- An integrated model for optimal water resources allocation under “3 Redlines” water policy of the upper Hanjiang river basinJournal of Hydrology: Regional Studies, 2022
- acados—a modular open-source framework for fast embedded optimal controlMathematical Programming Computation, 2021
- Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms2021
- Modeling Relational Data with Graph Convolutional Networks2018
- Water management: Current and future challenges and research directionsWater Resources Research, 2015