Learning Multi-Agent Coordination for Replenishment At Sea
- 16 December 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 10 (2) , 1018-1025
Abstract
Optimizing large-scale logistics is computationally challenging due to its scale and requirement to be robust to stochastic and time-varying weather disturbances. However, prior research in multi-agent reinforcement learning (MARL) does not address scenarios that capture complexity of logistics operations influenced by dynamic weather patterns. To address this gap, we suggest a new MARL environment, MARINE that has two types of agents equipped with limited resources and integrates real wave data to model the influences of weather on the replenishment at sea (RAS) operation. To this end, we propose SchedHGNN, a novel MARL algorithm that incorporates a heterogeneous graph neural network and an intrinsic reward scheme to enhance agent coordination and mitigate challenges induced by environment non-stationarity. Our results show that the combination of effective RAS scheduling and improved communication enables our model to outperform competitive baselines by up to 37.8%. This achievement marks a significant advancement in applying MARL to complex, real-world logistics scenarios.Keywords
Funding Information
- Naval Research Laboratory (N00173-21-1-G009)
This publication has 19 references indexed in Scilit:
- Heterogeneous Policy Networks for Composite Robot Team Communication and CoordinationIEEE Transactions on Robotics, 2024
- LJIR: Learning Joint-Action Intrinsic Reward in cooperative multi-agent reinforcement learningNeural Networks, 2023
- An epsilon-constraint-based exact multi-objective optimization approach for the ship schedule recovery problem in liner shippingComputers & Industrial Engineering, 2023
- Deep multiagent reinforcement learning: challenges and directionsArtificial Intelligence Review, 2022
- Robust liner ship routing and scheduling schemes under uncertain weather and ocean conditionsTransportation Research Part C: Emerging Technologies, 2022
- Berth allocation problem with uncertain vessel handling times considering weather conditionsComputers & Industrial Engineering, 2021
- Investigation into control strategies for manoeuvring in adverse weather conditionsOcean Engineering, 2020
- Validation of a distributed simulation of ship replenishment at sea with model testsJournal of Marine Science and Technology, 2018
- Scheduling combat logistics force replenishments at sea for the US NavyNaval Research Logistics (NRL), 2017
- Curiosity-Driven Exploration by Self-Supervised Prediction2017