Potential-Game-Driven Formation Control of AUVs: An Inverse-Reinforcement-Learning-Based Solution
- 7 January 2025
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Oceanic Engineering
- Vol. 50 (2) , 1165-1183
Abstract
The formation control of autonomous underwater vehicles (AUVs) is widely used to perform complex maritime missions. However, most works simply add all individual interests of AUVs together to achieve the formation task, which cannot reveal their interaction relationship and guarantee the full exploitation of formation interests. In this article, we are concerned with a potential-game-driven formation issue for AUVs. Specifically, a potential game-driven formation framework is first developed to formulate the decision-making procedure of AUVs, where the cost function not only reflects the common formation objective but also captures the self-interest. Based on this, the formation optimization problem of AUVs is constructed. To solve the aforementioned problem, an inverse reinforcement learning (IRL)-based formation controller is designed, whose aim is to maximize the formation benefits and drive AUVs to the desired formation shape. Besides that, the theory analyses, including the satisfiability of the potential game, the accessibility of the Nash equilibrium, and the convergence of the formation controller, are provided. The main innovations of this article are as follows: 1) the potential-game-driven framework can fully exploit individual AUV interests and enhance formation robustness over traditional cooperative frameworks and 2) the IRL-based formation controller has better environmental adaptability to the unknown underwater environment compared with traditional reinforcement learning controllers. Finally, simulations and experiments conducted to validate the effectiveness of our solution.Keywords
Funding Information
- National Natural Science Foundation of China (62222314)
- Yanzhao Young Scientist Project of Hebei Province (F2024203047)
- National Science Foundation of Hebei Province (F2022203001, F2024203072)
- Central Guidance Local Foundation of Hebei Province (226Z3201G)
This publication has 30 references indexed in Scilit:
- Differential energy pricing of industrial park operator based on potential game considering public loss allocationEnergy Reports, 2023
- A neural network based efficient leader–follower formation control approach for multiple autonomous underwater vehiclesEngineering Applications of Artificial Intelligence, 2023
- Security defense decision method based on potential differential game for complex networksComputers & Security, 2023
- Inverse reinforcement learning for multi-player noncooperative apprentice gamesAutomatica, 2022
- Robust formation tracking of multiple autonomous surface vessels with individual objectives: A noncooperative game-based approachControl Engineering Practice, 2022
- An evidential reasoning-based leader-follower game for hierarchical multi-agent decision making under uncertaintyJournal of Hydrology, 2020
- Robust fuzzy 3D path following for autonomous underwater vehicle subject to uncertaintiesComputers & Operations Research, 2017
- Adaptive sliding-mode attitude control for autonomous underwater vehicles with input nonlinearitiesOcean Engineering, 2016
- On potential equations of finite gamesAutomatica, 2016
- Weierstrass and Approximation TheoryJournal of Approximation Theory, 2000