BOSA: Bayesian Online Strategy Adaptation for Unexpected Events in Multi-Agent Teams
Open Access
- 28 April 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Systems, Man, and Cybernetics Letters
- Vol. 1, 6-14
Abstract
Reinforcement learning provides multi-agent systems with team strategies to effectively operate in dynamic yet enclosed environments, but the learned expertise does not generalize to novelty, i.e., unexpected scenarios diverging from the known properties of an initial environment. Adaptation is particularly crucial for multi-agent teams in collaborative tasks because even minor disturbances can cascade into failures. Continual learning or online expertise adjustment methods require re-training and lack flexibility, making them inefficient and unrealistic to deploy in urgent and fluctuating situations. To enable efficient adaptation, we propose a Bayesian Online Strategy Adaptation (BOSA). Our approach achieves instant online adaptation at the team strategy level by preparing a strategy pool offline, observing unusual task dynamics caused by novelty, and inferring the most suitable strategy to adapt through Bayesian reasoning and strategy selection heuristics. Experiments are designed to validate the strategy inference mechanism and the adaptation to unseen agent capability degradation scenarios. We investigate how BOSA can scale for large operations and demonstrate that BOSA can effectively address sudden capability degradation events within one decision step with a strategy selection heuristic on belief inclinationKeywords
Funding Information
- Automotive Research Center
- University of Michigan
- Ann Arbor
- MI
- USA. Distribution Statement A: Approved for public release (#8586)
This publication has 25 references indexed in Scilit:
- AI Autonomy: Self‐initiated Open‐world Continual Learning and AdaptationAI Magazine, 2023
- RAPid-Learn: A Framework for Learning to Recover for Handling Novelties in Open-World Environments2022
- Task Allocation with Load Management in Multi-Agent Teams2022
- A survey of inverse reinforcement learning: Challenges, methods and progressArtificial Intelligence, 2021
- Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms2021
- Interventions in Complex Urban Systems: How to Enable Modeling to Account for Disruptive Innovation2016
- Human-level control through deep reinforcement learningNature, 2015
- Multi-agent team cooperation: A game theory approachAutomatica, 2009
- Mean field gamesJapanese Journal of Mathematics, 2007
- Markov games as a framework for multi-agent reinforcement learning1994