Abstract
Reinforcement learning provides multi-agent systems with team strategies to effectively operate in dynamic yet enclosed environments, but the learned expertise does not generalize to novelty, i.e., unexpected scenarios diverging from the known properties of an initial environment. Adaptation is particularly crucial for multi-agent teams in collaborative tasks because even minor disturbances can cascade into failures. Continual learning or online expertise adjustment methods require re-training and lack flexibility, making them inefficient and unrealistic to deploy in urgent and fluctuating situations. To enable efficient adaptation, we propose a Bayesian Online Strategy Adaptation (BOSA). Our approach achieves instant online adaptation at the team strategy level by preparing a strategy pool offline, observing unusual task dynamics caused by novelty, and inferring the most suitable strategy to adapt through Bayesian reasoning and strategy selection heuristics. Experiments are designed to validate the strategy inference mechanism and the adaptation to unseen agent capability degradation scenarios. We investigate how BOSA can scale for large operations and demonstrate that BOSA can effectively address sudden capability degradation events within one decision step with a strategy selection heuristic on belief inclination
Funding Information
  • Automotive Research Center
  • University of Michigan
  • Ann Arbor
  • MI
  • USA. Distribution Statement A: Approved for public release (#8586)