Decentralized Optimal Equilibrium Learning in Stochastic Games via Single-bit Feedback
Preprint
Abstract
We study decentralized equilibrium selection in stochastic games under severe information and communication constraints. In such settings, convergence to equilibrium alone is insufficient, as stochastic games typically admit many equilibria with markedly different welfare properties. We address decentralized optimal equilibrium selection, where agents coordinate on equilibria that optimize a designer-specified social welfare objective while allowing heterogeneous tolerance to deviations from strict best responses. Agents observe only the global state trajectory and their realized rewards, and exchange a single randomized bit of feedback per agent per round. This semantic content/discontent signaling mechanism implicitly aligns decentralized learning dynamics with the global welfare objective. We develop explore-and-commit and online variants applicable to general stochastic games, accommodating heterogeneous model-based or model-free methods for solving the induced Markov decision processes, and establish explicit finite-time regret guarantees, showing logarithmic expected regret under mild conditions.Keywords
All Related Versions
This publication has 26 references indexed in Scilit:
- Decentralized online convex optimization with compressed communicationsAutomatica, 2023
- Communication-Efficient Distributed Learning: An OverviewIEEE Journal on Selected Areas in Communications, 2023
- Provably Efficient Reinforcement Learning in Decentralized General-Sum Markov GamesDynamic Games and Applications, 2022
- Best-response dynamics in zero-sum stochastic gamesJournal of Economic Theory, 2020
- Mastering the game of Go without human knowledgeNature, 2017
- Learning efficient Nash equilibria in distributed systemsGames and Economic Behavior, 2012
- Revisiting log-linear learning: Asynchrony, completeness and payoff-based implementationGames and Economic Behavior, 2012
- Finite-time distributed consensus via binary control protocolsAutomatica, 2011
- On the Rate of Convergence of Continuous-Time Fictitious PlayGames and Economic Behavior, 1998
- Fictitious Play Property for Games with Identical InterestsJournal of Economic Theory, 1996