Learning to coordinate without communication in multi-user multi-armed bandit problems
Authors:
- Orly AvnerOAOrly Avner
- Shie MannorSMShie Mannor
Preprint
Abstract
We consider a setting where multiple users share multiple channels modeled as a multi-user multi-armed bandit (MAB) problem. The characteristics of each channel are initially unknown and may differ between the users. Each user can choose between the channels, but her success depends on the particular channel as well as on the selections of other users: if two users select the same channel their messages collide and none of them manages to send any data. Our setting is fully distributed, so there is no central control and every user only observes the channel she currently uses. As in many communication systems such as cognitive radio networks, the users cannot communicate among themselves so coordination must be achieved without direct communication. We develop algorithms for learning a stable configuration for the multiple user MAB problem. We further offer both convergence guarantees and experiments inspired by real communication networks.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- WLAN channel selection without communicationComputer Networks, 2012
- UCB revisited: Improved regret bounds for the stochastic multi-armed bandit problemPeriodica Mathematica Hungarica, 2010
- Almost Stable Matchings by Truncating the Gale–Shapley AlgorithmAlgorithmica, 2009
- A survey on spectrum management in cognitive radio networksIEEE Communications Magazine, 2008
- Finite-time Analysis of the Multiarmed Bandit ProblemMachine Learning, 2002
- The Nonstochastic Multiarmed Bandit ProblemSIAM Journal on Computing, 2002
- The auction algorithm: A distributed relaxation method for the assignment problemAnnals of Operations Research, 1988
- The Hungarian method for the assignment problemNaval Research Logistics Quarterly, 1955