Policy search for multi-robot coordination under uncertainty
- 1 December 2016
- journal article
- conference paper
- Published by SAGE Publications in The International Journal of Robotics Research
- Vol. 35 (14) , 1760-1778
Abstract
We introduce a principled method for multi-robot coordination based on a general model (termed a MacDec-POMDP) of multi-robot cooperative planning in the presence of stochasticity, uncertain sensing, and communication limitations. A new MacDec-POMDP planning algorithm is presented that searches over policies represented as finite-state controllers, rather than the previous policy tree representation. Finite-state controllers can be much more concise than trees, are much easier to interpret, and can operate over an infinite horizon. The resulting policy search algorithm requires a substantially simpler simulator that models only the outcomes of executing a given set of motor controllers, not the details of the executions themselves and can solve significantly larger problems than existing MacDec-POMDP planners. We demonstrate significant performance improvements over previous methods and show that our method can be used for actual multi-robot systems through experiments on a cooperative multi-robot bartending domain.Keywords
This publication has 25 references indexed in Scilit:
- Decentralized multi-robot cooperation with auctioned POMDPsThe International Journal of Robotics Research, 2013
- Optimizing fixed-size stochastic controllers for POMDPs and decentralized POMDPsAutonomous Agents and Multi-Agent Systems, 2009
- Hierarchical multi-agent reinforcement learningAutonomous Agents and Multi-Agent Systems, 2006
- NAVIGATION AMONG MOVABLE OBSTACLES: REAL-TIME REASONING IN COMPLEX ENVIRONMENTSInternational Journal of Humanoid Robotics, 2005
- A Formal Analysis and Taxonomy of Task Allocation in Multi-Robot SystemsThe International Journal of Robotics Research, 2004
- A survey of multi-agent organizational paradigmsThe Knowledge Engineering Review, 2004
- The Complexity of Decentralized Control of Markov Decision ProcessesMathematics of Operations Research, 2002
- Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learningArtificial Intelligence, 1999
- Planning and acting in partially observable stochastic domainsArtificial Intelligence, 1998
- ALLIANCE: an architecture for fault tolerant multirobot cooperationIEEE Transactions on Robotics and Automation, 1998