Who2com: Collaborative Perception via Learnable Handshake Communication
Authors:
- 1 May 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 6876-6883
Abstract
In this paper, we propose the problem of collaborative perception, where robots can combine their local observations with those of neighboring agents in a learnable way to improve accuracy on a perception task. Unlike existing work in robotics and multi-agent reinforcement learning, we formulate the problem as one where learned information must be shared across a set of agents in a bandwidth-sensitive manner to optimize for scene understanding tasks such as semantic segmentation. Inspired by networking communication protocols, we propose a multi-stage handshake communication mechanism where the neural network can learn to compress relevant information needed for each stage. Specifically, a target agent with degraded sensor data sends a compressed request, the other agents respond with matching scores, and the target agent determines who to connect with (i.e., receive information from). We additionally develop the AirSim-CP dataset and metrics based on the AirSim simulator where a group of aerial robots perceive diverse landscapes, such as roads, grasslands, buildings, etc. We show that for the semantic segmentation task, our handshake communication method significantly improves accuracy by approximately 20% over decentralized baselines, and is comparable to centralized ones using a quarter of the bandwidth.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Unsupervised Monocular Depth Estimation with Left-Right Consistency2017
- Pyramid Scene Parsing Network2017
- Deep Residual Learning for Image Recognition2016
- Multiple-Robot Simultaneous Localization and Mapping: A ReviewJournal of Field Robotics, 2015
- Fully convolutional networks for semantic segmentation2015
- Effective Approaches to Attention-based Neural Machine Translation2015
- DDF-SAM: Fully distributed SLAM using Constrained Factor Graphs2010
- Cooperative Multi-Agent Learning: The State of the ArtAutonomous Agents and Multi-Agent Systems, 2005
- Markov games as a framework for multi-agent reinforcement learning1994
- Multi-Agent Reinforcement Learning: Independent vs. Cooperative Agents1993