NOPA: Neurally-guided Online Probabilistic Assistance for Building Socially Intelligent Home Assistants
Authors:
- 29 May 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 7628-7634
Abstract
In this work, we study how to build socially intelligent robots to assist people in their homes. In particular, we focus on assistance with online goal inference, where robots must simultaneously infer humans' goals and how to help them achieve those goals. Prior assistance methods either lack the adaptivity to adjust helping strategies (i.e., when and how to help) in response to uncertainty about goals or the scalability to conduct fast inference in a large goal space. Our NOPA (Neurally-guided Online Probabilistic Assistance) method addresses both of these challenges. NOPA consists of (1) an online goal inference module combining neural goal proposals with inverse planning and particle filtering for robust inference under uncertainty, and (2) a helping planner that discovers valuable subgoals to help with and is aware of the uncertainty in goal inference. We compare NOPA against multiple baselines in a new embodied AI assistance challenge: Online Watch-And-Help, in which a helper agent needs to simultaneously watch a main agent's action, infer its goal, and help perform a common household task faster in realistic virtual home environments. Experiments show that our helper agent robustly updates its goal inference and adapts its helping plans to the changing level of uncertainty. 1 1 Code and a supplementary video are available at https://www.tshu.io/online_watch_and_help.Keywords
All Related Versions
Funding Information
- DARPA
- Lockheed Martin
This publication has 25 references indexed in Scilit:
- Human-level performance in 3D multiplayer games with population-based reinforcement learningScience, 2019
- Shared autonomy via hindsight optimization for teleoperation and teamingThe International Journal of Robotics Research, 2018
- Autonomous agents modelling other agents: A comprehensive survey and open problemsArtificial Intelligence, 2018
- Rational quantitative attribution of beliefs, desires and percepts in human mentalizingNature Human Behaviour, 2017
- Learning Physical Collaborative Robot Behaviors From Human DemonstrationsIEEE Transactions on Robotics, 2016
- Image retrieval using scene graphs2015
- A Decision-Theoretic Model of AssistanceJournal of Artificial Intelligence Research, 2014
- Planning under Uncertainty for Robotic Tasks with Mixed ObservabilityThe International Journal of Robotics Research, 2010
- Human–Robot Interaction: A SurveyFoundations and Trends® in Human-Computer Interaction, 2008
- Techniques for Plan RecognitionUser Modeling and User-Adapted Interaction, 2001