Bidirectional Intent Communication: A Role for Large Foundation Models
Preprint
Abstract
Integrating multimodal foundation models has significantly enhanced autonomous agents' language comprehension, perception, and planning capabilities. However, while existing works adopt a \emph{task-centric} approach with minimal human interaction, applying these models to developing assistive \emph{user-centric} robots that can interact and cooperate with humans remains underexplored. This paper introduces ``Bident'', a framework designed to integrate robots seamlessly into shared spaces with humans. Bident enhances the interactive experience by incorporating multimodal inputs like speech and user gaze dynamics. Furthermore, Bident supports verbal utterances and physical actions like gestures, making it versatile for bidirectional human-robot interactions. Potential applications include personalized education, where robots can adapt to individual learning styles and paces, and healthcare, where robots can offer personalized support, companionship, and everyday assistance in the home and workplace environments.Keywords
All Related Versions
This publication has 7 references indexed in Scilit:
- Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4V2025
- Grounding DINO: Marrying DINO with Grounded Pre-training for Open-Set Object Detection2024
- ADaPT: As-Needed Decomposition and Planning with Language Models2024
- Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A SurveyACM Computing Surveys, 2023
- Using Eye-Tracking Data to Predict Situation Awareness in Real Time During Takeover Transitions in Conditionally Automated DrivingIEEE Transactions on Intelligent Transportation Systems, 2021
- Joint action understanding improves robot-to-human object handover2013
- Wizard of Oz Studies in HRI: A Systematic Review and New Reporting GuidelinesJournal of Human-Robot Interaction, 2012