LaMI: Large Language Models for Multi-Modal Human-Robot Interaction
Top Cited Papers
Open Access
- 2 May 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-10
Abstract
This paper presents an innovative large language model (LLM)-based robotic system for enhancing multi-modal human-robot interaction (HRI). Traditional HRI systems relied on complex designs for intent estimation, reasoning, and behavior generation, which were resource-intensive. In contrast, our system empowers researchers and practitioners to regulate robot behavior through three key aspects: providing high-level linguistic guidance, creating "atomic actions" and expressions the robot can use, and offering a set of examples. Implemented on a physical robot, it demonstrates proficiency in adapting to multi-modal inputs and determining the appropriate manner of action to assist humans with its arms, following researchers’ defined guidelines. Simultaneously, it coordinates the robot’s lid, neck, and ear movements with speech output to produce dynamic, multi-modal expressions. This showcases the system’s potential to revolutionize HRI by shifting from conventional, manual state-and-flow design methods to an intuitive, guidance-based, and example-driven approach. Supplementary material can be found at https://hri-eu.github.io/Lami/Keywords
All Related Versions
This publication has 19 references indexed in Scilit:
- Large Language Models as Zero-Shot Human Models for Human-Robot Interaction2023
- Improved Trust in Human-Robot Collaboration With ChatGPTIEEE Access, 2023
- A Multimodal Emotional Human–Robot Interaction Architecture for Social Robots Engaged in Bidirectional CommunicationIEEE Transactions on Cybernetics, 2020
- Promoting Interactions Between Humans and Robots Using Robotic Emotional BehaviorIEEE Transactions on Cybernetics, 2015
- Development of a generic method to generate upper-body emotional expressions for different social robotsAdvanced Robotics, 2015
- Robotic gesture generation based on a cognitive basis for non-verbal communication2014
- Adaptive emotional expression in robot-child interaction2014
- A Kinect-Based Motion Capture System for Robotic Gesture Imitation2014
- An auto-adaptable algorithm to generate human-like locomotion for different humanoid robots based on motion capture data2010
- Captured Motion Data Processing for Real Time Synthesis of Sign LanguageLecture Notes in Computer Science, 2006