Dynamic Attention Analysis of Body Parts in Transformer-Based Human–Robot Imitation Learning with the Embodiment Gap
Open Access
Abstract
In imitation learning between humans and robots, the embodiment gap is a key challenge. By focusing on a specific body part and compensating for the rest according to the robot’s size, the embodiment gap can be overcome. In this paper, we analyze dynamic attention to body parts in imitation learning between humans and robots based on a Transformer model. To adapt human imitation movements to a robot, we solved forward and inverse kinematics using the Levenberg–Marquardt method and performed feature extraction using the k-means method to make the data suitable for Transformer input. The imitation learning process is carried out using the Transformer. UMAP is employed to visualize the attention layer within the Transformer. As a result, this system enabled imitation of movements while focusing on multiple body parts between humans and robots with an embodiment gap, revealing the transitions of body parts receiving attention and their relationships in the robot’s acquired imitation movements.Keywords
This publication has 36 references indexed in Scilit:
- DexMV: Imitation Learning for Dexterous Manipulation from Human Videos2022
- Adversarial Imitation Learning from Video Using a State Observer2022
- Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation2018
- Time-Contrastive Networks: Self-Supervised Learning from Multi-view Observation2017
- Incrementally assisted kinesthetic teaching for programming by demonstration2016
- A Study on the Measurement of Wrist Motion Range Using the iPhone 4 Gyroscope ApplicationAnnals of Plastic Surgery, 2014
- Real-time imitation of human whole-body motions by humanoids2014
- Principal component analysisWIREs Computational Statistics, 2010
- A survey of robot learning from demonstrationRobotics and Autonomous Systems, 2009
- Estimating mutual informationPhysical Review E, 2004