SafeHumanoid: VLM-RAG-driven Control of Upper Body Impedance for Humanoid Robot
Authors:
- Yara MahmoudYMYara Mahmoud
- Jeffrin SamJSJeffrin Sam
- Nguyen KhangNKNguyen Khang
- Marcelino FernandoMFMarcelino Fernando
- Issatay TokmurziyevITIssatay Tokmurziyev
- Miguel Altamirano CabreraMCMiguel Altamirano Cabrera
- Muhammad Haris KhanMKMuhammad Haris Khan
- Artem LykovALArtem Lykov
- Dzmitry TsetserukouDTDzmitry Tsetserukou
Preprint
Abstract
Safe and trustworthy Human Robot Interaction (HRI) requires robots not only to complete tasks but also to regulate impedance and speed according to scene context and human proximity. We present SafeHumanoid, an egocentric vision pipeline that links Vision Language Models (VLMs) with Retrieval-Augmented Generation (RAG) to schedule impedance and velocity parameters for a humanoid robot. Egocentric frames are processed by a structured VLM prompt, embedded and matched against a curated database of validated scenarios, and mapped to joint-level impedance commands via inverse kinematics. We evaluate the system on tabletop manipulation tasks with and without human presence, including wiping, object handovers, and liquid pouring. The results show that the pipeline adapts stiffness, damping, and speed profiles in a context-aware manner, maintaining task success while improving safety. Although current inference latency (up to 1.4 s) limits responsiveness in highly dynamic settings, SafeHumanoid demonstrates that semantic grounding of impedance control is a viable path toward safer, standard-compliant humanoid collaboration.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- ImpedanceGPT: VLM-driven Impedance Control of Swarm of Mini-drones for Intelligent Navigation in Dynamic Environment2025
- ARMOR: Egocentric Perception for Bimanual Robot Collision Avoidance and Motion Planning2025
- Shake-VLA: Vision-Language-Action Model-Based System for Bimanual Robotic Manipulations and Liquid Mixing2025
- Text2Motion: from natural language instructions to feasible plansAutonomous Robots, 2023
- Model-based variable impedance learning control for robotic manipulationRobotics and Autonomous Systems, 2023
- Code as Policies: Language Model Programs for Embodied Control2023
- Dynamic speed and separation monitoring for collaborative robot applications – Concepts and performanceRobotics and Computer-Integrated Manufacturing, 2019
- Progress and prospects of the human–robot collaborationAutonomous Robots, 2017
- A tutorial on task-parameterized movement learning and retrievalIntelligent Service Robotics, 2015
- Impedance Control: An Approach to Manipulation: Part I—TheoryJournal of Dynamic Systems, Measurement, and Control, 1985