ImpedanceGPT: VLM-driven Impedance Control of Swarm of Mini-drones for Intelligent Navigation in Dynamic Environment
Authors:
- 19 October 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2592-2597
Abstract
Swarm robotics plays a crucial role in enabling autonomous operations in dynamic and unpredictable environments. However, a major challenge remains ensuring safe and efficient navigation in environments shared by both dynamic alive (e.g., humans) and dynamic inanimate (e.g., non-living objects) obstacles. In this paper, we propose ImpedanceGPT, a novel system that leverages a Vision-Language Model (VLM) with Retrieval-Augmented Generation (RAG) framework to enable real-time reasoning for adaptive navigation of mini-drone swarm in complex environments. The key innovation of ImpedanceGPT lies in the merging VLM-RAG system with impedance control method, which is an active compliance strategy. This system provides drones with an enhanced semantic understanding of their surroundings and allows them to dynamically adjust impedance control parameters in response to obstacle types and environmental conditions. Our approach not only ensures safe and precise navigation but also improves coordination between drones in the swarm. Experimental evaluations demonstrate the effectiveness of our system. The VLM-RAG framework achieved an obstacle detection and retrieval accuracy of 80% under optimal lighting. In static environments, drones navigated dynamic inanimate obstacles at 1.4 m/s but slowed to 0.7 m/s with increased safety margin around humans. In dynamic environments, speed adjusted to 1.0 m/s near hard obstacles, while reducing to 0.6 m/s with higher deflection region to safely avoid moving humans.Video of ImpedanceGPT: https://youtu.be/JTdeg9bAzL4 Github: https://github.com/Faryal-Batool/ImpedanceGPTKeywords
All Related Versions
This publication has 19 references indexed in Scilit:
- HetSwarm: Cooperative Navigation of Heterogeneous Swarm in Dynamic and Dense Environments Through Impedance-Based Guidance2025
- VLM-RRT: Vision Language Model Guided RRT Search for Autonomous UAV Navigation2025
- Human Digital Twin in the context of Industry 5.0Robotics and Computer-Integrated Manufacturing, 2023
- SwarmGear: Heterogeneous Swarm of Drones with Morphogenetic Leader Drone and Virtual Impedance Links for Multi-Agent Inspection2023
- Adaptive stiffness estimation impedance control for achieving sustained contact in aerial manipulation2021
- Towards Vision-Based Impedance Control for the Contact Inspection of Unknown Generically-Shaped Surfaces with a Fully-Actuated UAV2020
- Swarms of Unmanned Aerial Vehicles — A SurveyJournal of Industrial Information Integration, 2019
- Impedance Control of an aerial-manipulator: Preliminary results2016
- Adaptive impedance control of UAVs interacting with environment using a robot manipulator2014
- Intelligent Variable Joint Impedance Control and Development of a New Whole-Sensitive Anthropomorphic Robot Arm2007