Augmenting First-Person Drone Swarm Teleoperation With Multisensory Feedback
Open Access
- 3 November 2026
- journal article
- Published by SAGE Publications in Proceedings of the Human Factors and Ergonomics Society Annual Meeting
Abstract
This paper evaluates how multisensory feedback can improve human teleoperation of aerial swarms. Effective oversight in swarm missions requires user interfaces that convey the swarm state and environment information. While Top-Down View (TDV) naturally provides this, it is impractical in real-world contexts. First-Person View (FPV), by contrast, aligns with the distributed nature of swarms but limits awareness of the swarm state. To address this limitation, we designed a multisensory interface that augments an FPV interface with haptic and audio feedback. Specifically, we propose a novel mapping that compresses the complex swarm state into intuitive cues, using key metrics conveyed through a vibrotactile jacket and spatial audio. Through a user study, we show that while TDV perspectives achieve the best performance, multisensory feedback enabled significant performance increases for FPV perspectives. These results suggest that multisensory feedback can effectively mitigate the limitations of FPV perspectives, improving performance and enabling more reliable human-swarm teleoperation.Keywords
Funding Information
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (200020_212077)
This publication has 34 references indexed in Scilit:
- AeroHaptix: A Wearable Vibrotactile Feedback System for Enhancing Collision Avoidance in UAV TeleoperationIEEE Robotics and Automation Letters, 2025
- Drone-NeRF: Efficient NeRF based 3D scene reconstruction for large-scale drone surveyImage and Vision Computing, 2024
- A survey of multi-agent Human–Robot Interaction systemsRobotics and Autonomous Systems, 2022
- Teleoperation methods and enhancement techniques for mobile robots: A comprehensive surveyRobotics and Autonomous Systems, 2022
- Swarm Robotics: Past, Present, and Future [Point of View]Proceedings of the IEEE, 2021
- Beyond drone vision: the embodied telepresence of first-person-view drone flightThe Senses and Society, 2020
- Methodology for Maximizing Information Transmission of Haptic Devices: A SurveyProceedings of the IEEE, 2020
- FlyJacket: An Upper Body Soft Exoskeleton for Immersive Drone ControlIEEE Robotics and Automation Letters, 2018
- Magnitude-dependence of equivalent comfort contours for fore-and-aft, lateral and vertical hand-transmitted vibrationJournal of Sound and Vibration, 2006
- Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research1988