Mixed-reality for quadruped-robotic guidance in SAR tasks
Open Access
- 26 June 2023
- journal article
- research article
- Published by Oxford University Press (OUP) in Journal of Computational Design and Engineering
- Vol. 10 (4) , 1479-1489
Abstract
In recent years, exploration tasks in disaster environments, victim localization and primary assistance have been the main focuses of Search and Rescue (SAR) Robotics. Developing new technologies in Mixed Reality (M-R) and legged robotics has taken a big step in developing robust field applications in the Robotics field. This article presents MR-RAS (Mixed-Reality for Robotic Assistance), which aims to assist rescuers and protect their integrity when exploring post-disaster areas (against collapse, electrical, and toxic risks) by facilitating the robot’s gesture guidance and allowing them to manage interest visual information of the environment. Thus, ARTU-R (A1 Rescue Tasks UPM Robot) quadruped robot has been equipped with a sensory system (lidar, thermal, and RGB-D cameras) to validate this proof of concept. On the other hand, Human-Robot interaction is executed by using the Hololens glasses. This work’s main contribution is the implementation and evaluation of a Mixed-Reality system based on a ROS-Unity solution, capable of managing at a high level the guidance of a complex legged robot through different interest zones (defined by a Neural Network and a vision system) of a post-disaster environment (PDE). The robot’s main tasks at each point visited involve detecting victims through thermal, RGB imaging, and neural networks and assisting victims with medical equipment. Tests have been carried out in scenarios that recreate the conditions of PDE (debris, simulation of victims, etc.). An average efficiency improvement of 48% has been obtained when using the immersive interface and a time optimization of 21.4% compared to conventional interfaces. The proposed method has proven to improve rescuers’ immersive experience of controlling a complex robotic system.Keywords
This publication has 45 references indexed in Scilit:
- Thermal image-based CNN's for ultra-low power people recognition2018
- The DARPA Robotics Challenge Finals: Humanoid Robots To The Rescue2018
- Human Object Detection using Very Low Resolution Thermal Cameras for Urban Search and Rescue2017
- Thermal cameras and applications: a surveyMachine Vision and Applications, 2013
- Rescue Robotics2009
- Real‐time localization and elevation mapping within urban search and rescue scenariosJournal of Field Robotics, 2007
- Environmental Disasters, Natural Recovery and Human Responses2007
- Nasa-Task Load Index (NASA-TLX); 20 Years LaterProceedings of the Human Factors and Ergonomics Society Annual Meeting, 2006
- Human-robot interactions during the robot-assisted urban search and rescue response at the World Trade CenterIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2003
- Reference test courses for autonomous mobile robots2001