Design of multi-modal feedback channel of human–robot cognitive interface for teleoperation in manufacturing
Authors:
Open Access
- 9 July 2024
- journal article
- research article
- Published by Springer Nature in Journal of Intelligent Manufacturing
- Vol. 36 (6) , 4283-4303
Abstract
Teleoperation, which is a specific mode of human–robot collaboration enabling a human operator to provide instructions and monitor the actions of the robot remotely, has proved beneficial for application to hazardous and unstructured manufacturing environments. Despite the design of a command channel from human operators to robots, most existing studies on teleoperation fail to focus on the design of the feedback channel from the robot to the human operator, which plays a crucial role in reducing the cognitive load, particularly in precise and concentrated manufacturing tasks. This paper focuses on designing a feedback channel for the cognitive interface between a human operator and a robot considering human cognition. Current studies on human–robot cognitive interfaces in robot teleoperation are extensively surveyed. Further, the modalities of human cognition that foster understanding and transparency during teleoperation are identified. In addition, the human–robot cognitive interface, which utilizes the proposed multi-modal feedback channel, is developed on a teleoperated robotic grasping system as a case study. Finally, a series of experiments based on different modal feedback channels are conducted to demonstrate the effectiveness of enhancing the performance of the teleoperated grasping of fragile products and reducing the cognitive load via the objective aspects of experimental results and the subjective aspects of operator feedback.Keywords
Funding Information
- Natural Science Basic Research Program of Shaanxi (2023-JC-YB-299)
- China Postdoctoral Science Foundation (2022M712933)
- National Natural Science Foundation of China (51805437)
- Monash University
This publication has 53 references indexed in Scilit:
- Multimodal Human–Robot Interaction for Human‐Centric Smart Manufacturing: A SurveyAdvanced Intelligent Systems, 2023
- Multimodal Approach to Measuring Cognitive Load Using Sternberg Memory and Input Diagrammatic Reasoning Tests2022
- Teleoperation of industrial robot manipulators based on augmented realityThe International Journal of Advanced Manufacturing Technology, 2020
- A conceptual framework to evaluate human-robot collaborationThe International Journal of Advanced Manufacturing Technology, 2020
- Industrial robotic machining: a reviewThe International Journal of Advanced Manufacturing Technology, 2019
- Cognitive Load Theory and Human Movement: Towards an Integrated Model of Working MemoryEducational Psychology Review, 2019
- Cognitive Architecture and Instructional Design: 20 Years LaterEducational Psychology Review, 2019
- Industrial wearable system: the human-centric empowering technology in Industry 4.0Journal of Intelligent Manufacturing, 2018
- A comparative review of dynamic neural networks and hidden Markov model methods for mobile on-device speech recognitionNeural Computing and Applications, 2017
- Effects of stress and effort on self-rated reports in experimental study of design activitiesJournal of Intelligent Manufacturing, 2016