A Vision-Based Human Digital Twin Modeling Approach for Adaptive Human–Robot Collaboration
- 21 July 2023
- journal article
- research article
- Published by ASME International in Journal of Manufacturing Science and Engineering
- Vol. 145 (12) , 1-11
Abstract
Human–robot collaboration (HRC) has been identified as a highly promising paradigm for human-centric smart manufacturing in the context of Industry 5.0. In order to enhance both human well-being and robotic flexibility within HRC, numerous research efforts have been dedicated to the exploration of human body perception, but many of these studies have focused only on specific facets of human recognition, lacking a holistic perspective of the human operator. A novel approach to addressing this challenge is the construction of a human digital twin (HDT), which serves as a centralized digital representation of various human data for seamless integration into the cyber-physical production system. By leveraging HDT, performance and efficiency optimization can be further achieved in an HRC system. However, the implementation of visual perception-based HDT remains underreported, particularly within the HRC realm. To this end, this study proposes an exemplary vision-based HDT model for highly dynamic HRC applications. The model mainly consists of a convolutional neural network that can simultaneously model the hierarchical human status including 3D human posture, action intention, and ergonomic risk. Then, on the basis of the constructed HDT, a robotic motion planning strategy is further introduced with the aim of adaptively optimizing the robotic motion trajectory. Further experiments and case studies are conducted in an HRC scenario to demonstrate the effectiveness of our approach.Keywords
Funding Information
- Hong Kong Polytechnic University (Laboratory for Artificial Intelligence in Design (Project Code: RP2-1))
- Research Grants Council, University Grants Committee (General Research Fund (PolyU15210222) and Collaborative Departmental General Research Fund (G-UAMS))
This publication has 34 references indexed in Scilit:
- Digital Twin: Values, Challenges and Enablers From a Modeling PerspectiveIEEE Access, 2020
- Toward Ergonomic Risk Prediction via Segmentation of Indoor Object Manipulation Actions Using Spatiotemporal Convolutional NetworksIEEE Robotics and Automation Letters, 2019
- Symbiotic human-robot collaborative assemblyCIRP Annals, 2019
- Digital twin driven human–robot collaborative assemblyCIRP Annals, 2019
- Controlling Object Hand-Over in Human–Robot Collaboration Via Natural Wearable SensingIEEE Transactions on Human-Machine Systems, 2018
- Digital Twin in Industry: State-of-the-ArtIEEE Transactions on Industrial Informatics, 2018
- Deep learning-based human motion recognition for predictive context-aware human-robot collaborationCIRP Annals, 2018
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- SMPLACM Transactions on Graphics, 2015
- Rapid Entire Body Assessment (REBA)Applied Ergonomics, 2000