Autonomous Human-Robot Collaborative Assembly Method Driven by the Fusion of Large Language Model and Digital Twin
Authors:
Open Access
- 1 August 2024
- conference paper
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 2832 (1)
Abstract
Human-robot collaboration (HRC) plays an important role in human-centric manufacturing, which requires cooperative robots to have the ability of collaborate with human autonomously. It is very complex to understand the intention of human during the assembly process, therefore, we proposed a method of autonomous HRC assembly driven by the fusion of large language model (LLM) and digital twin in this paper. The assembly state is recognized from two perspectives, including the perception of key parts based on transfer learning and YOLO, and perceive operator actions based on LSTM and attention mechanism. In order to improve the autonomy of HRC, a collaborative task decision method driven by fine-tuning LLM based on assembly domain knowledge is proposed. A case study of reducer assembly is presented to verify the effectiveness of the proposed method.Keywords
This publication has 13 references indexed in Scilit:
- Leveraging error-assisted fine-tuning large language models for manufacturing excellenceRobotics and Computer-Integrated Manufacturing, 2024
- Transfer learning and fine‐tuned transfer learning methods' effectiveness analyse in the CNN‐based deep learning modelsConcurrency and Computation: Practice and Experience, 2022
- Dynamic reconfiguration optimization of intelligent manufacturing system with human-robot collaboration based on digital twinJournal of Manufacturing Systems, 2022
- A visual reasoning-based approach for mutual-cognitive human-robot collaborationCIRP Annals, 2022
- Transfer Learning-enabled Action Recognition for Human-robot Collaborative AssemblyProcedia CIRP, 2021
- Survey of Human–Robot Collaboration in Industrial Settings: Awareness, Intelligence, and ComplianceIEEE Transactions on Systems, Man, and Cybernetics: Systems, 2020
- Task Allocation in Human-Robot Collaboration (HRC) Based on Task Characteristics and Agent Capability for Mold AssemblyProcedia Manufacturing, 2020
- Digital twin driven human–robot collaborative assemblyCIRP Annals, 2019
- Gesture recognition for human-robot collaboration: A reviewInternational Journal of Industrial Ergonomics, 2018
- Deep learning-based human motion recognition for predictive context-aware human-robot collaborationCIRP Annals, 2018