A methodological framework and experimental protocol for proactive human–robot collaboration with multimodal intention prediction and adaptive control
Open Access
- 7 May 2026
- journal article
- review article
- Published by Frontiers Media SA in Frontiers in Robotics and AI
- Vol. 13, 1825254
Abstract
Industry 5.0 requires collaborative robots that can anticipate operator needs to improve fluency and safety in assembly. However, many human–robot collaboration (HRC) systems still treat perception, intention inference, and control as separate components. This study presents a theoretical perception–cognition–action framework that explicitly couples multimodal intention prediction with proactive and adaptive control. Multimodal observations such as RGB-D vision, gaze, wrist force/torque, robot joint state, and previous robot action are encoded by a hybrid Convolutional Neural Network (CNN)– Long Short-Term Memory (LSTM)–Transformer to estimate (i) a probability distribution over future human intentions and (ii) a short-horizon motion trajectory, trained with a composite loss that jointly optimizes classification and regression with kinematic coherence. The predicted intention probability is embedded into an augmented Markov Decision Process state, enabling a Soft Actor–Critic agent to learn continuous policies with rewards designed for synergy, efficiency, safety, and fluency. The main contributions of this study are the formal probabilistic linkage from intention prediction to adaptive control, the definition of a multi-output cognitive objective, and the design of an implementation-ready experimental protocol for future empirical validation. Overall, the proposed methodological framework and experimental protocol provide a reproducible basis for future empirical validation of proactive human–robot collaboration in industrial assembly tasks.Keywords
This publication has 32 references indexed in Scilit:
- Adaptive control system for collaborative sorting robotic arms based on multimodal sensor fusion and edge computingScientific Reports, 2025
- Vision-Language-Action Models for Robotics: A Review Towards Real-World ApplicationsIEEE Access, 2025
- Advanced Algorithm Combinations for Safe Human-Robot Collaboration: A Review and Research ProposalProcedia Computer Science, 2025
- Reviewing human-robot collaboration in manufacturing: Opportunities and challenges in the context of industry 5.0Robotics and Computer-Integrated Manufacturing, 2024
- Multimodal data-based human motion intention prediction using adaptive hybrid deep learning network for movement challenged personScientific Reports, 2024
- Adaptive Behavior of Collaborative Robots: Review and Investigation of Human Predictive Ability.Procedia CIRP, 2024
- QUB-PHEO: A Visual-Based Dyadic Multi-View Dataset for Intention Inference in Collaborative AssemblyIEEE Access, 2024
- Enhancing Robotic Collaborative Tasks Through Contextual Human Motion Prediction and Intention InferenceInternational Journal of Social Robotics, 2024
- Deep reinforcement learning for cooperative robots based on adaptive sentiment feedbackExpert Systems with Applications, 2023
- Human–robot collaboration and machine learning: A systematic review of recent researchRobotics and Computer-Integrated Manufacturing, 2022