Huizhong Cao

Immersive technologies are entering operator training on the assumption that a more engaging modality produces better learning. Whether the modality itself changes what learners keep has rarely been tested against simpler or AI-agent alternatives. We compared three ways of delivering the same 16-step assembly procedure (immersive virtual training, a humanoid social robot driven by a large language model, and interactive video) in a between-subjects experiment with 36 novices. Each participant learned the procedure under one modality and then performed the task once without instruction. We recorded perceived subjective effort (RSME), heart-rate variability (HRV) normalised to each person's baseline, completion time, and error rates. During learning, perceived subjective mental effort did not differ between media, and VR was rated the easiest to follow. The central result is a dissociation between how learning felt and how it turned out: the instruction modality users rated easiest, VR, produced the slowest and least accurate performance once support was removed, and effort during learning was unrelated to later performance (ρ = -0.02). The mental workload imposed by an instruction modality is therefore not revealed by how effortless it feels subjectively during learning phase but a misalignment pattern. Its distinctive contribution, however, is a human-centric assessment of emerging digital technologies, together with a comprehensive, multi-measure evaluation approach intended to inform future industrial practice to adopt new learning technologies for Industry 5.0.

Effective human–AI interaction in manufacturing requires that AI systems do more than deliver information—they must adapt to the operator's cognitive state, augmenting rather than overloading human performance. This study presents the first comparative empirical investigation of mental workload across three distinct human–AI interaction modalities—Virtual Reality (VR), an AI-integrated humanoid social robot (Furhat + GPT-4o), and video instruction—evaluated during both learning and operational phases of an industrial assembly task (n = 36). Using a triangulated measurement protocol combining subjective ratings (RSME), physiological monitoring (heart rate variability via PPG), and performance outcomes (completion time, error rate), we identify a critical dissociation between perceived and physiological cognitive load: VR produced the lowest perceived effort during learning (RSME median = 25.50) yet the worst operational performance (completion time median = 490 s; error rate median = 8), while video yielded consistently low workload and the highest accuracy. The humanoid robot occupied a distinct middle position, generating the highest HRV ratios during learning (median = 1.02)—physiological evidence of social engagement not captured by self-report—alongside moderate performance outcomes comparable to video. Crucially, VR's low perceived effort masked a hidden physiological cost that subjective measures entirely failed to detect, demonstrating that self-report alone is insufficient to evaluate human–AI interaction quality. The findings surface a fundamental design principle for human–AI symbiosis: different interaction modalities engage distinct cognitive and autonomic processes, and only triangulated measurement can reveal this. Together, the results provide empirically grounded guidance for designing adaptive human–AI training systems in Industry 5.0 manufacturing.

Since the introduction of Industry 4.0, digital twin technology has advanced dramatically, and its applications have been extensively discussed. Through digital twins, real-time connected production systems are anticipated to be more efficient, resilient, and sustainable, facilitating communication and connectivity between digital and physical systems. However, challenges remain in environmental performance and in integration of virtual reality (VR) and artificial intelligence (AI). Further exploration of digital twin technologies is needed to validate the real-world impact and benefits. This paper investigates these challenges by implementing a real-time digital twin based on the ISO 23247, Automation Systems and Integration - Digital Twin Framework for Manufacturing, (Digital twin framework for manufacturing), connecting the physical factory and simulation software with VR capabilities. This digital twin system provides cognitive assistance and a user-friendly interface for operators, thereby improving cognitive ergonomics. The connection of the Internet of Things platform allows the digital twin for real-time bidirectional communication, collaboration, monitoring, and assistance. A laboratory-scale drone factory was used as the digital twin application to test and evaluate the ISO 23247 standard as well as its potential benefits. Additionally, AI integration and environmental performance key performance indicators have been considered as the next stages to improve VR-integrated digital twins. This paper further discusses how digital twins contribute to the improvement of the environmental performance of production systems, primarily from the perspective of data inventory management. Supported by a solid theoretical foundation and a laboratory-scale demonstration of the VR-integrated digital twins in a drone framework, the paper bridges the gap between possible technological integrations and the advancement of digital twins framework based on ISO 23247.

The European industry’s transition towards the human-centric paradigm of Industry 5.0 necessitates the development of advanced training systems that are not only effective but also inclusive. However, current industrial training practices lack a systematic understanding of operators’ diverse needs and often fail to adapt to individual worker characteristics, resulting in standardized approaches that inadequately serve multicultural, multigenerational workforces with varying technical competencies. A primary challenge is creating on-the-job training solutions that adapt to a diverse workforce, comprising individuals with varying cultural backgrounds, technical skills, and cognitive abilities—while minimizing mental workload and accelerating the learning curve. This research addresses this challenge by designing, implementing, and evaluating a humanoid social robot (HSR) demonstrator that provides adaptive, culturally aware, and engaging verbal instructions for industrial tasks. HSR is defined as “human-made technologies that can take physical or digital form, resemble people in form or behavior to some degree, and are designed to interact with people”. It emphasizes that HSRs exhibit both form anthropomorphism (human-like voice or appearance) and behavioral anthropomorphism (gestures, spoken messages, nonverbal expressions). To enhance the social interaction functionality of the humanoid social robot, a natural language model is introduced to enable fluent communication, and computer vision is used to enable eye contact and expression. The HSR’s instructions are conceptualised as the strategy for guidance. We conducted a comparative study between a baseline prototype and an improved version, focusing on four key adaptive dimensions: (1) speech pace control; (2) accent and dialect adaptation; (3) facial expression timing; and (4) instruction granularity. The evaluation will measure usability and instructional efficacy through a combination of quantitative metrics (task completion time, error rate, subjective metrics, and physiological data). The findings of this study imply that personalized HSR interfaces do not lower the mental workload significantly; standardizing HSR interfaces while allowing some flexibility during learning may optimize mental effort and improve the training process. The contribution is a practical, human-centered method to developing adaptive speech interactions with humanoid social robots, with tangible recommendations for inclusive, efficient upskilling in industry.

Industry 5.0 revolution is prioritizing human-centricity and adapting technologies to augment shopfloor workers’ cognitive ergonomics. To provide a user-friendly, efficient virtual planning tool, virtual reality (VR) is adopted by the industry to provide a virtual work environment for layout planning, design reviews, and training use cases. However, the user interface (UI) of VR programs is not yet standardized for universal design, and thus causes issues such as difficult scalable technology adoption due to high mental workload. Creating intuitive and accessible interfaces is a key challenge in VR. As the complexity of VR platforms grows, it is vital that users can effectively explore and engage with them. Improved UI design may improve the whole user experience, making VR more accessible, scalable, and attractive to a larger audience. Navigation in a VR environment can impose a significant mental workload on users, affecting their cognitive capacities, including layout perception, navigation, attention for collaboration, and response/completion time. This study aims to identify and assess the UI design features of a virtual work environment for manufacturing regarding mental workload, spatial navigation, and performance-based evaluation for human centricity. Three design features of typical mini maps examples, which are extracted from the literature and the gamification industry, are portability, tangibility, and dimensionality. By identifying the association between design features and user navigation experience, we may observe patterns for broader VR user interface standardization that address human factors. This study employed a qualitative approach to assess five different prototypes of interactive map designs, categorized into three design features, involving both students and industry practitioners, and resulting in 114 valid data collection sessions in the prototype-based experiment. Reducing mental workloads in VR interfaces can increase efficiency and user satisfaction in Industry 5.0 through intentional use of specific design features—portability proved to have the most significant impact, consistently reducing mental, physical and temporal demands, and frustration, while simultaneously improving performance, layout perception, navigation, and collaborative efficacy. The study provides effective design feature identification, highlighting the importance of user-centric approaches in VR development for cognitive ergonomics.

Over the past decade, the value and potential of VR applications in manufacturing have gained significant attention in accordance with the rise of Industry 4.0 and beyond. Its efficacy in layout planning, virtual design reviews, and operator training has been well-established in previous studies. However, many functional requirements and interaction parameters of VR for manufacturing remain ambiguously defined. One area awaiting exploration is spatial recognition and learning, crucial for understanding navigation within the virtual manufacturing system and processing spatial data. This is particularly vital in multi-user VR applications where participants’ spatial awareness in the virtual realm significantly influences the efficiency of meetings and design reviews. This paper investigates the interaction parameters of multi-user VR, focusing on interactive positioning maps for virtual factory layout planning and exploring the user interaction design of digital maps as navigation aid. A literature study was conducted in order to establish frequently used technics and interactive maps from the VR gaming industry. Multiple demonstrators of different interactive maps provide a comprehensive A/B test which were implemented into a VR multi-user platform using the Unity game engine. Five different prototypes of interactive maps were tested, evaluated and graded by the 20 participants and 40 validated data streams collected. The most efficient interaction design of interactive maps is thus analyzed and discussed in the study.

Over the past decade, the value and potential of VR applications in manufacturing have gained significant attention in accordance with the rise of Industry 4.0 and beyond. Its efficacy in layout planning, virtual design reviews, and operator training has been well-established in previous studies. However, many functional requirements and interaction parameters of VR for manufacturing remain ambiguously defined. One area awaiting exploration is spatial recognition and learning, crucial for understanding navigation within the virtual manufacturing system and processing spatial data. This is particularly vital in multi-user VR applications where participants' spatial awareness in the virtual realm significantly influences the efficiency of meetings and design reviews. This paper investigates the interaction parameters of multi-user VR, focusing on interactive positioning maps for virtual factory layout planning and exploring the user interaction design of digital maps as navigation aid. A literature study was conducted in order to establish frequently used technics and interactive maps from the VR gaming industry. Multiple demonstrators of different interactive maps provide a comprehensive A/B test which were implemented into a VR multi-user platform using the Unity game engine. Five different prototypes of interactive maps were tested, evaluated and graded by the 20 participants and 40 validated data streams collected. The most efficient interaction design of interactive maps is thus analyzed and discussed in the study.