Paulo Dias

Stroke survivors are often left with physical and cognitive limitations. Effective rehabilitation is essential to promote functional recovery and improve quality of life. However, existing approaches are limited by their repetitive/monotonous nature, which can lead to decreased motivation, emotional disengagement, and depression, as well as a lack of personalization and little to no social interaction. A five-month longitudinal study conducted at a rehabilitation center, employing a virtual reality supermarket, led to the creation of a research line focused on social interaction. Building on this, a broader extended reality rehabilitation framework was introduced, aimed at fostering motivation through multiuser experiences. We report initial results on healthcare professionals’ and stroke survivors’ acceptance, and explore the impact of collaborative versus competitive dynamics. Finally, we discuss the promising role of augmented reality for daily living environments.

Nowadays, despite the advancements in several technological areas, the education process of various subjects shows minimal evolution from the approaches used in prior years. In light of these, some fields struggle to capture the student’s attention and motivation, in particular, when the subject addresses remote locations that students are unable to visit and relate to. Therefore, an opportunity exists to explore novel technologies for such scenarios. This work introduces an educational approach that integrates 3D Reconstruction, Virtual Reality (VR), and a Large Display to enrich Geoscience learning at the university level. In this teacher-centric approach, manipulation of virtual replicas of real-world geological sites can be performed, creating an immersive yet asymmetric collaborative environment for students in the classroom. The teacher’s VR interactions are mirrored on a large display, enabling clear demonstrations of complex concepts. This allows students, who cannot physically visit these locations, to explore and understand the sites more deeply. To evaluate the effectiveness of this approach, a user study was conducted with 20 participants from Geoscience and Computer Science disciplines, comparing the VR-based method with a conventional approach. Analysis of the collected data suggests that, across multiple relevant dimensions, participants generally favored the VR condition, highlighting its potential for enhancing engagement and comprehension.

Remote assistance has become integral in today’s work environments, enabling offsite experts to guide team members in need of assistance. One technology that has gained momentum is mixed reality (MR), combining virtual and augmented reality elements, bridging physical distances, and facilitating knowledge sharing. One critical area is the representation of remote experts. Understanding how different representations influence the learning process is essential for improving the shared experience. Effective representation can enhance communication, trust, and the perceived expertise of the remote expert, thereby improving the learning and collaboration experiences. This study focuses on evaluating how different representations affect various dimensions of collaboration. The study involved 57 participants and examined six different conditions: no visual representation, representation using emojis, representation using a cartoon avatar, representation using a robot avatar, representation using a realistic avatar, and live stream using video.

Reflective surfaces are notoriously difficult to detect and reconstruct, as they reflect light from surrounding objects. Thus, the quality of 3D reconstructions and the performance of downstream computer vision tasks, such as navigation, depth prediction, or object detection are severely impaired by the prevalence of mirrors in indoor environments. This paper proposes a novel reflection-aware method for 3D mirror segmentation and pose estimation, based on the reinterpretation of captured RGB-D data. It is a top-down approach, where information consensus, unsupervised learning, and ray casting are employed, formulating mirror pose estimation as an optimization problem. Experimental results show that the proposed approach significantly outperforms the existing benchmarks for RGB and RGB-D-based 3D mirror segmentation and pose estimation.

With the rise in popularity of different human-centred applications using 3D reconstruction data, the problem of generating photo-realistic models has become an important task. In a multiview acquisition system, particularly for large indoor scenes, the acquisition conditions will differ along the environment, causing colour differences between captures and unappealing visual artefacts in the produced models. We propose a novel neural-based approach to colour correction for indoor 3D reconstruction. It is a lightweight and efficient approach that can be used to harmonize colour from sparse captures over complex indoor scenes. Our approach uses a fully connected deep neural network to learn an implicit representation of the colour in 3D space, while capturing camera-dependent effects. We then leverage this continuous function as reference data to estimate the required transformations to regenerate pixels in each capture. Experiments to evaluate the proposed method on several scenes of the MP3D dataset show that it outperforms other relevant state-of-the-art approaches.

The use of Pervasive Augmented Reality (AR) in industrial settings, particularly in the context of Industry 4.0, is a growing area of research. Industry 4.0 is characterized by the integration of advanced technologies, including AR, to enable smart, connected, and autonomous systems in manufacturing and other industries. This study focuses on two systems that use Pervasive AR for logistics and data monitoring on assembly lines. Such systems were designed using cooperative Human-Centered Design (HCD) methodologies in collaboration with partners from the industry sector. We report lessons learned and guidelines on the development of AR systems for industrial environments. In addition, a user study based on a data monitoring laboratory realistic procedure with 8 participants is described. The study compares the usability and effectiveness of a dashboard on Pervasive AR and a web tablet. Overall, results emphasize that AR was better suited when requiring spatial awareness and interventions, while traditional web tablet was better for tasks that require more interaction with the content and faster data monitoring. These findings had implications for the design of industrial tools, such as prototype improvements and idealize new methods, which can guide future efforts to develop and deploy AR applications.