Francesco De Pace

Over the last decade, robot telemanipulation has been increasingly utilized in various applications so as to replace human operators in hazardous or remote environments. However, the telemanipulation of robots remains a challenging task, especially when high precision and dexterity are required. Peg-in-hole tasks are considered some of the most challenging tasks as they require high precision. To facilitate the execution of such complex tasks, this paper introduces and compares different semi-autonomous strategies for virtual reality (VR) based remote robotic telemanipulation of a robot arm. Four modalities of robotic telemanipulation with varying degrees of autonomy are presented and thoroughly compared. Finally, the comparative user study highlights the differences between the proposed modalities and showcases the advantages and disadvantages of each approach in detail.

Understanding how people effectively perform actions together is fundamental when designing Collaborative Mixed Reality (CMR) applications. While most of the studies on CMR mostly considered either how users are immersed in the CMR (e.g., in virtual or augmented reality) or how the physical workspace is shared by users (i.e., distributed or collocated), little is known about how their combination could influence user’s interaction in CMR. In this paper, we present a user study (n = 46, 23 pairs) that investigates the effect of the mixed reality setup on the user’s immersion and spatial interaction during a joint-action task. Groups of two participants had to perform two types of joint actions while carrying a virtual rope to maintain a certain distance: (1) Gate, where participants had to pass through a virtual aperture together, and (2) Fruit, where participants had to use a rope to slice a virtual fruit moving in the CMR. Users were either in a distributed or collocated setup and either immersed in virtual or augmented reality. Our results showed that the immersion type and location setup altered users’ proxemics as well as the users’ subjective experience. In particular, we noticed better task performance when users were in augmented reality and more considerable distances between players while interacting in a distributed setup. These results contribute to the understanding of joint action in CMR and are discussed to improve the design of CMR applications.

Storyboards are one of the most effective methods to pre-visualize a story: a sequence of illustrations or pictures portraying settings, characters, actions, and events. This graphical representation is a sort of roadmap that guides users throughout the story. The storyboarding process is a key step for both movie and videogame productions, allowing creators to visualize the scenes and find potential problems before they occur. Whereas storyboards are usually drawn or sketched in 2D, recent technological advances allow storytellers to cope with 3D characters and scenes. A 3D approach overcomes 2D methodologies as the storytellers can achieve a better understanding of spaces, objects, lights, and camera movements. Moreover, extended reality (XR) technologies can enhance these benefits by adding a feeling of immersion otherwise not achievable. This manuscript investigates and compares two XR-based storyboarding solutions, which share a common graphical user interface. The proposed system has been tested involving both domain experts and students in a tabletop scale scenario. Experimental results clearly indicate that the virtual reality-based solution outperforms the augmented reality one from the usability point of view; moreover, a lower workload is experienced when an input interface based on physical controllers is available.

This article presents a brain–computer interface (BCI) coupled with an augmented reality (AR) system to support human–robot interaction in controlling a robotic arm for pick-and-place tasks. BCIs can process steady-state visual evoked potentials (SSVEPs), which are signals generated through visual stimuli. The visual stimuli may be conveyed to the user with AR systems, expanding the range of possible applications. The proposed approach leverages the capabilities of the NextMind BCI to enable users to select objects in the range of the robotic arm. By displaying a visual anchor associated with each object in the scene with projected AR, the NextMind device can detect when users focus their eyesight on one of them, thus triggering the pick-up action of the robotic arm. The proposed system has been designed considering the needs and limitations of mobility-impaired people to support them when controlling a robotic arm for pick-and-place tasks. Two different approaches for positioning the visual anchors are proposed and analyzed. Experimental tests involving users show that both approaches are highly appreciated. The system performances are extremely robust, thus allowing the users to select objects in an easy, fast, and reliable way.

Global Positioning Satellite (GPS) systems sample points on the Earth’s surface with meter accuracy. Real-Time Kinematic (RTK) devices improve GPS performances by providing real-time correction data from ground stations, achieving centimeter accuracy. Reliable tracking approaches are essential for Augmented Reality (AR) applications, especially for outdoor scenarios, which still present unsolved challenges. AR handheld tracking capabilities have been greatly improved by integrating visual tracking approaches with RTK devices, whereas little is known about combining wearable AR interfaces with RTK systems. Although wearable AR devices are intrinsically designed for AR applications, their performance dramatically reduces in large outdoor areas, comprising the user experience. Hence, this paper provides a rigorous evaluation of a small-size RTK device that does not need any additional software integration to collect positional data. The main goal of the assessment is to verify whether its integration with a wearable AR device is advantageous or not. The evaluation has been performed considering both static and dynamic scenarios in open-sky and urban areas. The results show that the RTK device can achieve 1 cm accuracy when used in open-sky areas. In contrast, its accuracy dramatically reduces in the proximity of buildings and obstacles, showing average errors ranging from 76 to 2561%. Since wearable AR devices have an average accuracy of 2 cm, the outcomes indicate that RTK devices should be combined with wearable AR devices only when the RTK device is far from obstacles. On the contrary, the positional data should be completely avoided when barriers surround the RTK device.

Design and computer graphics curricula for tertiary education include 3-D modeling skills. Students have to learn to represent (complex) 3-D objects by means of parametric surfaces or polygonal meshes. Three-dimensional modeling may be a complex task and students have to be able to accrue a certain deal of experience in the field before passing the exam. The main difficulties the students have to face mainly concern the comprehension of 3-D shapes and the choice of the most appropriate modeling techniques. This paper proposes a framework to support bachelor students of the design degree in modeling 3-D objects by means of parametric surfaces. The proposed framework provides two augmented reality (AR) apps for smartphones and a web portal. The mobile AR apps allow students to deeply visualize 3-D object shapes, thus performing a set of guided exercises in order to take practice in basic modeling techniques. On the other hand, the web portal allows students to share their models with teachers and classmates to get feedback and comments. Preliminary results show the effectiveness of the proposed solution as all volunteers involved in the experimental phase achieved better results after using the tools. Moreover, the students' opinions about the proposed framework are positive.