Joong-Jae Lee

The combination of immersive virtual reality (VR) environments and distance education has led to a new educational paradigm. In this study, an immersive VR-based distance education system is proposed to enable multiple remote users to send, share, and experience images from the Solar Dynamics Observatory (SDO) via streaming. In contrast to the conventional system in which only experts use SDO data, the proposed system provides a head-mounted-display-based visualization platform that can be easily used by experts and non-experts. Real-time SDO image streaming must be possible to realistically observe changes in the Sun and increase involvement. Thus, multichannel-based SDO image transmission was applied to increase the network bandwidth utilization. To improve the social presence of participants, realistic avatar models controlled by the motions synchronized with the user are provided. In addition, free communication is possible through verbal interactions. This allows multiple remote users to participate simultaneously without having to be physically present. A user study with 20 subjects showed that the participants could observe SDO images in a more immersive manner by using the proposed system. In addition, they experienced social presence because of the user avatar models and an enriched educational experience by conversing with and listening to experts.

There is an increasing demand for the manipulation of virtual objects in 3D virtual reality (VR) space, which begins with the user selecting a desired object. Existing selection methods aim to provide an intuitive and natural experience that minimizes user fatigue. Although typical ray-casting methods are effective when objects in the virtual space are sparsely or evenly placed, the selection becomes challenging when these objects are smaller or farther away, particularly when they overlap. In this study, we propose a method to address these challenges via an effective selection process. Based on 3D mirroring, the proposed method provides a new interaction metaphor that allows for a more efficient selection of occluded virtual objects. We conduct systematic experiments with 21 subjects and use various levels of target visibility determined by object size, spatial density, and degrees of occlusion. The results indicate that, in occluded 3D environments, the proposed 3D mirrored selection process outperforms traditional gaze-supported selection processes and offers better user satisfaction.

Attention capability is an essential component of human–robot interaction. Several robot attention models have been proposed which aim to enable a robot to identify the attentiveness of the humans with which it communicates and gives them its attention accordingly. However, previous proposed models are often susceptible to noisy observations and result in the robot’s frequent and undesired shifts in attention. Furthermore, most approaches have difficulty adapting to change in the number of participants. To address these limitations, a novel attentiveness determination algorithm is proposed for determining the most attentive person, as well as prioritizing people based on attentiveness. The proposed algorithm, which is based on relevance theory, is named the Scalable Hidden Markov Model (Scalable HMM). The Scalable HMM allows effective computation and contributes an adaptation approach for human attentiveness; unlike conventional HMMs, Scalable HMM has a scalable number of states and observations and online adaptability for state transition probabilities, in terms of changes in the current number of states, i.e., the number of participants in a robot’s view. The proposed approach was successfully tested on image sequences (7567 frames) of individuals exhibiting a variety of actions (speaking, walking, turning head, and entering or leaving a robot’s view). From these experimental results, Scalable HMM showed a detection rate of 76% in determining the most attentive person and over 75% in prioritizing people’s attention with variation in the number of participants. Compared to recent attention approaches, Scalable HMM’s performance in people attention prioritization presents an approximately 20% improvement.

This paper presents a stereo camera-based head-eye calibration method that aims to find the globally optimal transformation between a robot’s head and its eye. This method is highly intuitive and simple, so it can be used in a vision system for humanoid robots without any complex procedures. To achieve this, we introduce an extended minimum variance approach for head-eye calibration using surface normal vectors instead of 3D point sets. The presented method considers both positional and orientational error variances between visual measurements and kinematic data in head-eye calibration. Experiments using both synthetic and real data show the accuracy and efficiency of the proposed method.

We propose a natural object selection technique of 3D virtual object using user's gaze and hand pose information. The proposed technique aims to provide fast and realistic interaction with the virtual objects in the grasp. The experimental results show that the proposed technique enables a user to naturally select 3D virtual object fast.