Sang Ho Yoon

Eye gaze has become an essential input for spatial computing, but its coarse targeting and saccadic nature limit precision and complicate continuous interactions such as dragging, especially under user motion. Gaze+pinch has also become standard in XR for its convenience, yet mid-air gestures remain imprecise, fatiguing, and socially unacceptable. These limitations underscore the need for an approach that preserves the speed of gaze while enabling stable, fine control. We present GazeTune, a cascaded multimodal interaction technique combining gaze and touch to refine gaze-based selection and manipulation. Touch serves as a refinement channel within gaze pointing, allowing precise cursor and target control. Our work investigates how gaze-and-touch enhances dragging and mitigates Motion-Induced instability. In a study (N=20), we compared GazeTune against gaze-only and gaze-pinch methods in 2D dragging. Results show that GazeTune achieves significantly lower error with comparable execution time, validating its effectiveness and balanced trade-off between time and accuracy.

With the advancement of haptic interfaces, recent studies have focused on enabling detailed haptic experiences in virtual reality (VR), such as fluid-haptic interaction. However, rendering forces from fluid contact often causes a high-cost computation. Given that motion-induced fluid feedback is crucial to the overall experience, we focus on hand-perceivable forces to enhance underwater haptic sensation by achieving high-fidelity rendering while considering human perceptual capabilities. We present a new multimodal (tactile and kinesthetic) haptic rendering pipeline. Here, we employ drag and added mass forces by dynamically adapting to the user's hand movement and posture with pneumatic-based haptic gloves. We defined decaying and damping effects to indicate fluid properties caused by inertia and confirmed their significant perceptual impacts compared to using only physics-based equations in a perception study. By modulating pressure variations, we reproduced fluid smoothness via exponential tactile deflation and light fluid mass via linear kinesthetic feedback. Our pipeline enabled richer and more immersive VR underwater experiences by accounting for precise hand regions and motion diversity.

Self-directed rehabilitation in everyday environments remains difficult due to limited feedback and the absence of contextually grounded guidance. In response, we present RehabGen, a generative vision-language model that generates personalized exercise plans contextualized to a patient's egocentric image and available everyday objects. We followed an expert-in-the-loop (EITL) framework: gathering domain requirements through expert interviews and focus group interviews ( N =8), constructing a 1.66k multimodal preference dataset, and aligning the model using supervised fine-tuning (SFT) and Direct Preference Optimization (DPO). In Expert Validation ( N =23), DPO achieved a 57.6% item-level win rate over the Baseline, with higher scores in Personalized Recommendation (85.7%), Identify Purpose (63.6%), and Recommendation Satisfaction (66.7%). In Patient Experience Evaluation ( N =8), both models outperformed the Baseline. This work contributes an EITL framework and open-source models available at http://rehabgen.hcitech.org/ for clinically grounded, context-aware rehabilitation planning.

Extended reality (XR) systems now use everyday physical surfaces as touch input spaces. However, existing systems rely primarily on flat surfaces due to the given device form factor and sensing capabilities. Despite advancements in sensing technologies that allow touch input across surfaces with diverse curvatures, the effects of surface curvature on user behavior and performance in indirect XR touch interaction remain underexplored. To this end, we present CurvedTouch, a study that investigates the effects of surface curvature and cursor mapping methods on XR indirect touch interaction. We evaluated 4 surface curvatures (flat, R = 75mm, 60mm, and 45mm) and two cursor mapping methods (Projection-based and Distance-based) in a within-subject study by measuring task performance, behavioral pattern, workload, and usability. 16 participants performed the indirect Tap, Drag, and Trace task under certain conditions. Our results reveal that the task performance and usability significantly dropped for a curvature between R = 45mm and R = 60mm. As curvature increases, users change their input strategy from finger-level motion to whole-hand movement. Unlike prior findings in direct touch interaction, we found that minimum button size increases on a high curvature surface in an indirect touch scenario. For the cursor mapping method, distance-based mapping showed higher usability under high curvature. Based on these findings, we came up with design guidelines for interaction type, button size, interaction area, and cursor mapping method for the XR indirect touch interface on a curved surface.

As extended reality (XR) technologies rapidly become as ubiquitous as today’s mobile devices, supporting one-handed interaction becomes essential for XR. However, the prevalent Gaze + Pinch interaction model partially supports unimanual interaction, where users select, move, and rotate objects with one hand, but scaling typically requires both hands. In this work, we leverage the spatial alignment between gaze and hand as a mode switch to enable single-handed pinch-to-scale. We design and evaluate several techniques geared for one-handed scaling and assess their usability in a compound translate-scale task. Our findings show that all proposed methods effectively enable one-handed scaling, but each method offers distinct advantages and trade-offs. To this end, we derive design guidelines to support futuristic 3D interfaces with unimanual interaction. Our work helps make eye-hand 3D interaction in XR more mobile, flexible, and accessible.