Youjin Sung

Self-directed rehabilitation in everyday environments remains difficult due to limited feedback and the absence of contextually grounded guidance. In response, we present RehabGen, a generative vision-language model that generates personalized exercise plans contextualized to a patient's egocentric image and available everyday objects. We followed an expert-in-the-loop (EITL) framework: gathering domain requirements through expert interviews and focus group interviews ( N =8), constructing a 1.66k multimodal preference dataset, and aligning the model using supervised fine-tuning (SFT) and Direct Preference Optimization (DPO). In Expert Validation ( N =23), DPO achieved a 57.6% item-level win rate over the Baseline, with higher scores in Personalized Recommendation (85.7%), Identify Purpose (63.6%), and Recommendation Satisfaction (66.7%). In Patient Experience Evaluation ( N =8), both models outperformed the Baseline. This work contributes an EITL framework and open-source models available at http://rehabgen.hcitech.org/ for clinically grounded, context-aware rehabilitation planning.

Designing expressive haptic feedback in XR remains challenging due to limited authoring tools and high expertise barriers. We address this through two complementary systems: HapticPilot, a VR-based toolkit for posture-aware haptic sketching that adapts vibrotactile designs across dynamic hand configurations, and HapticGen, the first text-to-vibration generative model optimized via expert preference alignment. Together, these systems reduce iteration costs, lower the entry barrier for novice designers, and support scalable tactile content creation. This work further lays the groundwork for multimodal, context-aware haptic generation that integrates cross-modal cues for adaptive and personalized tactile experiences in XR.

Extended Reality(XR) technologies rapidly evolve, offering remarkable opportunities for creating immersive and interactive experiences. However, the integration of effective haptic feedback remains a significant challenge. We aim to explore and enhance haptic experience design based on three major themes: wearable hardware, rendering methods, and design systems incorporating algorithms. Through this comprehensive approach, we seek to improve the quality of haptic interactions in XR environments significantly.

In this Interactivity, we present a lab demo on adaptive and immersive wearable interfaces that enhance extended reality (XR) interactions. Advances in wearable hardware with state-of-the-art software support have great potential to promote highly adaptive sensing and immersive haptic feedback for enhanced user experiences. Our research projects focus on novel sensing techniques, innovative hardware/devices, and realistic haptic rendering to achieve these goals. Ultimately, our work aims to improve the user experience in XR by overcoming the limitations of existing input control and haptic feedback. Our lab demo features three highly enhanced experiences with wearable interfaces. First, we present novel sensing techniques that enable a more precise understanding of user intent and status, enriched with a broader context. Then, we showcase innovative haptic devices and authoring toolkits that leverage the captured user intent and status. Lastly, we demonstrate immersive haptic rendering with body-based wearables that enhance the user experience.

The emergence of vibrotactile feedback in hand wearables enables immersive virtual reality (VR) experience with whole-hand haptic rendering. However, existing haptic rendering neglects inconsistent sensations caused by hand postures. In our study, we observed that changing hand postures alters the distribution of vibrotactile signals which might degrade one's haptic perception. To address the issues, we present HapticPilot which allows an in-situ haptic experience design for hand wearables in VR. We developed an in-situ authoring system supporting instant haptic design. In the authoring tool, we applied our posture-adaptive haptic rendering algorithm with a novel haptic design abstraction called phantom grid. The algorithm adapts phantom grid to the target posture and incorporates 1D & 2D phantom sensation with a unique actuator arrangement to provide a whole-hand experience. With this method, HapticPilot provides a consistent haptic experience across various hand postures is available. Through measuring perceptual haptic performance and collecting qualitative feedback, we validated the usability of the system. In the end, we demonstrated our system with prospective VR scenarios showing how it enables an intuitive, empowering, and responsive haptic authoring framework.

In this study, we explore the effect of hand posture on haptic experience within the hand. With the wide acceptability of using the hand as an input in the XR, it is important to find out the positive and negative effects of hand postures on the haptic experience. We measured the vibration intensity of the hand using an accelerometer on various hand postures. Our results showed that distinctive hand postures alter the vibrotactile actuator’s tactile feedback across the hand.