M. Kilchenman O'MAlley

Vision and sound often dominate human-robot interaction systems, but touch can convey nuanced, real-time task and environment information. Vibrotactile feedback is increasingly common in commercial devices, and the wrist offers a promising location for wearable haptics: it is unobtrusive and sensitive to tactile input. While prior work has studied vibrotactile feedback in navigation and action confirmation tasks, quantifying information transfer for wrist-worn vibrotactile cues remains underexplored. This study estimates information transfer on the wrist (bits per stimulus) using a confusion-matrix-based metric alongside accuracy and pleasantness ratings. We conducted three linked studies with the same participants: identifying single-tactor cues spanning frequency, amplitude, and modulation; identifying the same cues under sequential vibrotactile masking and during a concurrent typing task; and identifying multi-tactor spatiotemporal patterns and rating pleasantness. Participants reliably discriminated complex signals, with amplitude and feature interactions playing key roles. Information transfer ranged from 0.75 bits/stimulus for multi-tactor patterns to 2.28 bits/stimulus for single-tactor testing across all participants (2.56 for experienced participants); masking and typing yielded 1.64 and 1.85 bits/stimulus, respectively. Performance was not solely driven by amplitude-normalized intensity.

Hybrid FES-exoskeleton systems can drive users through trajectories that mimic functional upper limb movements with higher accuracy and reduced motor torque compared to either system alone, situating them well as assistive devices. Limitations in FES-driven movement accuracy, exoskeleton size, and electrical energy requirements prevent the widespread adoption of hybrid systems as assistive devices in real-world settings. Additionally, these systems are limited in their adoption due to difficulties in efficiently coordinating the two subsystems, and given the heterogeneity of motor capabilities across neurologically-impaired individuals. We present a methodology for nonlinear trajectory optimization to create feasible, personalized trajectories between desired set-points that preemptively considers each user’s dynamic constraints. We evaluate personalized and minimum jerk trajectories experimentally with eight neurologically intact participants and in a case study with one individual with a cervical spinal cord injury, extending our prior simulation-based validation. The personalized trajectories maintain significant exoskeleton torque reduction from the exoskeleton-alone system to the hybrid FES-exoskeleton system without sacrificing functional tracking accuracy.

Training for complex motor tasks, such as those encountered in minimally invasive surgery, benefits from effective performance feedback mechanisms to accelerate skill acquisition and ensure retention. Prior work has demonstrated that haptic feedback based on movement smoothness quantified by the metric spectral arc length (SPARC), when provided in real-time as trainees perform complex motor tasks, can cause beneficial changes in task completion strategies resulting in faster completion times without loss of accuracy. The concept of movement smoothness is abstract, however, and more intuitive measures of movement smoothness like idle time and average velocity can be good alternatives to SPARC. Here, we demonstrate the effect of real-time objective performance feedback of movement smoothness, conveyed through a vibrotactile cue encoding alternative measures of movement smoothness, compared to feedback based on SPARC. Subjects receiving smoothness-based feedback based on average velocity performed the task fastest, but their accuracy was lower than the other two groups. We evaluated the effect of removing feedback for additional trials, and showed that performance improvements ceased. After training, the three groups were indistinguishable from each other.

Robot-assisted minimally invasive surgery (RAMIS) offers numerous benefits over traditional open surgery, resulting in greater prevalence of use and range of approved procedures. The proliferation of RAMIS has highlighted a need for effective, robust, and objective methods for assessing robotic surgical skills. Traditionally, assessment has relied on expert observation using structured grading rubrics. Although validated and widely used, this method is also resource intensive and subject to reviewer bias. In response, recent work has explored the potential for more robust assessment methods, including the development of skill-based metrics, crowd-sourced assessment techniques, and automated evaluation systems. This review summarizes recent developments in robotic surgical technical skill assessment, focusing on studies using the da Vinci platform. Assessment methods are grouped into four categories: structured rubrics, skill-based metrics, crowd-sourcing techniques, and automated assessment models. Trends of note include adaptation of established rubrics for specific areas of specialty, the implementation of deep learning models for automated assessment, and a move to integrate crowd-sourcing platforms for efficient and inexpensive evaluation. While traditional grading rubric structures remain the standard, multilevel assessment strategies and objective feedback systems are gaining traction. Future work should seek to integrate task- and movement-based assessment into procedure-level evaluations to create more robust and generalizable models for assessment. These advances show a shift towards data-driven and objective assessment methods, which could improve surgical training and patient outcomes.

Individuals who lack tactile and/or proprioceptive sensations in their lower limbs commonly report postural and locomotive imbalance. To mitigate imbalance, haptic feedback has been implemented using devices that employ external sensors and waist-worn actuators for sensory augmentation. Analyses of the effectiveness of these devices have primarily focused on balance outcomes and generally disregard the potential effect of the user's varying activities or anthropometric data on their perception of the haptic feedback. Since motor activity can influence haptic perception, we investigate vibrotactile cue localizability in a waist-worn haptic device for two conditions, standing and walking on a treadmill at a self-selected speed. In this preliminary study, ten participants without sensorimotor deficits wore a waistband equipped with seven vibrotactile actuators. Vibrotactile cues were played at randomized locations during standing and walking, and participants reported the perceived stimulation location. In addition, we recorded relevant anthropometric data for each participant. In both standing and walking conditions, participants correctly localized 51% of the vibrotactile cues on average. When considering zonal accuracy, or localization within one vibrotactor position, participants, on average, achieved an accuracy of 91 % in standing and 90% in walking.

Wearable haptic devices can provide haptic feedback in both active and passive touch interactions. In virtual or extended reality environments, wearable devices also enable referred haptic feedback, where touch sensation expected in one place on the body is directed to another. How people perceive the distinction between these feedback forms has been relatively under-explored, especially considering the additional role that vision plays in our multisensory understanding of the world. To explore how active and passive touch affect the perception of referred haptic feedback, we conducted an experiment in VR where participants chose the stiffer of two springs in a 2-interval, 2-alternative forced-choice design. We find that participants can be categorized into two groups based on the different strategies they employ for making the decision - those with a haptic or a visual prior, similar to related work. We also considered both active and passive feedback conditions. Notably, people are more accurate in judging haptic stiffness during the active case. Our results have implications for designers of virtual systems and simulations where users receive various sensory inputs, both via active and passive interactions, with potential mismatches due to latency, bandwidth, or design issues.

Hybrid Functional Electrical Stimulation (FES)- exoskeleton systems are emerging as promising assistive technologies because they can drive users through trajectories that mimic functional movements with higher precision than FES-alone and reduced torque consumption compared to an exoskeleton alone. Limitations in FES movement accuracy and exoskeleton power and size requirements from high torque commands prevent hybrid systems as prevalent assistive devices in real-world settings. Additionally, difficulties in effectively coordinating the two subsystems and heterogeneity of motor capabilities across neurologically impaired individuals limit their widespread adoption. This letter presents a methodology for nonlinear trajectory optimization to create feasible, personalized trajectories between desired set-points that were evaluated in simulation with seven neurologically intact user models. The personalized trajectories ensure dynamic feasibility while maintaining significant exoskeleton torque reduction in the hybrid FES-exoskeleton system compared to the exoskeleton-alone. The personalized trajectories also maintain similar tracking accuracy compared to minimum jerk trajectories.

Holistic consideration of the human and the robot is necessary to overcome hurdles in human-robot interaction.