Hojung Choi

Skin deformation haptic devices worn on the finger pad provide realistic touch feedback during interactions with virtual objects. Two primary challenges in creating such devices are: first, making a multidegree-of-freedom device (DoF) that is small and lightweight so it does not encumber the wearer and second, providing accurate control of forces displayed to the finger pad. This work presents a 4-DoF finger pad haptic device, called Fourigami, that addresses these challenges. We address the first challenge using origami manufacturing methods and pneumatic actuation to fabricate a 25 g prototype that displays normal, shear, and twist and can be easily worn on the finger pad. We address the second challenge using a low-profile, 6-DoF, force/torque sensor to control forces displayed to the finger. Fourigami has a bandwidth ranging from 2 to 4 Hz depending on direction, and when acting on a human finger, it exerts forces ranging from $\pm$ 1.0 N in shear, 4.2 N in normal, and $\pm$ 4.2 N $\cdot$ mm of twist. Finally, we demonstrate the device’s efficacy when rendering haptic feedback to a user tracking a sinusoidal trajectory and a trajectory representing interactions with a virtual object.

Imitation learning requires high-quality demonstrations consisting of sequences of state-action pairs. For contact-rich dexterous manipulation tasks that require dexterity, the actions in these state-action pairs must produce the right forces. Current widely-used methods for collecting dexterous manipulation demonstrations are difficult to use for demonstrating contact-rich tasks due to unintuitive human-to-robot motion retargeting and the lack of direct haptic feedback. Motivated by these concerns, we propose DexForce. DexForce leverages contact forces, measured during kinesthetic demonstrations, to compute force-informed actions for policy learning. We collect demonstrations for six tasks and show that policies trained on our force-informed actions achieve an average success rate of 76% across all tasks. In contrast, policies trained directly on actions that do not account for contact forces have near-zero success rates. We also conduct a study ablating the inclusion of force data in policy observations. We find that while using force data never hurts policy performance, it helps most for tasks that require advanced levels of precision and coordination, like opening an AirPods case and unscrewing a nut.

Soft sensors that can perceive multiaxial forces, such as normal and shear, are of interest for dexterous robotic manipulation and monitoring of human performance. Typical planar fabrication techniques have substantial design constraints that often prohibit the creation of functionally compelling and complex architectures. Moreover, they often require multiple-step operations for production. Here, we use an additive manufacturing process based on continuous liquid interface production to create high-resolution (30-micrometer) three-dimensional elastomeric polyurethane lattices for use as dielectric layers in capacitive sensors. We show that the capacitive responses and sensitivities are highly tunable through designs of lattice type, thickness, and material-void volume percentage. Microcomputed tomography and finite element simulation are used to elucidate the influence of lattice design on the deformation mechanism and concomitant sensing behavior. The advantage of three-dimensional printing is exhibited with examples of fully printed representative athletic equipment with integrated sensors.

Wearable haptic devices on the forearm can relay information from virtual agents, robots, and other humans while leaving the hands free. We introduce and test a new wearable haptic device that uses soft actuators to provide normal and shear force to the skin of the forearm. A rigid housing and gear motor are used to control the direction of the shear force. A 6-axis force/torque sensor, distance sensor, and pressure sensors are integrated to quantify how the soft tactor interacts with the skin. When worn by participants, the device delivered consistent shear forces of up to 0.64 N and normal forces of up to 0.56 N over distances as large as 14.3 mm. To understand cue saliency, we conducted a user study asking participants to identify linear shear directional cues in a 4-direction task and an 8-direction task with different cue speeds, travel distances, and contact patterns. Participants identified cues with longer travel distances best, with an 85.1% accuracy in the 4-direction task, and a 43.5% accuracy in the 8-direction task. Participants had a directional bias, with a preferential response in the axis towards and away from the wrist bone.

We present an integrated pneumatic sensor and actuator for soft haptic devices that can estimate contact force, location, and shape using a multi-headed neural network. The sensor uses channels cast into a multi-layer silicone bubble that change flow resistance as the bubble deforms, allowing measurements of strain and external contact at the surface. Changes in flow resistance are measured using pressure transducers in a pneumatic circuit. Arranging the channels in a two-layer array allows localization of the strain with an accuracy of over 80% for contact locations in a 3×3 grid and force measurements from 0.9–4.5 N with an RMS error of $\approx$ 0.39 N. This technology can enable closed-loop control for future soft pneumatic devices.

When humans socially interact with another agent (e.g., human, pet, or robot) through touch, they do so by applying varying amounts of force with different directions, locations, contact areas, and durations. While previous work on touch gesture recognition has focused on the spatio-temporal distribution of normal forces, we hypothesize that the addition of shear forces will permit more reliable classification. We present a soft, flexible skin with an array of tri-axial tactile sensors for the arm of a person or robot. We use it to collect data on 13 touch gesture classes through user studies and train a Convolutional Neural Network (CNN) to learn spatio-temporal features from the recorded data. The network achieved a recognition accuracy of 74% with normal and shear data, compared to 66% using only normal force data. Adding distributed shear data improved classification accuracy for 11 out of 13 touch gesture classes.

Our aim is to provide effective interaction with virtual objects, despite the lack of co-location of virtual and real-world contacts, while taking advantage of relatively large skin area and ease of mounting on the forearm. We performed two human participant studies to determine the effects of haptic feedback in the normal and shear directions during virtual manipulation using haptic devices worn near the wrist. In the first study, participants performed significantly better while discriminating stiffness values of virtual objects when the feedback consisted of normal displacements compared to shear displacements. Participants also commented that they could detect normal cues much easier than shear, which motivated us to perform a second study to find the point of subjective equality (PSE) between normal and shear stimuli. Our results show that shear stimuli require a larger actuator displacement but less force than normal stimuli to achieve perceptual equality for our haptic bracelets. We found that normal and shear stimuli cannot be equalized through skin displacement nor the interaction forces across all users. Rather, a calibration method is needed to find the point of equality for each user where normal and shear stimuli create the same intensity on the user's skin.

We present a new gripper and exploration approach that uses a finger with very low reflected inertia for probing and then grasping objects. The finger employs a transparent transmission, resulting in a light touch when contact occurs. The finger elements are stiff and mounted on precise Cartesian axes for accurate proprioceptive sensing. Experiments show that the finger can safely move faster into contacts than industrial parallel jaw grippers or even most force-controlled grippers with backdrivable transmissions. This property allows rapid proprioceptive probing of objects. Contact information is leveraged to execute grasping actions with a contact-first strategy and to reduce environment state uncertainty. We evaluate a particle filtering algorithm that inputs contact information from either proprioception, or a combination of tactile sensing and proprioception, to estimate object location. Both methods can estimate location within 2 mm; combined tactile sensing and proprioception requires fewer observations.

We present a new tactile sensor intended for manipulation by mobile robots, for example in the home. The surface consists of an array of small, rounded bumps or "nibs", which provide reliable traction on objects like wet dishes. When the nibs contact a surface they deflect, and capacitive sensors measure the corresponding local normal and shear forces. A key feature of the sensor is the ability to reconfigure dynamically depending on which combinations of sensing elements it samples. By interrogating different combinations of elements the sensor can detect and distinguish between linear and rotational sliding, and other dynamic events such as making and breaking contact. These dynamic events, combined with sensing the grasp and load forces, are useful for acquiring objects and performing simple in-hand manipulations. The proposed slip detection method estimates minimum required grasping force with an error less than 1.5N and uses tactile controlled rotational slips to reorient an unknown weight/surface object with 78% success rate.