Close-Fitting Dressing Assistance Based on State Estimation of Feet and Garments With Semantic-Based Visual Attention
- 13 February 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (4) , 3923-3930
Abstract
As the population continues to age, a shortage of caregivers is expected in the future. Dressing assistance, in particular, is crucial for opportunities for social participation. Especially dressing close-fitting garments, such as socks, remains challenging due to the need for fine force adjustments to handle the friction or snagging against the skin, while considering the shape and position of the garment. This study introduces a method that uses multi-modal information including not only the robot’s camera images, joint angles, joint torques, but also tactile forces for proper force interaction that can adapt to individual differences in humans. Furthermore, by introducing semantic information based on object concepts, rather than relying solely on RGB data, it can be generalized to unseen feet and background. In addition, incorporating depth data helps infer relative spatial relationship between the sock and the foot. To validate its capability for semantic object conceptualization and to ensure safety, training data were collected using a mannequin, and subsequent experiments were conducted with human subjects. In experiments, the robot successfully adapted to previously unseen human feet and was able to put socks on 10 participants, achieving a higher success rate than Action Chunking with Transformer and Diffusion Policy. These results demonstrate that the proposed model can estimate the state of both the garment and the foot, enabling precise dressing assistance for close-fitting garments.Keywords
All Related Versions
Funding Information
- JST Moonshot (JPMJMS2031)
This publication has 30 references indexed in Scilit:
- Model Predictive Control with Graph Dynamics for Garment Opening Insertion during Robot-Assisted Dressing2024
- Skeleton recognition-based motion generation and user emotion evaluation with in-home rehabilitation assistive humanoid robot2022
- Reconfigurability of the origami-inspired integrated 8R kinematotropic metamorphic mechanism and its evolved 6R and 4R mechanismsMechanism and Machine Theory, 2021
- Learning Grasping Points for Garment Manipulation in Robot-Assisted Dressing2020
- A framework for robotic clothing assistance by imitation learningAdvanced Robotics, 2019
- Personalized collaborative plans for robot-assisted dressing via optimization and simulationAutonomous Robots, 2019
- Selective Kernel Networks2019
- Imitation Learning for Object Manipulation Based on Position/Force Information Using Bilateral Control2018
- Adapting robot task planning to user preferences: an assistive shoe dressing exampleAutonomous Robots, 2018
- Interactions between dorsal and ventral streams for controlling skilled graspNeuropsychologia, 2015