Deforming Garment Classification With Shallow Temporal Extraction and Tree-Based Fusion
- 12 December 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (2) , 1114-1121
Abstract
A novel RGB-based continuous perception garment classification approach is proposed in this letter, with the aim of identifying the correct category of the garment from a set of categories. It has been observed that treating a video of the continuous deformation of cloth as a set of disordered static figures leads to low classification precision. On the contrary, investigating the temporal information between frames can significantly improve the quality of extracted features and increase classification performance. In this regard, we propose a hybrid temporal fusion RGB-based algorithm, including an improved image-level shallow temporal feature extraction module (STEM) and a binary-tree fusion module (BiTF) for adaptive feature fusion. STEM incorporates multi-scale optical flow and long-short-term memorised information to capture both static features in every single image and dynamic features in consecutive images. BiTF constructs a tree-shaped structure to fuse an arbitrary number of extracted features in a video. The proposed algorithm achieves 97.5% accuracy on the public dataset, surpassing the state-of-the-art RGB-based classification algorithm by 44.0% and achieving comparable performance with RGBD-based algorithms.Keywords
Funding Information
- Zhejiang Provincial Natural Science Foundation of China (LDT23F01013F01)
- Fundamental Research Funds for the Central Universities
This publication has 42 references indexed in Scilit:
- Robotic Garment Folding: Precision Improvement and Workspace Enlargement2015
- Pose and category recognition of highly deformable objects using deep learning2015
- Regrasping and unfolding of garments using predictive thin shell modeling2015
- Accurate garment surface analysis using an active stereo robot head with application to dual-arm flattening2015
- Garment perception and its folding using a dual-arm robot2014
- Large-Scale Video Classification with Convolutional Neural Networks2014
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
- Folding paper with anthropomorphic robot hands using real-time physics-based modeling2012
- Bringing clothing into desired configurations with limited perception2011
- Determining optical flowArtificial Intelligence, 1981