RC6D: An RFID and CV Fusion System for Real-time 6D Object Pose Estimation
Authors:
- 2 May 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 690-699
Abstract
This paper studies the problem of 6D pose estimation, which is practically important in various application scenarios such as robotic-based object grasping, obstacle avoidance in autonomous driving scene, and object integration in mixed reality. However, existing methods suffer from at least one of the five major limitations: dependence on object identification, complex deployment, difficulty in data collection, low accuracy, and incomplete estimation. To overcome the above limitations, this paper proposes an RC6D system, which is the first to estimate 6D poses by fusing RFID and Computer Vision (CV) data with multi-modal deep learning techniques. In RC6D, we first detect 2D keypoints through a deep learning approach. We then propose a novel RFID-CV fusion neural network to predict the depth of the scene, and use the estimated depth information to expand the 2D keypoints to 3D keypoints. Finally, we model the coordinate correspondences between the detected 2D-3D keypoints, which is applied to estimate the 6D pose of the target object. When implementing RC6D, we mainly address the following three technical challenges. (i) To predict 6D poses without using the CAD model, we propose a network architecture for monocular depth estimation. (ii) To train the neural network for 6D pose estimation without time-consuming 6D labeling, we use an unsupervised learning algorithm based on 2D-3D point pair matching. (iii) To detect the subject of the object without identification, we leverage optical flow to restrict the object and RFID to directly obtain its information. The experimental results show that the localization error of RC6D is less than 10 cm with a probability higher than 90.64% and its orientation estimation error is less than 10° with a probability higher than 79.63%. Hence, the proposed RC6D system performs much better than the state-of-the-art related solutions.Keywords
Funding Information
- Tencent
This publication has 25 references indexed in Scilit:
- A Survey of Augmented, Virtual, and Mixed Reality for Cultural HeritageJournal on Computing and Cultural Heritage, 2018
- SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again2017
- 3D Bounding Box Estimation Using Deep Learning and Geometry2017
- Fusing RFID and computer vision for fine-grained object tracking2017
- 6-DoF object pose from semantic keypoints2017
- Design and Implementation of an RFID-Based Customer Shopping Behavior Mining SystemIEEE/ACM Transactions on Networking, 2017
- Semantic Pose Using Deep Networks Trained on Synthetic RGB-D2015
- ImageNet: A large-scale hierarchical image database2009
- Complete solution classification for the perspective-three-point problemIEEE Transactions on Pattern Analysis and Machine Intelligence, 2003
- A flexible new technique for camera calibrationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2000