Deming Wang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Deming Wang has a total of 18 co-authors. Here are the top co-authors by number of shared publications:
- Qijun Chen
Tongji University
8 shared publications - Yi Yan
College of Electronics and Information Engineering, Tongji University, Shanghai, China
6 shared publications - Guangliang Zhou
College of Electronics and Information Engineering, Tongji University, Shanghai, China
5 shared publications - Chengju Liu
Tongji University
3 shared publications Bohuan Xue
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong, SAR, China
1 shared publication- Changxin Gao
Huazhong University of Science and Technology
1 shared publication
Latest Publications
- 29 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 22 (15455955) , 895-904
Three-Filters-to-Normal+: Revisiting Discontinuity Discrimination in Depth-to-Normal Translation
This article introduces three-filters-to-normal $+$ (3F2N $+$ ), an extension of our previous work three-filters-to-normal (3F2N), with a specific focus on incorporating discontinuity discrimination capability into surface normal estimators (SNEs). 3F2N $+$ achieves this capability by utilizing a novel discontinuity discrimination module (DDM), which combines depth curvature minimization and correlation coefficient maximization through conditional random fields (CRFs). To evaluate the robustness of SNEs on noisy data, we create a large-scale synthetic surface normal (SSN) dataset containing 20 scenarios (ten indoor scenarios and ten outdoor scenarios with and without random Gaussian noise added to depth images). Extensive experiments demonstrate that 3F2N $+$ achieves greater performance than all other geometry-based surface normal estimators, with average angular errors of 7.85 $^\circ$ , 8.95 $^\circ$ , 9.25 $^\circ$ , and 11.98 $^\circ$ on the clean-indoor, clean-outdoor, noisy-indoor, and noisy-outdoor datasets, respectively. We conduct three additional experiments to demonstrate the effectiveness of incorporating our proposed 3F2N $+$ into downstream robot perception tasks, including freespace detection, 6D object pose estimation, and point cloud completion. Our source code and datasets are publicly available at https://mias.group/3F2Nplus. Note to Practitioners —The primary motivation behind this work arises from the need to develop a high-performing surface normal estimator for practical robotics and computer vision applications. While geometry-based surface normal estimators have been widely used in these domains, the existing solutions focus merely on discontinuity discrimination. To tackle this problem, this article introduces a plug-and-play module that leverages both depth curvature and correlation coefficient to quantify discontinuity levels, thereby optimizing surface normal estimation, particularly near or on discontinuous regions. Moreover, this article also introduces a large-scale public dataset with random noise added to depth images, providing a more realistic and robust platform for algorithm evaluation within this research community. Extensive experimental results demonstrate that our method outperforms other state-of-the-art algorithms.
- 29 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 22 (15455955) , 895-904
Three-Filters-to-Normal+: Revisiting Discontinuity Discrimination in Depth-to-Normal Translation
This article introduces three-filters-to-normal $+$ (3F2N $+$ ), an extension of our previous work three-filters-to-normal (3F2N), with a specific focus on incorporating discontinuity discrimination capability into surface normal estimators (SNEs). 3F2N $+$ achieves this capability by utilizing a novel discontinuity discrimination module (DDM), which combines depth curvature minimization and correlation coefficient maximization through conditional random fields (CRFs). To evaluate the robustness of SNEs on noisy data, we create a large-scale synthetic surface normal (SSN) dataset containing 20 scenarios (ten indoor scenarios and ten outdoor scenarios with and without random Gaussian noise added to depth images). Extensive experiments demonstrate that 3F2N $+$ achieves greater performance than all other geometry-based surface normal estimators, with average angular errors of 7.85 $^\circ$ , 8.95 $^\circ$ , 9.25 $^\circ$ , and 11.98 $^\circ$ on the clean-indoor, clean-outdoor, noisy-indoor, and noisy-outdoor datasets, respectively. We conduct three additional experiments to demonstrate the effectiveness of incorporating our proposed 3F2N $+$ into downstream robot perception tasks, including freespace detection, 6D object pose estimation, and point cloud completion. Our source code and datasets are publicly available at https://mias.group/3F2Nplus. Note to Practitioners —The primary motivation behind this work arises from the need to develop a high-performing surface normal estimator for practical robotics and computer vision applications. While geometry-based surface normal estimators have been widely used in these domains, the existing solutions focus merely on discontinuity discrimination. To tackle this problem, this article introduces a plug-and-play module that leverages both depth curvature and correlation coefficient to quantify discontinuity levels, thereby optimizing surface normal estimation, particularly near or on discontinuous regions. Moreover, this article also introduces a large-scale public dataset with random noise added to depth images, providing a more realistic and robust platform for algorithm evaluation within this research community. Extensive experimental results demonstrate that our method outperforms other state-of-the-art algorithms.
- 5 November 2023
- Published by Springer Nature
- p. 165-178
Portrait Matting via Semantic and Detail Guidance
- 5 November 2023
- Published by Springer Nature
- p. 165-178
Portrait Matting via Semantic and Detail Guidance
- 26 May 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 72 (00189456) , 1-12
Semi-Supervised Bolt Anomaly Detection Based on Local Feature Reconstruction
During the process of high-speed railway online maintenance, anomaly detection of targets in grease injection facility is extremely challenging. Environmental uncertainty (noise disturbance, various illumination conditions, changeable camera perspective, etc.) limits the performance of existing unsupervised anomaly detection methods, which tend to struggle in such complex backgrounds without prior knowledge of anomalies. Therefore, we propose a novel semi-supervised anomaly detection algorithm that only learns from anomaly-free samples and annotations while can recognize both anomaly-free and anomaly targets (key-areas) and detecting anomalous targets simultaneously. We firstly propose a key-area recognition module (KRM) to segment anomaly-free targets and predict coordinates of both anomaly-free and anomalous key-areas’ center point (key-points). By using the predicted coordinates, we generated a circular mask to erase features within the key-areas. Based on the erased features, a local feature reconstruction and segmentation module (LFRSM) is proposed to restore the latent embeddings within the key-areas and segment both anomaly-free and anomalous key-areas. The final anomaly results are obtained by analyzing the original segmentation results of KRM and the reconstructed segmentation results of LFRSM. To validate the effectiveness of the proposed algorithm, we develop a dataset, called BOLT-DET, for bolt anomaly detection in complex scenes. The proposed algorithm outperforms other existing state-of-the-art (SOTA) anomaly detection algorithms (SPADE, DFR, CFLOW, Student-Teacher, PatchCore, and DRAEM) with mIoU of 0.66, AUROC of 0.92 and F1-Measure of 0.69 on BOLT-DET. It can accurately detect anomalies and meet the requirement of practical industrial applications.
- 26 May 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 72 (00189456) , 1-12
Semi-Supervised Bolt Anomaly Detection Based on Local Feature Reconstruction
During the process of high-speed railway online maintenance, anomaly detection of targets in grease injection facility is extremely challenging. Environmental uncertainty (noise disturbance, various illumination conditions, changeable camera perspective, etc.) limits the performance of existing unsupervised anomaly detection methods, which tend to struggle in such complex backgrounds without prior knowledge of anomalies. Therefore, we propose a novel semi-supervised anomaly detection algorithm that only learns from anomaly-free samples and annotations while can recognize both anomaly-free and anomaly targets (key-areas) and detecting anomalous targets simultaneously. We firstly propose a key-area recognition module (KRM) to segment anomaly-free targets and predict coordinates of both anomaly-free and anomalous key-areas’ center point (key-points). By using the predicted coordinates, we generated a circular mask to erase features within the key-areas. Based on the erased features, a local feature reconstruction and segmentation module (LFRSM) is proposed to restore the latent embeddings within the key-areas and segment both anomaly-free and anomalous key-areas. The final anomaly results are obtained by analyzing the original segmentation results of KRM and the reconstructed segmentation results of LFRSM. To validate the effectiveness of the proposed algorithm, we develop a dataset, called BOLT-DET, for bolt anomaly detection in complex scenes. The proposed algorithm outperforms other existing state-of-the-art (SOTA) anomaly detection algorithms (SPADE, DFR, CFLOW, Student-Teacher, PatchCore, and DRAEM) with mIoU of 0.66, AUROC of 0.92 and F1-Measure of 0.69 on BOLT-DET. It can accurately detect anomalies and meet the requirement of practical industrial applications.
- 23 November 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 72 (00189456) , 1-11
6-D Object Pose Estimation Using Multiscale Point Cloud Transformer
Predicting 6D object pose is an essential task in vision measurement for robotic manipulation. RGB-based methods have a natural disadvantage due to the lack of 3D information, thus leading to inferior results. Therefore, exploiting the geometry information in depth images is crucial to achieve accurate predictions. To this end, we propose a multi-scale point cloud Transformer (MSPCT) to better learn point cloud feature representations. MSPCT mainly consists of three types of modules: Local Transformer (LT), DownSampling module and Global Transformer (GT). Specifically, the Local Transformer is designed to dynamically divide a local region centered on each point and further extract point-level feature with local context awareness. The DownSampling module is utilized to decrease the resolution and enlarge the receptive field. Global Transformer is employed to capture global-range dependencies between extracted local features. Based on the proposed Transformer blocks, we design a network architecture for object pose estimation, where we further obtain multi-scale features by fusing the local features from LT and global features from GT to predict object’s pose. Extensive experiments verify the effectiveness of Local Transformer and Global Transformer, and our pose estimation pipeline achieves promising results on three benchmark datasets.
- 23 November 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 72 (00189456) , 1-11
6-D Object Pose Estimation Using Multiscale Point Cloud Transformer
Predicting 6D object pose is an essential task in vision measurement for robotic manipulation. RGB-based methods have a natural disadvantage due to the lack of 3D information, thus leading to inferior results. Therefore, exploiting the geometry information in depth images is crucial to achieve accurate predictions. To this end, we propose a multi-scale point cloud Transformer (MSPCT) to better learn point cloud feature representations. MSPCT mainly consists of three types of modules: Local Transformer (LT), DownSampling module and Global Transformer (GT). Specifically, the Local Transformer is designed to dynamically divide a local region centered on each point and further extract point-level feature with local context awareness. The DownSampling module is utilized to decrease the resolution and enlarge the receptive field. Global Transformer is employed to capture global-range dependencies between extracted local features. Based on the proposed Transformer blocks, we design a network architecture for object pose estimation, where we further obtain multi-scale features by fusing the local features from LT and global features from GT to predict object’s pose. Extensive experiments verify the effectiveness of Local Transformer and Global Transformer, and our pose estimation pipeline achieves promising results on three benchmark datasets.
- 21 November 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 35 (6) , 7884-7898
NDNet: Spacewise Multiscale Representation Learning via Neighbor Decoupling for Real-Time Driving Scene Parsing
As a safety-critical application, autonomous driving requires high-quality semantic segmentation and real-time performance for deployment. Existing method commonly suffers from information loss and massive computational burden due to high-resolution input-output and multiscale learning scheme, which runs counter to the real-time requirements. In contrast to channelwise information modeling commonly adopted by modern networks, in this article, we propose a novel real-time driving scene parsing framework named NDNet from a novel perspective of spacewise neighbor decoupling (ND) and neighbor coupling (NC). We first define and implement the reversible operations called ND and NC, which realize lossless resolution conversion for complementary thumbnails sampling and collation to facilitate spatial modeling. Based on ND and NC, we further propose three modules, namely, local capturer and global dependence builder (LCGB), spacewise multiscale feature extractor (SMFE), and high-resolution semantic generator (HSG), which form the whole pipeline of NDNet. The LCGB serves as a stem block to preprocess the large-scale input for fast but lossless resolution reduction and extract initial features with global context. Then the SMFE is used for dense feature extraction and can obtain rich multiscale features in spatial dimension with less computational overhead. As for high-resolution semantic output, the HSG is designed for fast resolution reconstruction and adaptive semantic confusion amending. Experiments show the superiority of the proposed method. NDNet achieves the state-of-the-art performance on the Cityscapes dataset which reports 76.47% mIoU at 240 $+$ frames/s and 78.8% mIoU at 150 $+$ frames/s on the benchmark. Codes are available at https://github.com/LiShuTJ/NDNet.
- 21 November 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 35 (6) , 7884-7898
NDNet: Spacewise Multiscale Representation Learning via Neighbor Decoupling for Real-Time Driving Scene Parsing
As a safety-critical application, autonomous driving requires high-quality semantic segmentation and real-time performance for deployment. Existing method commonly suffers from information loss and massive computational burden due to high-resolution input-output and multiscale learning scheme, which runs counter to the real-time requirements. In contrast to channelwise information modeling commonly adopted by modern networks, in this article, we propose a novel real-time driving scene parsing framework named NDNet from a novel perspective of spacewise neighbor decoupling (ND) and neighbor coupling (NC). We first define and implement the reversible operations called ND and NC, which realize lossless resolution conversion for complementary thumbnails sampling and collation to facilitate spatial modeling. Based on ND and NC, we further propose three modules, namely, local capturer and global dependence builder (LCGB), spacewise multiscale feature extractor (SMFE), and high-resolution semantic generator (HSG), which form the whole pipeline of NDNet. The LCGB serves as a stem block to preprocess the large-scale input for fast but lossless resolution reduction and extract initial features with global context. Then the SMFE is used for dense feature extraction and can obtain rich multiscale features in spatial dimension with less computational overhead. As for high-resolution semantic output, the HSG is designed for fast resolution reconstruction and adaptive semantic confusion amending. Experiments show the superiority of the proposed method. NDNet achieves the state-of-the-art performance on the Cityscapes dataset which reports 76.47% mIoU at 240 $+$ frames/s and 78.8% mIoU at 150 $+$ frames/s on the benchmark. Codes are available at https://github.com/LiShuTJ/NDNet.
- 23 December 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 32 (8) , 5163-5174
Semi-Supervised 6D Object Pose Estimation Without Using Real Annotations
6D object pose estimation is a longstanding computer vision problem. Existing deep learning-based methods have achieved inspiring results in this task. However, large-scale training data with annotations is extremely needed to guarantee these methods’ performance, and acquiring real 6D object pose annotations is fairly labor-intensive and time-consuming. To overcome this drawback, we propose a semi-supervised pose estimation method using labeled synthetic data and unlabeled real data. For unlabeled real data, we form a self-supervised pipeline by minimizing the distance between the input point cloud, which is under ground-truth pose, and the model points transformed based on predicted pose. The labeled synthetic data is used to supervise the network to converge correctly. And we utilize a feature mapping to eliminate the domain gap between the real and synthetic features to further enhance the network’s performance. Moreover, we propose an attention-based pose estimation network, which can concentrate more on the distinguishing features, thus improving the accuracy of pose estimation. Experiments show that our proposed semi-supervised method is able to achieve good performance without the real annotations and outperforms all other methods relying on synthetic data or self-supervision strategy, indicating that the proposed method is effective.
- 23 December 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 32 (8) , 5163-5174
Semi-Supervised 6D Object Pose Estimation Without Using Real Annotations
6D object pose estimation is a longstanding computer vision problem. Existing deep learning-based methods have achieved inspiring results in this task. However, large-scale training data with annotations is extremely needed to guarantee these methods’ performance, and acquiring real 6D object pose annotations is fairly labor-intensive and time-consuming. To overcome this drawback, we propose a semi-supervised pose estimation method using labeled synthetic data and unlabeled real data. For unlabeled real data, we form a self-supervised pipeline by minimizing the distance between the input point cloud, which is under ground-truth pose, and the model points transformed based on predicted pose. The labeled synthetic data is used to supervise the network to converge correctly. And we utilize a feature mapping to eliminate the domain gap between the real and synthetic features to further enhance the network’s performance. Moreover, we propose an attention-based pose estimation network, which can concentrate more on the distinguishing features, thus improving the accuracy of pose estimation. Experiments show that our proposed semi-supervised method is able to achieve good performance without the real annotations and outperforms all other methods relying on synthetic data or self-supervision strategy, indicating that the proposed method is effective.
- 8 October 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 24 (15209210) , 4394-4408
GeoPose: Dense Reconstruction Guided 6D Object Pose Estimation With Geometric Consistency
6D object pose estimation for texture-less objects from RGB images remains challenging, especially in occlusion scenarios. Instead of localizing sparse keypoints by regressing their image coordinates or heatmaps, which are sensitive to occlusion, we introduce GeoPose, a novel reconstruction guided pose estimation pipeline that predicts dense correspondences and leverages geometric consistency effectively. We first design a dense reconstruction network (ReconNet) to reconstruct pixel-wise object coordinates in normalization space. Dense 2D-3D correspondences are generated intuitively by our explicit pa-rameterization for 3D object models, which dismisses keypoint selection efforts. These 2D-3D correspondences are then utilized to estimate 6D poses by the PnP algorithm with RANSAC iterations. Furthermore, a novel Cycle Loss is proposed to provide 3D prior supervision, which significantly correlates with the pose estimation task by guiding geometric consistency between reconstruction (pixel to 3D) and reprojection (3D to pixel). In addition, a training data augmentation method is proposed to handle the insufficiency of 6D datasets, the acquisition of which is error-prone and time-consuming. Extensive experiments demonstrate that, compared with existing RGB-based methods, our GeoPose can achieve state-of-the-art (SOTA) 6D pose estimation performance on the LINEMOD, Occlusion LINEMOD and T-LESS datasets.
- 8 October 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 24 (15209210) , 4394-4408
GeoPose: Dense Reconstruction Guided 6D Object Pose Estimation With Geometric Consistency
6D object pose estimation for texture-less objects from RGB images remains challenging, especially in occlusion scenarios. Instead of localizing sparse keypoints by regressing their image coordinates or heatmaps, which are sensitive to occlusion, we introduce GeoPose, a novel reconstruction guided pose estimation pipeline that predicts dense correspondences and leverages geometric consistency effectively. We first design a dense reconstruction network (ReconNet) to reconstruct pixel-wise object coordinates in normalization space. Dense 2D-3D correspondences are generated intuitively by our explicit pa-rameterization for 3D object models, which dismisses keypoint selection efforts. These 2D-3D correspondences are then utilized to estimate 6D poses by the PnP algorithm with RANSAC iterations. Furthermore, a novel Cycle Loss is proposed to provide 3D prior supervision, which significantly correlates with the pose estimation task by guiding geometric consistency between reconstruction (pixel to 3D) and reprojection (3D to pixel). In addition, a training data augmentation method is proposed to handle the insufficiency of 6D datasets, the acquisition of which is error-prone and time-consuming. Extensive experiments demonstrate that, compared with existing RGB-based methods, our GeoPose can achieve state-of-the-art (SOTA) 6D pose estimation performance on the LINEMOD, Occlusion LINEMOD and T-LESS datasets.
- 24 August 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 70 (00189456) , 1-12
Unsupervised Anomaly Segmentation Via Multilevel Image Reconstruction and Adaptive Attention-Level Transition
In product inspection scenarios, the objects have different characteristics of textures and different potential anomalies. Still, traditional methods cannot precisely segment structural anomalies and achieve optimal performance on all object categories. This article introduces a flexible multilevel image reconstruction (MLIR) framework for the unsupervised anomaly segmentation task. The MLIR integrates multiple generators with different downsampling ratios to serve as multiple reconstruction levels and leverage an adjustable perceptual similarity measurement to obtain the featurewise reconstruction errors, overcoming the challenge of detecting multiscale anomalies. Leveraging embeddings with different receptive fields for reconstruction, generators perform differently on the same target. Therefore, we depend on the reconstruction uncertainty to propose an adaptive attention-level transition (ALT) strategy that optimizes the detection performance automatically according to the target's category. ALT adjusts the weights of both reconstruction levels and feature measurement scales simultaneously to utilize features at a consistent level for reconstruction and anomaly detection (AD). This strategy remarkably improves the system's adaptivity for different applications. Moreover, we propose reducing the signal-to-noise ratio (SNR) of the images by adding pepper noise so that the reconstruction process is framed as an image denoising task, which improves feature extraction and image restoration robustness. We conduct extensive experiments on the MVTec AD dataset and demonstrate that the proposed method performs satisfactorily to different categories and achieves the state-of-the-art (SOTA) performance.
- 24 August 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 70 (00189456) , 1-12
Unsupervised Anomaly Segmentation Via Multilevel Image Reconstruction and Adaptive Attention-Level Transition
In product inspection scenarios, the objects have different characteristics of textures and different potential anomalies. Still, traditional methods cannot precisely segment structural anomalies and achieve optimal performance on all object categories. This article introduces a flexible multilevel image reconstruction (MLIR) framework for the unsupervised anomaly segmentation task. The MLIR integrates multiple generators with different downsampling ratios to serve as multiple reconstruction levels and leverage an adjustable perceptual similarity measurement to obtain the featurewise reconstruction errors, overcoming the challenge of detecting multiscale anomalies. Leveraging embeddings with different receptive fields for reconstruction, generators perform differently on the same target. Therefore, we depend on the reconstruction uncertainty to propose an adaptive attention-level transition (ALT) strategy that optimizes the detection performance automatically according to the target's category. ALT adjusts the weights of both reconstruction levels and feature measurement scales simultaneously to utilize features at a consistent level for reconstruction and anomaly detection (AD). This strategy remarkably improves the system's adaptivity for different applications. Moreover, we propose reducing the signal-to-noise ratio (SNR) of the images by adding pepper noise so that the reconstruction process is framed as an image denoising task, which improves feature extraction and image restoration robustness. We conduct extensive experiments on the MVTec AD dataset and demonstrate that the proposed method performs satisfactorily to different categories and achieves the state-of-the-art (SOTA) performance.
- 11 June 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 23 (15209210) , 1630-1639
A Novel Depth and Color Feature Fusion Framework for 6D Object Pose Estimation
This paper aims to solve the problem of estimating the 6D pose of an object under severe occlusion using RGB-D images. Most existing methods typically use the information of color and depth images separately to make predictions, which limits their performances in the presence of occlusion. Instead, we propose a pipeline to effectively fuse color and depth information and perform region-level pose estimation. Our method first uses a CNN to extract the color features, and then we obtain the fusion features by combining the color features into the point cloud. Unlike existing methods, the fusion features are in the form of point sets instead of feature maps. We further use a PointNet++-like network to process the fusion features, obtaining several region-level features. Each region-level feature can predict a pose with confidence. The pose with the highest confidence is chosen as the final output. Experiments show that the proposed method outperforms the-state-of-the-art methods on both the LINEMOD and Occlusion LINEMOD datasets, indicating that the proposed pipeline can obtain accurate pose estimation results and is robust to occlusion.
- 11 June 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 23 (15209210) , 1630-1639
A Novel Depth and Color Feature Fusion Framework for 6D Object Pose Estimation
This paper aims to solve the problem of estimating the 6D pose of an object under severe occlusion using RGB-D images. Most existing methods typically use the information of color and depth images separately to make predictions, which limits their performances in the presence of occlusion. Instead, we propose a pipeline to effectively fuse color and depth information and perform region-level pose estimation. Our method first uses a CNN to extract the color features, and then we obtain the fusion features by combining the color features into the point cloud. Unlike existing methods, the fusion features are in the form of point sets instead of feature maps. We further use a PointNet++-like network to process the fusion features, obtaining several region-level features. Each region-level feature can predict a pose with confidence. The pose with the highest confidence is chosen as the final output. Experiments show that the proposed method outperforms the-state-of-the-art methods on both the LINEMOD and Occlusion LINEMOD datasets, indicating that the proposed pipeline can obtain accurate pose estimation results and is robust to occlusion.