A Unified Framework for Depth-Assisted Monocular Object Pose Estimation
Authors:
- Dinh-Cuong HoangDHDinh-Cuong Hoang
- Phan Xuan TanPTPhan Xuan Tan
- Thu-Uyen NguyenTNThu-Uyen Nguyen
- Hai-Nam PhamHPHai-Nam Pham
- Chi-Minh NguyenCNChi-Minh Nguyen
- Son-Anh BuiSBSon-Anh Bui
- Quang-Tri DuongQDQuang-Tri Duong
- Van-Duc VuVVVan-Duc Vu
- Van-Thiep NguyenVNVan-Thiep Nguyen
- Van-Hiep DuongVDVan-Hiep Duong
- Ngoc-Anh HoangNHNgoc-Anh Hoang
- Khanh-Toan PhanKPKhanh-Toan Phan
Open Access
- 13 August 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 111723-111740
Abstract
Monocular Depth Estimation (MDE) and Object Pose Estimation (OPE) are important tasks in visual scene understanding. Traditionally, these challenges have been addressed independently, with separate deep neural networks designed for each task. However, we contend that MDE, which provides information about object distances from the camera, and OPE, which focuses on determining precise object position and orientation, are inherently connected. Combining these tasks in a unified approach facilitates the integration of spatial context, offering a more comprehensive understanding of object distribution in three-dimensional space. Consequently, this work addresses both challenges simultaneously, treating them as a multi-task learning problem. Our proposed solution is encapsulated in a Unified Framework for Depth-Assisted Monocular Object Pose Estimation. Leveraging Red-Green-Blue (RGB) images as input, our framework estimates pose of multiple object instances alongside an instance-level depth map. During training, we utilize both depth and color images, but during inference, the model relies exclusively on color images. To enhance the depth-aware features crucial for robust object pose estimation, we introduce a depth estimation branch supervised by depth images. These features undergo further refinement through a cross-task attention module, contributing to the innovation of our method in significantly improving feature discriminability and robustness in object pose estimation. Through extensive experiments, our approach demonstrates competitive performance compared to state-of-the-art methods in object pose estimation. Moreover, our method operates in real-time, underscoring its efficiency and practical applicability in various scenarios. This unified framework not only advances the state of the art in monocular depth estimation and object pose estimation but also underscores the potential of multi-task learning for enhancing the understanding of complex visual scenes.Keywords
This publication has 73 references indexed in Scilit:
- Grasp Configuration Synthesis from 3D Point Clouds with Attention MechanismJournal of Intelligent & Robotic Systems, 2023
- Deep learning for 6D pose estimation of objects — A case study for autonomous drivingExpert Systems with Applications, 2023
- YOLOPose: Transformer-Based Multi-object 6D Pose Estimation Using Keypoint Regression2023
- A Brief Survey on Semantic Segmentation with Deep LearningNeurocomputing, 2020
- Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a reviewArtificial Intelligence Review, 2020
- Holistic and local patch framework for 6D object pose estimation in RGB-D imagesComputer Vision and Image Understanding, 2019
- CBAM: Convolutional Block Attention Module2018
- Implicit 3D Orientation Learning for 6D Object Detection from RGB Images2018
- Dynamic Task Prioritization for Multitask Learning2018
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015