VNVan-Thiep Nguyen

Van-Thiep Nguyen

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Van-Thiep Nguyen has a total of 36 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Depth-Guided Monocular Object Pose Estimation for Warehouse Automation

    by Phan Xuan Tan, Dinh-Cuong Hoang, Anh-Nhat Nguyen, Eiji Kamioka, Ta Huu Anh Duong, Tuan-Minh Huynh, Duc-Manh Nguyen, Duc-Huy Ngo, Minh-Duc Cao, Thu-Uyen Nguyen, et al.

    Accurate object pose estimation is essential for warehouse automation, enabling tasks such as robotic picking, sorting, and inventory management. Current state-of-the-art approaches rely on both color (RGB) and depth (D) images, as depth information provides critical geometric cues that enhance object localization and improve robustness against occlusions. However, RGBD-based methods require specialized depth sensors, which can be costly and may not function reliably in warehouse environments with reflective surfaces, varying lighting conditions, or sensor occlusions. To address these limitations, researchers have explored RGB-only approaches, but the absence of depth cues makes it challenging to handle occlusions, estimate object geometry, and differentiate between textureless or highly similar objects, which are common in warehouses. In this paper, we propose a novel end-to-end depth-guided object pose estimation method tailored for warehouse automation. Our approach leverages both depth and color images during training but relies solely on RGB images during inference. Depth images are used to supervise the training of a depth estimation network, which generates initial depth-aware features. These features are then refined using our proposed depth-guided feature enhancement module to improve spatial understanding and robustness. The enhanced features are subsequently utilized for keypoint-based 6D object pose estimation. By integrating depth-guided feature learning, our method significantly enhances pose estimation accuracy, especially in cluttered warehouse environments with severe occlusions and textureless objects. Extensive experiments on warehouse-specific datasets, as well as standard benchmark datasets, demonstrate that our approach outperforms existing RGB-based methods while maintaining real-time inference speeds, making it a highly practical solution for real-world warehouse automation applications.


  • Self-Supervised Object Pose Estimation With Multitask Learning

    by Dinh-Cuong Hoang, Phan Xuan Tan, Ta Huu Anh Duong, Tuan-Minh Huynh, Duc-Manh Nguyen, Anh-Nhat Nguyen, Duc-Long Pham, Van-Duc Vu, Thu-Uyen Nguyen, Ngoc-Anh Hoang, et al.

    Object pose estimation using learning-based methods often necessitates vast amounts of meticulously labeled training data. The process of capturing real-world object images under diverse conditions and annotating these images with 6 Degrees of Freedom (6DOF) object poses is both time-consuming and resource-intensive. In this study, we propose an innovative approach to monocular 6D pose estimation through self-supervised learning, eliminating the need for labor-intensive manual annotations. Our method initiates by training a multi-task neural network in a fully supervised manner, leveraging synthetic RGBD data. We leverage semantic segmentation, instance-level depth estimation, and vector-field prediction as auxiliary tasks to enhance the primary task of pose estimation. Subsequently, we harness advancements in multi-task learning to further self-supervise the model using unlabeled real-world RGB data. A pivotal element of our self-supervised object pose estimation is a geometry-guided pseudo-label filtering module that relies on estimated depth from instance-level depth estimation. Our extensive experiments conducted on benchmark datasets demonstrate the effectiveness and potential of our approach in achieving accurate monocular 6D pose estimation. Importantly, our method showcases a promising avenue for overcoming the challenges associated with the labor-intensive annotation process, offering a more efficient and scalable solution for real-world object pose estimation.


  • Open Access

    Efficient Multimodal Fusion for Hand Pose Estimation With Hourglass Network

    by Dinh-Cuong Hoang, Phan Xuan Tan, Duc-Long Pham, Hai-Nam Pham, Son-Anh Bui, Chi-Minh Nguyen, An-Binh Phi, Khanh-Duong Tran, Viet-Anh Trinh, Van-Duc Tran, et al.

    Hand pose estimation is vital for various applications, including virtual reality (VR), augmented reality (AR), gesture recognition, human-computer interaction (HCI), and robotics. Achieving accurate and real-time hand pose estimation is challenging due to factors such as the high degree of articulation in the human hand and the variability in hand shapes and sizes. While multimodal data offers advantages, developing a fast and resource-efficient hand pose estimation system remains challenging. Current state-of-the-art methods often require powerful graphics processing units (GPUs) for high performance, limiting deployment on edge platforms with limited computational resources. There is a critical need for higher efficiency without compromising accuracy, especially in real-world applications like mobile devices and embedded systems. Additionally, real-time performance is essential for practical applications, where systems must respond immediately to user interactions. Unfortunately, most current methods struggle to achieve real-time speeds, even on powerful GPUs, let alone on resource-constrained devices. To address these challenges, we propose an efficient hand pose estimation system that leverages both red-green-blue (RGB) and depth (RGBD) data through a unified fusion strategy. Our method combines appearance and geometric data early in the processing pipeline, significantly reducing computational complexity while maintaining real-time performance on resource-constrained devices. Experimental results show that the proposed model runs at over 110 fps on GPU, and 30 fps on the edge platform of NVidia Jetson NX Xavier, which is 4 to 5 times faster than existing methods, while achieving competitive accuracy.


  • Open Access

    A Unified Framework for Depth-Assisted Monocular Object Pose Estimation

    by Dinh-Cuong Hoang, Phan Xuan Tan, Thu-Uyen Nguyen, Hai-Nam Pham, Chi-Minh Nguyen, Son-Anh Bui, Quang-Tri Duong, Van-Duc Vu, Van-Thiep Nguyen, Van-Hiep Duong, et al.

    Monocular Depth Estimation (MDE) and Object Pose Estimation (OPE) are important tasks in visual scene understanding. Traditionally, these challenges have been addressed independently, with separate deep neural networks designed for each task. However, we contend that MDE, which provides information about object distances from the camera, and OPE, which focuses on determining precise object position and orientation, are inherently connected. Combining these tasks in a unified approach facilitates the integration of spatial context, offering a more comprehensive understanding of object distribution in three-dimensional space. Consequently, this work addresses both challenges simultaneously, treating them as a multi-task learning problem. Our proposed solution is encapsulated in a Unified Framework for Depth-Assisted Monocular Object Pose Estimation. Leveraging Red-Green-Blue (RGB) images as input, our framework estimates pose of multiple object instances alongside an instance-level depth map. During training, we utilize both depth and color images, but during inference, the model relies exclusively on color images. To enhance the depth-aware features crucial for robust object pose estimation, we introduce a depth estimation branch supervised by depth images. These features undergo further refinement through a cross-task attention module, contributing to the innovation of our method in significantly improving feature discriminability and robustness in object pose estimation. Through extensive experiments, our approach demonstrates competitive performance compared to state-of-the-art methods in object pose estimation. Moreover, our method operates in real-time, underscoring its efficiency and practical applicability in various scenarios. This unified framework not only advances the state of the art in monocular depth estimation and object pose estimation but also underscores the potential of multi-task learning for enhancing the understanding of complex visual scenes.

  • Collision-Free Grasp Detection From Color and Depth Images

    by Dinh-Cuong Hoang, Anh-Nhat Nguyen, Chi-Minh Nguyen, An-Binh Phi, Quang-Tri Duong, Khanh-Duong Tran, Viet-Anh Trinh, Van-Duc Tran, Hai-Nam Pham, Phuc-Quan Ngo, et al.

    Efficient and reliable grasp pose generation plays a crucial role in robotic manipulation tasks. The advancement of deep learning techniques applied to point cloud data has led to rapid progress in grasp detection. However, point cloud data has limitations: no appearance information and susceptibility to sensor noise. In contrast, color (RGB) images offer high-resolution and intricate textural details, making them a valuable complement to the three-dimensional geometry offered by point clouds or depth (D) images. Nevertheless, the effective integration of appearance information to enhance point cloud-based grasp detection remains an open question. In this study, we extend the concepts of VoteGrasp [1] and introduce an innovative deep learning approach referred to as VoteGrasp (RGBD). To build robustness to occlusion, the proposed model generates candidates by casting votes and accumulating evidence for feasible grasp configurations. This methodology revolves around fusing votes extracted from images and point clouds. To further enhance the collaborative effect of merging appearance and geometry features, we introduce a context learning module. We exploit contextual information by encoding the dependency of objects in the scene into features to boost the performance of grasp generation. The contextual information enables our model to increase the likelihood that the generated grasps are collision-free. The efficacy of our model is verified through comprehensive evaluations on the demanding GraspNet-1Billion dataset, leading to a significant improvement of 9.3 in Average Precision (AP) over the existing state-of-the-art results. Additionally, we provide extensive analyses through ablation studies to elucidate the contributions of each design decision.


  • Open Access

    Object Pose Estimation Using Color Images and Predicted Depth Maps

    by Dinh-Cuong Hoang, Phan Xuan Tan, Anh-Nhat Nguyen, Duy-Quang Vu, Van-Duc Vu, Thu-Uyen Nguyen, Quang-Tri Duong, Van-Thiep Nguyen, Ngoc-Anh Hoang, Khanh-Toan Phan, et al.

    The task of object pose estimation in computer vision heavily relies on both color (RGB) and depth (D) images to provide crucial appearance and geometric information, assisting algorithms in understanding occlusions and object geometry, thereby enhancing accuracy. However, the dependency on specialized sensors capable of capturing depth poses challenges in terms of cost and availability. Consequently, researchers are exploring methods to estimate object poses solely from RGB images. Nevertheless, this approach encounters difficulties in handling occlusions, discerning object geometry, and resolving ambiguities arising from similar color or texture patterns. This paper introduces a novel geometry-aware method for object pose estimation utilizing RGB images as input to determine the poses of multiple object instances. Our approach leverages both depth and color images during training but only relies on color images during inference. Departing from traditional depth sensors, our method computes predicted point clouds directly from estimated depth images derived from RGB inputs. A key innovation lies in the formulation of a multi-scale fusion module adept at seamlessly integrating features extracted from RGB images with those inferred from the predicted point clouds. This fusion process significantly fortifies the pose estimation pipeline by harnessing the strengths of both modalities, resulting in notably improved object poses. Extensive experimentation demonstrates that our approach markedly outperforms state-of-the-art RGB-based methods on Occluded-LINEMOD and YCB-Video datasets. Moreover, our method achieves competitive results compared to RGB-D approaches that necessitate both RGB and depth data from physical sensors.


  • Open Access

    Attention-Based Grasp Detection With Monocular Depth Estimation

    by Phan Xuan Tan, Dinh-Cuong Hoang, Anh-Nhat Nguyen, Van-Thiep Nguyen, Van-Duc Vu, Thu-Uyen Nguyen, Ngoc-Anh Hoang, Khanh-Toan Phan, Duc-Thanh Tran, Duy-Quang Vu, et al.

    Grasp detection plays a pivotal role in robotic manipulation, allowing robots to interact with and manipulate objects in their surroundings. Traditionally, this has relied on three-dimensional (3D) point cloud data acquired from specialized depth cameras. However, the limited availability of such sensors in real-world scenarios poses a significant challenge. In many practical applications, robots operate in diverse environments where obtaining high-quality 3D point cloud data may be impractical or impossible. This paper introduces an innovative approach to grasp generation using color images, thereby eliminating the need for dedicated depth sensors. Our method capitalizes on advanced deep learning techniques for depth estimation directly from color images. Instead of relying on conventional depth sensors, our approach computes predicted point clouds based on estimated depth images derived directly from Red-Green-Blue (RGB) input data. To our knowledge, this is the first study to explore the use of predicted depth data for grasp detection, moving away from the traditional dependence on depth sensors. The novelty of this work is the development of a fusion module that seamlessly integrates features extracted from RGB images with those inferred from the predicted point clouds. Additionally, we adapt a voting mechanism from our previous work (VoteGrasp) to enhance robustness to occlusion and generate collision-free grasps. Experimental evaluations conducted on standard datasets validate the effectiveness of our approach, demonstrating its superior performance in generating grasp configurations compared to existing methods. With our proposed method, we achieved a significant 4% improvement in average precision compared to state-of-the-art grasp detection methods. Furthermore, our method demonstrates promising practical viability through real robot grasping experiments, achieving an impressive 84% success rate.


  • Open Access

    Multi-Modal Hand-Object Pose Estimation With Adaptive Fusion and Interaction Learning

    by Dinh-Cuong Hoang, Phan Xuan Tan, Anh-Nhat Nguyen, Duy-Quang Vu, Van-Duc Vu, Thu-Uyen Nguyen, Ngoc-Anh Hoang, Khanh-Toan Phan, Duc-Thanh Tran, Van-Thiep Nguyen, et al.

    Hand-object configuration recovery is an important task in computer vision. The estimation of pose and shape for both hands and objects during interactive scenarios has various applications, particularly in augmented reality, virtual reality, or imitation-based robot learning. The problem is particularly challenging when the hand is interacting with objects in the environment, as this setting features both extreme occlusions and non-trivial shape deformations. While existing works treat the problem of estimating hand configurations (that is pose and shape parameters) in isolation from the recovery of parameters related to the object acted upon, we stipulate that the two problems are related and can be solved more accurately concurrently. We introduce an approach that jointly learns the features of hand and object from color and depth (RGB-D) images. Our approach fuses appearance and geometric features in an adaptive manner which allows us to accent or suppress features that are more meaningful for the upstream task of hand-object configuration recovery. We combine a deep Hough voting strategy that builds on our adaptive features with a graph convolutional network (GCN) to learn the interaction relationships between the hand and held object shapes during interaction. Experimental results demonstrate that our proposed approach consistently outperforms state-of-the-art methods on popular datasets.


  • Open Access

    Grasp Generation with Depth Estimation from Color Images

    by Nguyen Van Thiep, Vu Van Duc, Hoang Ngoc Anh, Nguyen Thu Uyen, Vu Duy Quang, Tran Duc Thanh, Phan Khanh Toan, Mai Truong Anh, Duong Van Hiep, Tran Cong Trinh, et al.

    Grasp generation plays a fundamental role in robot manipulation, often relying on three-dimensional (3D) point cloud data acquired through specialized depth cameras. However, the limited availability of such sensors in practical scenarios emphasizes the necessity for alternative approaches. This paper introduces an innovative method for grasp generation directly from color (RGB) images, negating the reliance on dedicated depth sensors. The proposed method employs tailored deep learning techniques for depth estimation from color images. Instead of traditional depth sensors, our approach computes predicted point clouds from estimated depth images directly generated from RGB inputs. A significant contribution lies in the design of a fusion module adept at seamlessly integrating features extracted from RGB images with those inferred from the predicted point clouds. This fusion process significantly strengthens the grasp generation pipeline by strengthening the advantages of both modalities, yielding notably improved grasp configurations. Experimental evaluations on standard datasets validate the efficacy of our approach, demonstrating its superior performance in generating grasp configurations compared to existing methods.

  • Open Access

    Multi Task-Guided 6D Object Pose Estimation

    by Thu-Uyen Nguyen, Van-Duc Vu, Van-Thiep Nguyen, Ngoc-Anh Hoang, Duy-Quang Vu, Duc-Thanh Tran, Khanh-Toan Phan, Anh-Truong Mai, Van-Hiep Duong, Cong-Trinh Tran, et al.

    Object pose estimation remains a fundamental challenge in computer vision, with cutting-edge methods relying on both RGB and depth data. Depth information is pivotal, offering crucial geometric cues that enable algorithms to navigate occlusions, fostering a more comprehensive scene under- standing and precise pose estimation. However, RGBD-based methods often require specialized depth sensors, which can be costlier and less accessible compared to standard RGB cameras. Consequently, research has explored techniques aiming to estimate object pose solely from color images. Yet, the absence of depth cues poses challenges in handling occlusions, comprehending object geometry, and resolving ambiguities arising from similar colors or textures. This paper introduces a end-to-end multi-task-guided object pose estimation method, utilizing RGB images as input and producing the 6D pose of multiple object instances. While our approach employs both depth and color images during training, inference relies solely on color images. We incorporate depth images to supervise a depth estimation branch, generating depth-aware features further refined through a cross-task attention module. These enhanced features are pivotal for our object pose estimation. Our method's innovation lies in significantly enhancing feature discriminability and robustness for object pose estimation. Through extensive experiments, we demonstrate competitive performance compared to state-of-the-art methods in object pose estimation.