GBOT: Graph-Based 3D Object Tracking for Augmented Reality-Assisted Assembly Guidance
Authors:
- 16 March 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 513-523
Abstract
Guidance for assemblable parts is a promising field for augmented reality. Augmented reality assembly guidance requires 6D object poses of target objects in real time. Especially in time-critical medical or industrial settings, continuous and markerless tracking of individual parts is essential to visualize instructions superimposed on or next to the target object parts. In this regard, occlusions by the user’s hand or other objects and the complexity of different assembly states complicate robust and real-time markerless multi-object tracking. To address this problem, we present Graph-based Object Tracking (GBOT), a novel graph-based single-view RGB-D tracking approach. The real-time markerless multi-object tracking is initialized via 6D pose estimation and updates the graph-based assembly poses. The tracking through various assembly states is achieved by our novel multi-state assembly graph. We update the multi-state assembly graph by utilizing the relative poses of the individual assembly parts. Linking the individual objects in this graph enables more robust object tracking during the assembly process. For evaluation, we introduce a synthetic dataset of publicly available and 3D printable assembly assets as a benchmark for future work. Quantitative experiments in synthetic data and further qualitative study in real test data show that GBOT can outperform existing work towards enabling context-aware augmented reality assembly guidance. Dataset and code will be made publically available. ** ** https://github.com/roth-hex-lab/gbotKeywords
All Related Versions
Funding Information
- Ministry of Education
This publication has 57 references indexed in Scilit:
- YOLOPose: Transformer-Based Multi-object 6D Pose Estimation Using Keypoint Regression2023
- SRT3D: A Sparse Region-Based 3D Object Tracking Approach for the Real WorldInternational Journal of Computer Vision, 2022
- Augmented reality instructions for construction toys enabled by accurate model registration and realistic object/hand occlusionsVirtual Reality, 2021
- DeepIM: Deep Iterative Matching for 6D Pose Estimation2018
- HoloLens for Assembly Assistance - A Focus Group Report2017
- UnrealCV: Connecting Computer Vision to Unreal Engine2016
- DART: dense articulated real-time tracking with consumer depth camerasAutonomous Robots, 2015
- Learning 6D Object Pose Estimation Using 3D Object Coordinates2014
- Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes2013
- PWP3D: Real-Time Segmentation and Tracking of 3D ObjectsInternational Journal of Computer Vision, 2012