Direct Camera-Only Bundle Adjustment for 3-D Textured Colon Surface Reconstruction Based on Pre-Operative Model
- 13 December 2024
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Robotics and Bionics
- Vol. 7 (1) , 242-253
Abstract
This paper addresses the problem of reconstructing textured colon surface maps using a sequence of monocular colonoscopic images together with a 3D colon mesh model that has been segmented in CT colonography. The problem is formulated as a direct bundle adjustment (BA) problem which simultaneously optimizes all camera poses and the intensity of vertices on the pre-operative mesh model. This optimization is achieved by maximizing photometric consistency among multiple views of 2D images and the pre-operative 3D mesh model.The key properties of our proposed direct BA formulation involve eliminating the need for reference image specification, data association (feature extraction and matching), and image depth information. Thus, the proposed method is particularly suitable for scenarios where distinct features and image depth are not available, such as 2D colonoscopic images.Furthermore, we have proven that solving the proposed direct BA using the Gauss-Newton (GN) algorithm has the merit of optimizing camera poses only, which is equivalent to optimizing camera poses and the intensities of 3D vertices on the mesh together. Thus, a direct camera-only BA algorithm is proposed and used for 3D textured colon reconstruction from textureless 2D colonoscopic images.Validations using simulation, phantom, and in-vivo datasets are performed to demonstrate the accuracy and feasibility of the proposed algorithm.Keywords
Funding Information
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (203145/Z/16/Z)
- Department of Science, Innovation and Technology
- Royal Academy of Engineering under the Chair in Emerging Technologies Programme
- NIHR UCLH Biomedical Research Centre (NIHR203328)
- Australian Research Council (ARC) Discovery Project (DP200100982)
This publication has 37 references indexed in Scilit:
- EndoSurf: Neural Surface Reconstruction of Deformable Tissues with Stereo Endoscope Videos2023
- Neural Rendering for Stereo 3D Reconstruction of Deformable Tissues in Robotic Surgery2022
- Lighting Enhancement Aids Reconstruction of Colonoscopic Surfaces2021
- Deep Multi-view Stereo for Dense 3D Reconstruction from Monocular Endoscopic Video2020
- Real-Time 3D Reconstruction of Colonoscopic Surfaces for Determining Missing Regions2019
- OpenMVG: Open Multiple View Geometry2017
- Photometric Bundle Adjustment for Vision-Based SLAM2017
- Automated visibility map of the internal colon surface from colonoscopy videoInternational Journal of Computer Assisted Radiology and Surgery, 2016
- 3D Surface Reconstruction from Endoscopic Videos2008
- Bundle Adjustment — A Modern Synthesis2000