Visual Localization for Robotic Bronchoscopy With Transferred Domain: A Fluoroscopy Validation

Authors:
Publisher website
Google Scholar
Abstract
Bronchoscopy is increasingly enhanced by the robotic technologies, which facilitate the examination of the airway and raise both diagnostic and therapeutic capabilities for pulmonary diseases. Accurate localization is essential for these robot-assisted endobronchial interventions. As an effective and low-cost solution, visual localization plays a pivotal role in advancing the automation of bronchoscopic robots. However, conventional visual localization approaches are constrained by the homogeneity and texture insufficiency within the bronchial lumen scene, leading to unreliable feature tracking and significant time-consuming image processing. This paper develops an innovative visual localization scheme for bronchoscopic robots, which can accurately estimate the position of the bronchoscope directly from captured airway lumen images. A novel domain transfer network, grounded in transformer technology, can significantly enhance the similarity between synthetic images generated from computed tomograph (CT) data and real video frames. Subsequently, by taking advantage of both personalized generative images and continuous frame sequences, a one-step direct pose estimation module is designed to recover position from endobronchial movements. The proposed method achieves an absolute trajectory error (ATE) of less than 3.85 mm on a human bronchial phantom, representing a more than 55% improvement over previous methods. To the best of our knowledge, this paper achieves the first generative-data-guaranteed visual localization for robotic bronchoscopy, which has a satisfactory accuracy validated by the fluoroscopy.
Funding Information
  • National Natural Science Foundation of China (62303248)
  • National Natural Science Foundation of China (U24A20284)
  • Basic and Applied Basic Research Foundation of Guangdong Province (2024A1515010102)
  • Basic and Applied Basic Research Foundation of Guangdong Province (2023A1515110678)
  • Natural Science Foundation of Tianjin (25JCZDJC01060)
  • State Key Laboratory of Robotics (2024-024)
  • Academy for Advanced Interdisciplinary Studies, Nankai University

This publication has 36 references indexed in Scilit: