Deep-learning-based endoscopic single-shot fringe projection profilometry
Open Access
- 1 August 2025
- journal article
- imaging
- Published by SPIE-Intl Soc Optical Eng in Journal of Biomedical Optics
- Vol. 30 (08) , 1-15
Abstract
SignificanceConventional fringe projection profilometry (FPP) requires multiple image acquisitions and therefore long acquisition times that make it slow for high-speed dynamic measurements. We propose and demonstrate a deep-learning-based single-shot FPP system utilizing a single endoscope for surgical guidance.AimWe aim to achieve real-time depth map generation of target tissues with high accuracy for robotic surgical guidance.ApproachWe proposed an endoscopic single-shot FPP system based on a deep learning network to generate real-time accurate tissue depth maps for surgical guidance. The system utilizes a dual-channel endoscope, where one channel projects fringe patterns from a projector and the other channel collects images using a camera. In addition, we developed a data synthesis method to generate a large number of diverse training datasets. The network consists of MaskNet, which segments the tissue from the background, and DepthNet, which predicts the depth map of the image. The results from both networks are combined to generate the final depth map.ResultsWe tested our algorithm using fringe patterns with different frequencies and found that the optimal frequency for single-shot FPP in our setup is 20 Hz. The algorithm has been tested on both synthetic and experimental data, achieving a maximum depth prediction error of ∼2 mm and a processing time of about 12.75 ms per frame.ConclusionA deep-learning-based single-shot FPP endoscopic system was shown to be highly effective in real-time depth map generation with millimeter-scale error. Implementing such a system has the potential to improve the reliability of image-guided robotic surgery.Keywords
Funding Information
- National Institutes of Health (1R56EB033807)
This publication has 19 references indexed in Scilit:
- Fringe projection profilometry by conducting deep learning from its digital twinOptics Express, 2020
- Single-Shot 3D Shape Reconstruction Using Structured Light and Deep Convolutional Neural NetworksSensors, 2020
- Dynamic 3-D measurement based on fringe-to-fringe transformation using deep learningOptics Express, 2020
- High-speed three-dimensional shape measurement based on shifting Gray-code lightOptics Express, 2019
- Demonstration of a laparoscopic structured-illumination three-dimensional imaging system for guiding reconstructive bowel anastomosisJournal of Biomedical Optics, 2018
- Semi-Autonomous Laparoscopic Robotic Electro-Surgery with a Novel 3D Endoscope2018
- Real-time, high-accuracy 3D imaging and shape measurementApplied Optics, 2014
- Coherent two-beam interference fringe projection for highspeed three-dimensional shape measurementsApplied Optics, 2013
- Three-dimensional shape measurement with an arbitrarily arranged fringe projection profilometry systemOptics Letters, 2007
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004