Deep Learning and Conditional Random Fields-based Depth Estimation and Topographical Reconstruction from Conventional Endoscopy
Authors:
- Faisal MahmoodFMFaisal Mahmood
- Nicholas J. DurrNDNicholas J. Durr
Preprint
Abstract
Colorectal cancer is the fourth leading cause of cancer deaths worldwide and the second leading cause in the United States. The risk of colorectal cancer can be mitigated by the identification and removal of premalignant lesions through optical colonoscopy. Unfortunately, conventional colonoscopy misses more than 20% of the polyps that should be removed, due in part to poor contrast to lesion topography. Imaging tissue topography during a colonoscopy is difficult because of the size constraints of the endoscope and the deforming mucosa. Most existing methods make geometric assumptions or incorporate a priori information, which limits accuracy and sensitivity. In this paper, we present a method that avoids these restrictions, using a joint deep convolutional neural network-conditional random field (CNN-CRF) framework. Estimated depth is used to reconstruct the topography of the surface of the colon from a single image. We train the unary and pairwise potential functions of a CRF in a CNN on synthetic data, generated by developing an endoscope camera model and rendering over 100,000 images of an anatomically-realistic colon. We validate our approach with real endoscopy images of phantom and porcine colons, with ground truth from registered computed tomography measurements. The CNN-CRF approach estimates depths with a relative error of 0.152 for synthetic endoscopy images and 0.242 for real endoscopy images. We show that the estimated depth maps can be used for reconstructing the topography of the mucosa from conventional colonoscopy images. This approach can easily be integrated into existing endoscopy systems and provides a foundation for improving computer-aided detection algorithms for detection, segmentation and classification of lesions.Keywords
All Related Versions
This publication has 35 references indexed in Scilit:
- Deep monocular 3D reconstruction for assisted navigation in bronchoscopyInternational Journal of Computer Assisted Radiology and Surgery, 2017
- WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physiciansComputerized Medical Imaging and Graphics, 2015
- The History of Colorectal Cancer Screening: A Personal PerspectiveDigestive Diseases and Sciences, 2015
- 3D Reconstruction of virtual colon structures from colonoscopy imagesComputerized Medical Imaging and Graphics, 2014
- Toward embedded detection of polyps in WCE images for early diagnosis of colorectal cancerInternational Journal of Computer Assisted Radiology and Surgery, 2013
- Prevalence of Nonpolypoid (Flat and Depressed) Colorectal Neoplasms in Asymptomatic and Symptomatic AdultsJAMA, 2008
- 3-D Depth Reconstruction from a Single Still ImageInternational Journal of Computer Vision, 2007
- Monocular Vision for Mobile Robot Localization and Autonomous NavigationInternational Journal of Computer Vision, 2007
- Shape from Texture without BoundariesInternational Journal of Computer Vision, 2006
- Anatomical study of the length of the human intestineSurgical and Radiologic Anatomy, 2002