Computer-Aided Breast Surgery Framework Using a Markerless Augmented Reality Method
Open Access
- 11 December 2022
- journal article
- research article
- Published by MDPI AG in Diagnostics
- Vol. 12 (12) , 3123
Abstract
This study proposes a markerless Augmented Reality (AR) surgical framework for breast lesion removal using a depth sensor and 3D breast Computed Tomography (CT) images. A patient mesh in the real coordinate system is acquired through a patient 3D scan using a depth sensor for registration. The patient mesh on the virtual coordinate system is obtained by contrast-based skin segmentation in 3D mesh generated from breast CT scans. Then, the nipple area is detected based on the gradient in the segmented skin area. The region of interest (ROI) is set based on the detection result to select the vertices in the virtual coordinate system. The mesh on the real and virtual coordinate systems is first aligned by matching the center of mass, and the Iterative Closest Point (ICP) method is applied to perform more precise registration. Experimental results of 20 patients’ data showed 98.35 ± 0.71% skin segmentation accuracy in terms of Dice Similarity Coefficient (DSC) value, 2.79 ± 1.54 mm nipple detection error, and 4.69 ± 1.95 mm registration error. Experiments using phantom and patient data also confirmed high accuracy in AR visualization. The proposed method in this study showed that the 3D AR visualization of medical data on the patient’s body is possible by using a single depth sensor without having to use markers.Keywords
Funding Information
- the National Research Foundation of Korea, a government (Ministry of Science and ICT) (2020R1A2C1102727)
- the National Research Foundation of Korea, a government (Ministry of Education) (2020R1A6A3A01099507)
This publication has 17 references indexed in Scilit:
- Robust augmented reality guidance with fluorescent markers in laparoscopic surgeryInternational Journal of Computer Assisted Radiology and Surgery, 2016
- Efficient liver segmentation using a level-set method with optimal detection of the initial liver boundary from level-set speed imagesComputer Methods and Programs in Biomedicine, 2007
- Image-based Lighting2006
- Automated segmentations of skin, soft-tissue, and skeleton, from torso CT images2004
- Iterative point matching for registration of free-form curves and surfacesInternational Journal of Computer Vision, 1994
- Seeded region growingIEEE Transactions on Pattern Analysis and Machine Intelligence, 1994
- A method for registration of 3-D shapesIEEE Transactions on Pattern Analysis and Machine Intelligence, 1992
- Scale-space and edge detection using anisotropic diffusionIEEE Transactions on Pattern Analysis and Machine Intelligence, 1990
- Marching cubes: A high resolution 3D surface construction algorithmACM SIGGRAPH Computer Graphics, 1987
- A Threshold Selection Method from Gray-Level HistogramsIEEE Transactions on Systems, Man, and Cybernetics, 1979