Delong Zhu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Delong Zhu has a total of 30 co-authors. Here are the top co-authors by number of shared publications:
- Max Qing-Hu Meng
Southern University of Science and Technology, Shenzhen, China
6 shared publications - Zhe Min
School of Control Science and Engineering, Shandong University, Jinan, China
5 shared publications - Yunzhou Zhang
College of Information Science and Engineering, Northeastern University, Shenyang, China
3 shared publications - Dermot Kerr
Ulster University
2 shared publications - Hongliang Ren
The Chinese University of Hong Kong
2 shared publications - Sonya Coleman
Intelligent Systems Research Centre, University of Ulster, Derry, UK
2 shared publications
Latest Publications
- 17 May 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Robotics
- Vol. 39 (4) , 2912-2932
An Object SLAM Framework for Association, Mapping, and High-Level Tasks
Object SLAM is considered increasingly significant for robot high-level perception and decision-making. Existing studies fall short in terms of data association, object representation, and semantic mapping and frequently rely on additional assumptions, limiting their performance. In this article, we present a comprehensive object SLAM framework that focuses on object-based perception and object-oriented robot tasks. First, we propose an ensemble data association approach for associating objects in complicated conditions by incorporating parametric and nonparametric statistic testing. In addition, we suggest an outlier-robust centroid and scale estimation algorithm for modeling objects based on the iForest and line alignment. Then a lightweight and object-oriented map is represented by estimated general object models. Taking into consideration the semantic invariance of objects, we convert the object map to a topological map to provide semantic descriptors to enable multimap matching. Finally, we suggest an object-driven active exploration strategy to achieve autonomous mapping in the grasping scenario. A range of public datasets and real-world results in mapping, augmented reality, scene matching, relocalization, and robotic manipulation have been used to evaluate the proposed object SLAM framework for its efficient performance.
- 17 May 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Robotics
- Vol. 39 (4) , 2912-2932
An Object SLAM Framework for Association, Mapping, and High-Level Tasks
Object SLAM is considered increasingly significant for robot high-level perception and decision-making. Existing studies fall short in terms of data association, object representation, and semantic mapping and frequently rely on additional assumptions, limiting their performance. In this article, we present a comprehensive object SLAM framework that focuses on object-based perception and object-oriented robot tasks. First, we propose an ensemble data association approach for associating objects in complicated conditions by incorporating parametric and nonparametric statistic testing. In addition, we suggest an outlier-robust centroid and scale estimation algorithm for modeling objects based on the iForest and line alignment. Then a lightweight and object-oriented map is represented by estimated general object models. Taking into consideration the semantic invariance of objects, we convert the object map to a topological map to provide semantic descriptors to enable multimap matching. Finally, we suggest an object-driven active exploration strategy to achieve autonomous mapping in the grasping scenario. A range of public datasets and real-world results in mapping, augmented reality, scene matching, relocalization, and robotic manipulation have been used to evaluate the proposed object SLAM framework for its efficient performance.
- 24 April 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 33 (11) , 6404-6417
ELWNet: An Extremely Lightweight Approach for Real-Time Salient Object Detection
Existing lightweight salient object detection (SOD) methods aim to solve the problem of high computational costs that is prevalent with heavyweight methods. However, compared with heavyweight methods, the detection accuracy of lightweight methods is greatly reduced while real-time performance is not significantly improved. Therefore, we aim to establish a trade off between computational cost and detection performance by improving the network efficiency. We propose a fast and extremely lightweight end-to-end wavelet neural network (ELWNet) for real-time salient object detection. ELWNet can achieve salient object detection and segmentation at approximately 70FPS (GPU), 19FPS (CPU) with 76K parameters and 0.38G FLOPs. We introduce wavelet transform theory into a neural network, proposing a wavelet transform module (WTM), a wavelet transform fusion module (WTFM), a novel feature residual mechanism, and construct an efficient architecture. The wavelet transform theory is integrated into the neural network to realize the interaction between the features in the frequency and the time domain. Meanwhile, ELWNet does not rely on a pre-trained model, which significantly reduces redundant features. We validate the performance of ELWNet using five well-known datasets, and demonstrate state-of-the-art performance compared with 24 other SOD models in terms of being lightweight, detection accuracy and real-time capabilities. Our method maintains high detection performance while reducing the number of model parameters by approximately 99% compared with heavyweight methods.
- 24 April 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 33 (11) , 6404-6417
ELWNet: An Extremely Lightweight Approach for Real-Time Salient Object Detection
Existing lightweight salient object detection (SOD) methods aim to solve the problem of high computational costs that is prevalent with heavyweight methods. However, compared with heavyweight methods, the detection accuracy of lightweight methods is greatly reduced while real-time performance is not significantly improved. Therefore, we aim to establish a trade off between computational cost and detection performance by improving the network efficiency. We propose a fast and extremely lightweight end-to-end wavelet neural network (ELWNet) for real-time salient object detection. ELWNet can achieve salient object detection and segmentation at approximately 70FPS (GPU), 19FPS (CPU) with 76K parameters and 0.38G FLOPs. We introduce wavelet transform theory into a neural network, proposing a wavelet transform module (WTM), a wavelet transform fusion module (WTFM), a novel feature residual mechanism, and construct an efficient architecture. The wavelet transform theory is integrated into the neural network to realize the interaction between the features in the frequency and the time domain. Meanwhile, ELWNet does not rely on a pre-trained model, which significantly reduces redundant features. We validate the performance of ELWNet using five well-known datasets, and demonstrate state-of-the-art performance compared with 24 other SOD models in terms of being lightweight, detection accuracy and real-time capabilities. Our method maintains high detection performance while reducing the number of model parameters by approximately 99% compared with heavyweight methods.
- 27 September 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2978-2984
PiPo-Net: A Semi-automatic and Polygon-based Annotation Method for Pathological Images
Metastatic involvement of lymph nodes is one of the most important prognostic variables for many cancers. Several deep learning based algorithms have been developed to segment metastatic regions in pathological images to help predict prognosis. However, the training of these methods requires a large amount of annotated data, and the labeling task is an extremely time-consuming process for human annotators. In order to reduce the annotation burden, we for the first time propose a semi-automatic annotation method (PiPo-Net) for the labeling of pathological images. The method is comprised of two subnetworks, a pixel-wise segmentation network (Pi-Net) and a polygon-based annotation network (Po-Net). The Pi-Net adopts an improved encoder-decoder architecture and can effectively aggregate multi-scale image features. The Po-Net is built on the Pi-Net and leverages a two-layer recurrent neural network to generate tight-bounded polygons for the metastatic regions. Corresponding to the proposed network architecture, a loss function called PiPo-loss is introduced to help optimize the whole network. The main advantage of our method is that it integrates human annotators into the prediction loop, allowing to iteratively refine the predictions according to the suggestions from human annotators. We evaluate our method on Camelyon16 database and achieve a Dice score of 91% in the initial annotation attempt. We also demonstrate the effectiveness of the human-network collaborative annotation, which achieves promising labeling results, verifying the advantages of our proposed method.
- 27 September 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2978-2984
PiPo-Net: A Semi-automatic and Polygon-based Annotation Method for Pathological Images
Metastatic involvement of lymph nodes is one of the most important prognostic variables for many cancers. Several deep learning based algorithms have been developed to segment metastatic regions in pathological images to help predict prognosis. However, the training of these methods requires a large amount of annotated data, and the labeling task is an extremely time-consuming process for human annotators. In order to reduce the annotation burden, we for the first time propose a semi-automatic annotation method (PiPo-Net) for the labeling of pathological images. The method is comprised of two subnetworks, a pixel-wise segmentation network (Pi-Net) and a polygon-based annotation network (Po-Net). The Pi-Net adopts an improved encoder-decoder architecture and can effectively aggregate multi-scale image features. The Po-Net is built on the Pi-Net and leverages a two-layer recurrent neural network to generate tight-bounded polygons for the metastatic regions. Corresponding to the proposed network architecture, a loss function called PiPo-loss is introduced to help optimize the whole network. The main advantage of our method is that it integrates human annotators into the prediction loop, allowing to iteratively refine the predictions according to the suggestions from human annotators. We evaluate our method on Camelyon16 database and achieve a Dice score of 91% in the initial annotation attempt. We also demonstrate the effectiveness of the human-network collaborative annotation, which achieves promising labeling results, verifying the advantages of our proposed method.
- 30 August 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 34 (3) , 1278-1290
Structure-Aware Feature Disentanglement With Knowledge Transfer for Appearance-Changing Place Recognition
Long-term visual place recognition (VPR) is challenging as the environment is subject to drastic appearance changes across different temporal resolutions, such as time of the day, month, and season. A wide variety of existing methods address the problem by means of feature disentangling or image style transfer but ignore the structural information that often remains stable even under environmental condition changes. To overcome this limitation, this article presents a novel structure-aware feature disentanglement network (SFDNet) based on knowledge transfer and adversarial learning. Explicitly, probabilistic knowledge transfer (PKT) is employed to transfer knowledge obtained from the Canny edge detector to the structure encoder. An appearance teacher module is then designed to ensure that the learning of appearance encoder does not only rely on metric learning. The generated content features with structural information are used to measure the similarity of images. We finally evaluate the proposed approach and compare it to state-of-the-art place recognition methods using six datasets with extreme environmental changes. Experimental results demonstrate the effectiveness and improvements achieved using the proposed framework. Source code and some trained models will be available at http://www.tianshu.org.cn.
- 30 August 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 34 (3) , 1278-1290
Structure-Aware Feature Disentanglement With Knowledge Transfer for Appearance-Changing Place Recognition
Long-term visual place recognition (VPR) is challenging as the environment is subject to drastic appearance changes across different temporal resolutions, such as time of the day, month, and season. A wide variety of existing methods address the problem by means of feature disentangling or image style transfer but ignore the structural information that often remains stable even under environmental condition changes. To overcome this limitation, this article presents a novel structure-aware feature disentanglement network (SFDNet) based on knowledge transfer and adversarial learning. Explicitly, probabilistic knowledge transfer (PKT) is employed to transfer knowledge obtained from the Canny edge detector to the structure encoder. An appearance teacher module is then designed to ensure that the learning of appearance encoder does not only rely on metric learning. The generated content features with structural information are used to measure the similarity of images. We finally evaluate the proposed approach and compare it to state-of-the-art place recognition methods using six datasets with extreme environmental changes. Experimental results demonstrate the effectiveness and improvements achieved using the proposed framework. Source code and some trained models will be available at http://www.tianshu.org.cn.
- 26 April 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Robotics and Bionics
- Vol. 3 (2) , 372-383
Aligning 3D Curve With Surface Using Tangent and Normal Vectors for Computer-Assisted Orthopedic Surgery
Registration that aligns different views of one interested organ together is an essential technique and outstanding problem in medical robotics and image-guided surgery (IGS). This work introduces a novel rigid point set registration (PSR) approach that aims to accurately map the pre-operative space with the intra-operative space to enable successful image guidance for computer-assisted orthopaedic surgery (CAOS). The normal vectors and tangent vectors are first extracted from the pre-operative and intra-operative point sets (PSs) respectively, and are further utilized to enhance the registration accuracy and robustness. The contributions of this article are three-folds. First, we propose and formulate a novel distribution that describes the error between one normal vector and the corresponding tangent vector based on the von-Mises Fisher (vMF) distribution. Second, by modelling the anisotropic position localization error with the multi-variate Gaussian distribution, we formulate the PSR considering anisotropic localization error as a maximum likelihood estimation (MLE) problem and then solve it under the expectation maximization (EM) framework. Third, to facilitate the optimization process, the gradients of the objective function with respect to the desired parameters are computed and presented. Extensive experimental results on the human femur and pelvis models verify that the proposed approach outperforms the state-of-the-art methods, and demonstrate potential clinical values for relevant surgical navigation applications.
- 26 April 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Robotics and Bionics
- Vol. 3 (2) , 372-383
Aligning 3D Curve With Surface Using Tangent and Normal Vectors for Computer-Assisted Orthopedic Surgery
Registration that aligns different views of one interested organ together is an essential technique and outstanding problem in medical robotics and image-guided surgery (IGS). This work introduces a novel rigid point set registration (PSR) approach that aims to accurately map the pre-operative space with the intra-operative space to enable successful image guidance for computer-assisted orthopaedic surgery (CAOS). The normal vectors and tangent vectors are first extracted from the pre-operative and intra-operative point sets (PSs) respectively, and are further utilized to enhance the registration accuracy and robustness. The contributions of this article are three-folds. First, we propose and formulate a novel distribution that describes the error between one normal vector and the corresponding tangent vector based on the von-Mises Fisher (vMF) distribution. Second, by modelling the anisotropic position localization error with the multi-variate Gaussian distribution, we formulate the PSR considering anisotropic localization error as a maximum likelihood estimation (MLE) problem and then solve it under the expectation maximization (EM) framework. Third, to facilitate the optimization process, the gradients of the objective function with respect to the desired parameters are computed and presented. Extensive experimental results on the human femur and pelvis models verify that the proposed approach outperforms the state-of-the-art methods, and demonstrate potential clinical values for relevant surgical navigation applications.
- 14 January 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Electronics
- Vol. 69 (1) , 582-591
OCR-RCNN: An Accurate and Efficient Framework for Elevator Button Recognition
Autonomous elevator operation is considered a promising solution for mobile navigation in office buildings. As a fundamental function, elevator button recognition remains unsolved due to the challenging image conditions and severe data imbalance problem. In this article, we propose an accurate and efficient framework, named OCR-RCNN, for elevator button recognition. The framework is comprised of an region-based convolutional neural network (R-CNN)-based button detector and an attention-RNN-based character recognizer. Leveraging the two components, we further propose an end-to-end architecture and a cascaded architecture to explore the most effective network design for the framework. Moreover, a perspective distortion removal algorithm is also developed to enhance the inference performance of OCR-RCNN. Another key contribution of this work is that we release the first large-scale elevator panel dataset with 2005 images and 21 767 button labels. Extensive experiments are conducted on the released dataset and other two publicly available datasets. The proposed framework achieves an F1 score of 0.94, 1.00, and 1.00 in detection task, and an accuracy of 79.6% 96.5%, and 96.4% in character recognition task. The results demonstrate the advantages of our method, outperforming alternative strategies and other state-of-the-art methods in the literature. The data and code are available on the project webpage https://github.com/zhudelong/ocr-rcnn-v2 .
- 14 January 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Electronics
- Vol. 69 (1) , 582-591
OCR-RCNN: An Accurate and Efficient Framework for Elevator Button Recognition
Autonomous elevator operation is considered a promising solution for mobile navigation in office buildings. As a fundamental function, elevator button recognition remains unsolved due to the challenging image conditions and severe data imbalance problem. In this article, we propose an accurate and efficient framework, named OCR-RCNN, for elevator button recognition. The framework is comprised of an region-based convolutional neural network (R-CNN)-based button detector and an attention-RNN-based character recognizer. Leveraging the two components, we further propose an end-to-end architecture and a cascaded architecture to explore the most effective network design for the framework. Moreover, a perspective distortion removal algorithm is also developed to enhance the inference performance of OCR-RCNN. Another key contribution of this work is that we release the first large-scale elevator panel dataset with 2005 images and 21 767 button labels. Extensive experiments are conducted on the released dataset and other two publicly available datasets. The proposed framework achieves an F1 score of 0.94, 1.00, and 1.00 in detection task, and an accuracy of 79.6% 96.5%, and 96.4% in character recognition task. The results demonstrate the advantages of our method, outperforming alternative strategies and other state-of-the-art methods in the literature. The data and code are available on the project webpage https://github.com/zhudelong/ocr-rcnn-v2 .
- 15 December 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 70 (00189456) , 1-13
An Autonomous Eye-in-Hand Robotic System for Elevator Button Operation Based on Deep Recognition Network
Autonomous elevator button operation is an indispensable function for robot-elevator interaction, which has long been considered an intelligent solution for multi-floor navigation of mobile robots. In this paper, we present an autonomous robotic system with an eye-in-hand configuration to address the button operation problem. Firstly, we develop a deep neural network for simultaneous button detection and character recognition, which provides accurate and robust perception inputs for the button operation system. Secondly, we present a button pose estimation algorithm with consideration of perception uncertainties, and the button pose is accurately estimated by fitting a least-uncertainty model. Based on the perception and pose estimation algorithms, a coarse-to-fine control schema is then proposed for driving the manipulator to accomplish the button operation task. Experiment results show that the proposed perception algorithm can outperform state-of-the-art methods in both recognition accuracy and running efficiency. The pose estimation and control schema also demonstrate their effectiveness in real-world button operation tasks. The data and code are available at our project webpage.
- 15 December 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 70 (00189456) , 1-13
An Autonomous Eye-in-Hand Robotic System for Elevator Button Operation Based on Deep Recognition Network
Autonomous elevator button operation is an indispensable function for robot-elevator interaction, which has long been considered an intelligent solution for multi-floor navigation of mobile robots. In this paper, we present an autonomous robotic system with an eye-in-hand configuration to address the button operation problem. Firstly, we develop a deep neural network for simultaneous button detection and character recognition, which provides accurate and robust perception inputs for the button operation system. Secondly, we present a button pose estimation algorithm with consideration of perception uncertainties, and the button pose is accurately estimated by fitting a least-uncertainty model. Based on the perception and pose estimation algorithms, a coarse-to-fine control schema is then proposed for driving the manipulator to accomplish the button operation task. Experiment results show that the proposed perception algorithm can outperform state-of-the-art methods in both recognition accuracy and running efficiency. The pose estimation and control schema also demonstrate their effectiveness in real-world button operation tasks. The data and code are available at our project webpage.
- 11 August 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 21 (10) , 11799-11809
ABC-Net: Area-Boundary Constraint Network With Dynamical Feature Selection for Colorectal Polyp Segmentation
Untreated colorectal polyps can develop into colorectal cancer, which is a leading cause of cancer-related deaths. Colonoscopy is a commonly-used method for colorectal polyp scanning, but limited to the experience and subjectivity of clinicians, one out of four polyps cannot be correctly recognized. In this paper, we propose an automatic colorectal polyp segmentation system based on the deep convolutional neural network, aiming to improve the accuracy of colorectal polyp scanning. The proposed ABC-Net is comprised of a shared encoder and two novel mutually-constrained decoders for simultaneous polyp area and boundary segmentation. To sufficiently exploit multi-scale image information, the selective feature modules are embedded into the network and used for dynamically learning and fusing multi-scale feature representations. Furthermore, a new boundary-sensitive loss is proposed to model the interdependencies between the area and boundary branches, the information of the two branches are reciprocally propagated and constrained, yielding a significant improvement in segmentation accuracy. Extensive experiments are conducted on three public colorectal polyp datasets, and the results, e.g., F1 scores are 0.866, 0.915, 0.874 in EndoScene, Kvasir-SEG, and ETIS-Larib datasets, demonstrate the advantages of the proposed method.
- 11 August 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 21 (10) , 11799-11809
ABC-Net: Area-Boundary Constraint Network With Dynamical Feature Selection for Colorectal Polyp Segmentation
Untreated colorectal polyps can develop into colorectal cancer, which is a leading cause of cancer-related deaths. Colonoscopy is a commonly-used method for colorectal polyp scanning, but limited to the experience and subjectivity of clinicians, one out of four polyps cannot be correctly recognized. In this paper, we propose an automatic colorectal polyp segmentation system based on the deep convolutional neural network, aiming to improve the accuracy of colorectal polyp scanning. The proposed ABC-Net is comprised of a shared encoder and two novel mutually-constrained decoders for simultaneous polyp area and boundary segmentation. To sufficiently exploit multi-scale image information, the selective feature modules are embedded into the network and used for dynamically learning and fusing multi-scale feature representations. Furthermore, a new boundary-sensitive loss is proposed to model the interdependencies between the area and boundary branches, the information of the two branches are reciprocally propagated and constrained, yielding a significant improvement in segmentation accuracy. Extensive experiments are conducted on three public colorectal polyp datasets, and the results, e.g., F1 scores are 0.866, 0.915, 0.874 in EndoScene, Kvasir-SEG, and ETIS-Larib datasets, demonstrate the advantages of the proposed method.
- 23 June 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 18 (2) , 471-483
Feature-Guided Nonrigid 3-D Point Set Registration Framework for Image-Guided Liver Surgery: From Isotropic Positional Noise to Anisotropic Positional Noise
Registration is an essential problem in image-guided surgery (IGS) since it brings different involved coordinate frames together. Nonrigid or deformable registration still faces many challenges, such as two point sets (PSs) are partially overlapped. To tackle the challenges in the nonrigid registration, we introduce a new two-step point-based registration pipeline that includes two steps. In the first step, the rigid transformation between the two spaces is recovered where the orientation vectors are adopted. In the second step, built on the nonrigid coherent point drift (CPD) approach, the anisotropic positional noise is also assumed. Registration results on the human liver verify the proposed approach' great improvements over the other methods. First, the rotation and translation are recovered with smaller error values than the existing methods. Second, our registration method's performance is much more robust to the partial overlapping between two PSs. Third, the two-step registration framework achieves the best performances in most test cases when there is a localization error in acquiring the intraoperative data.
- 23 June 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 18 (2) , 471-483
Feature-Guided Nonrigid 3-D Point Set Registration Framework for Image-Guided Liver Surgery: From Isotropic Positional Noise to Anisotropic Positional Noise
Registration is an essential problem in image-guided surgery (IGS) since it brings different involved coordinate frames together. Nonrigid or deformable registration still faces many challenges, such as two point sets (PSs) are partially overlapped. To tackle the challenges in the nonrigid registration, we introduce a new two-step point-based registration pipeline that includes two steps. In the first step, the rigid transformation between the two spaces is recovered where the orientation vectors are adopted. In the second step, built on the nonrigid coherent point drift (CPD) approach, the anisotropic positional noise is also assumed. Registration results on the human liver verify the proposed approach' great improvements over the other methods. First, the rotation and translation are recovered with smaller error values than the existing methods. Second, our registration method's performance is much more robust to the partial overlapping between two PSs. Third, the two-step registration framework achieves the best performances in most test cases when there is a localization error in acquiring the intraoperative data.
- 1 May 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 9953-9959
Robust and Accurate 3D Curve to Surface Registration with Tangent and Normal Vectors
This paper presents a robust and accurate approach for the rigid registration of pre-operative and intraoperative point sets in image-guided surgery (IGS). Three challenges are identified in the pre-to-intraoperative registration: the intra-operative 3D data (usually forms a 3D curve in space) (1) is often contaminated with noise and outliers; (2) usually only covers a partial region of the whole pre-operative model; (3) is usually sparse. To tackle those challenges, we utilize the tangent vectors extracted from the sparse intraoperative data points and the normal vectors extracted from the pre-operative model points. Our first contribution is to formulate a novel probabilistic distribution of the error between a pair of corresponding tangent and normal vectors. The second contribution is, based on the novel distribution, we formulate the registration of two multi-dimensional (6D) point sets as a maximum likelihood (ML) problem and solve it under the expectation maximization (EM) framework. Our last contribution is, in order to facilitate the computation process, the derivatives of the objective function with respect to desired parameters are presented. We conduct extensive experiments to demonstrate that our approach outperforms the state-of-the-art methods. Importantly, in the context of anteriro cruciate ligament (ACL) reconstruction, our method can achieve as low as 0.6795 mm mean target registration error (TRE) value with considerable noises and very limited overlapping ratios.
- 1 May 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 9953-9959
Robust and Accurate 3D Curve to Surface Registration with Tangent and Normal Vectors
This paper presents a robust and accurate approach for the rigid registration of pre-operative and intraoperative point sets in image-guided surgery (IGS). Three challenges are identified in the pre-to-intraoperative registration: the intra-operative 3D data (usually forms a 3D curve in space) (1) is often contaminated with noise and outliers; (2) usually only covers a partial region of the whole pre-operative model; (3) is usually sparse. To tackle those challenges, we utilize the tangent vectors extracted from the sparse intraoperative data points and the normal vectors extracted from the pre-operative model points. Our first contribution is to formulate a novel probabilistic distribution of the error between a pair of corresponding tangent and normal vectors. The second contribution is, based on the novel distribution, we formulate the registration of two multi-dimensional (6D) point sets as a maximum likelihood (ML) problem and solve it under the expectation maximization (EM) framework. Our last contribution is, in order to facilitate the computation process, the derivatives of the objective function with respect to desired parameters are presented. We conduct extensive experiments to demonstrate that our approach outperforms the state-of-the-art methods. Importantly, in the context of anteriro cruciate ligament (ACL) reconstruction, our method can achieve as low as 0.6795 mm mean target registration error (TRE) value with considerable noises and very limited overlapping ratios.