Visual Perception and Convolutional Neural Network-Based Robotic Autonomous Lung Ultrasound Scanning Localization System
- 31 March 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control
- Vol. 70 (9) , 961-974
Abstract
Under the situation of severe COVID-19 epidemic, lung ultrasound (LUS) has been proved to be an effective and convenient method to diagnose and evaluate the extent of respiratory disease. However, the traditional clinical ultrasound (US) scanning requires doctors not only to be in close contact with patients, but also to have rich experience. In order to alleviate the shortage of medical resources and reduce the work stress and risk of infection for doctors, we propose a visual perception and convolutional neural network(CNN) based robotic autonomous LUS scanning localization system to realize scanned target recognition, probe pose solution and movement, and the acquisition of US images. The LUS scanned targets are identified through the target segmentation and localization algorithm based on the improved CNN, which is using the depth camera to collect the image information; Furthermore, the method based on multi-scale compensation normal vector is used to solve the attitude of the probe; Finally, a position control strategy based on force feedback is designed to optimize the position and attitude of the probe, which can not only obtain high-quality US images but also ensure the safety of patients and the system. The results of human LUS scanning experiment verify the accuracy and feasibility of the system. The positioning accuracy of the scanned targets is 15.63±0.18 mm, and the distance accuracy and rotation angle accuracy of the probe position calculation are 6.38 ± 0.25 mm and 8.60 ± 2.29 degrees, respectively. More importantly, the obtained high-quality US images can clearly capture the main pathological features of the lung. The system is expected to be applied in clinical practice.Keywords
Funding Information
- Guangdong Provincial Key Laboratory of Biomedical Optical Imaging Technology (2020B121201010-4)
- National Natural Science Foundation of China (62275062)
This publication has 33 references indexed in Scilit:
- A membrane parallel rapidly-exploring random tree algorithm for robotic motion planningIntegrated Computer-Aided Engineering, 2020
- Novel Coronavirus Pneumonia Outbreak in 2019: Computed Tomographic Findings in Two CasesKorean Journal of Radiology, 2020
- Robotic Arm Based Automatic Ultrasound Scanning for Three-Dimensional ImagingIEEE Transactions on Industrial Informatics, 2018
- Squeeze-and-Excitation Networks2018
- Robotic ultrasound trajectory planning for volume of interest coverage2016
- A review of robotic mechanisms for ultrasound examinationsIndustrial Robot: the international journal of robotics research and application, 2014
- Results of the Two Incidence Screenings in the National Lung Screening TrialNew England Journal of Medicine, 2013
- Robotic ultrasound systems in medicineIEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 2013
- Robotic Ultrasound Guidance by B-scan Plane Positioning ControlProcedia CIRP, 2013
- Control of an ultrasound probe by adaptive visual servoing2005