TPU Based Deep Learning Image Enhancement for Real-Time Point-of-Care Ultrasound
- 8 March 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computational Imaging
- Vol. 10 (25730436) , 461-468
Abstract
Handheld ultrasound devices are becoming more prevalent in point-of-care ultrasound workflows. However, these devices are computationally constrained which challenges the advances in deep learning methodology for real-time use on mobile Point-of-care ultrasound (POCUS) devices. In this work, we explore the feasibility of running MimickNet, a deep learning clinical post-processing model, on Tensor Processing Units (TPUs), hardware designed for deep learning operations capable of running on only 2 watts of power at 1.8 V with a form factor of 10 mm x 15 mm. We show that real-time deep learning based post-processing is feasible at 20 - 120 FPS for 1472x160 to 224x224 axial sample x B-mode scan line configurations. We refer to the TPU based model as MimickNet Mobile. MimickNet Mobile achieves outputs nearly identical to the original MimickNet with a structural similarity index measurement (SSIM) of 0.98 $\pm$ 0.001 and a mean squared error (MSE) of 0.0001 $\pm$ 0.0 over our test set of 588 frames consisting of 168 phantom frames and 420 prospectively acquired human liver frames. We investigate the latency of other common mobile architectures such as separable convolution. Finally, we investigate the distribution of model parameter error when quantizing MimickNet $\mathtt{float32}$ weights to MimickNet Mobile $\mathtt{int8}$ weights. This work demonstrates that real-time POCUS deep learning image enhancement is feasible using TPUs. Future ultrasound device manufacturers can consider incorporating a TPU for the added flexibility of supporting several deep learning architectures without compromising on power management and form factor. We make code and models available at: https://github.com/ouwen/mimicknetmobile .Keywords
Funding Information
- Ruth L. Kirschstein National Research Service (5F30HL156547-03)
- National Institute of Biomedical Imaging and Bioengineering (R01-EB026574, 5T32GM007171-44)
- National Institute of Health
This publication has 21 references indexed in Scilit:
- Real-time denoising of ultrasound images based on deep learningMedical & Biological Engineering & Computing, 2022
- Leapfrogging Medical AI in Low-Resource Contexts Using Edge Tensor Processing Unit2022
- Improving Contrast of Fundamental Ultrasound Imaging Using a Deep Neural Network2021
- Image Quality Improvement of Hand-Held Ultrasound Devices With a Two-Stage Generative Adversarial NetworkIEEE Transactions on Biomedical Engineering, 2019
- Point of care ultrasound: Breaking the sound barrier in the emergency departmentAustralasian Journal of Ultrasound in Medicine, 2019
- Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference2018
- Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks2017
- Xception: Deep Learning with Depthwise Separable Convolutions2017
- Utility of Hand-Held Echocardiography in Outpatient Pediatric Cardiology ManagementPediatric Cardiology, 2014
- Point-of-Care UltrasonographyNew England Journal of Medicine, 2011