Anatomical‐functional image fusion based on deep convolution neural networks in local Laplacian pyramid domain
- 20 November 2020
- journal article
- research article
- Published by Wiley in International Journal of Imaging Systems and Technology
- Vol. 31 (3) , 1246-1264
Abstract
Medical image fusion technology makes clinical diagnosis and treatment more accurate. This technique can solve the existing problem that single mode medical images conveying insufficient information. Therefore, the key to this technology is to retain as much as possible information in the original medical image of multiple modes. However, the existing methods often lose source image detail with a low contrast and cause color distortion. This article proposed a novel multi‐modal medical image fusion framework. Our framework has four key steps. First, extracting the super‐resolution images from anatomical images via a deep neural network is performed. Second, decomposing the source images into detail‐enhanced approximate and residual images, and then processing by local Laplacian filtering. Third, a convolution neural network is used to complete the mapping of the source image to the feature map and the local energy‐based strategy is used to integrate the coefficients. Finally, the inversed local Laplacian pyramid is adopted to reconstruct the fused image. The experimental results prove that the fused image via the proposed method has the characteristics of high contrast, high brightness, and high color saturation and great advantages in retaining the color and structural information.Keywords
Funding Information
- National Basic Research Program of China (2019YFE0110800)
- National Natural Science Foundation of China (61972060, U1713213)
- Natural Science Foundation of Chongqing Municipality (cstc2019jcyj‐cxttX0002, cstc2019jcyj‐zdxmX0011)
This publication has 45 references indexed in Scilit:
- Medical Image Fusion and Denoising with Alternating Sequential Filter and Adaptive Fractional Order Total VariationIEEE Transactions on Instrumentation and Measurement, 2017
- Multiple Visual Features Measurement With Gradient Domain Guided Filtering for Multisensor Image FusionIEEE Transactions on Instrumentation and Measurement, 2017
- A general framework for image fusion based on multi-scale transform and sparse representationInformation Fusion, 2015
- A new contrast based multimodal medical image fusion frameworkNeurocomputing, 2015
- Medical image fusion: A survey of the state of the artInformation Fusion, 2014
- Fusion of multimodal medical images using Daubechies complex wavelet transform – A multiresolution approachInformation Fusion, 2014
- Including Anatomical and Functional Information in MC Simulation of PET and SPECT Brain Studies. Brain-VISET: A Voxel-Based Iterative MethodIEEE Transactions on Medical Imaging, 2014
- Image Fusion With Guided FilteringIEEE Transactions on Image Processing, 2013
- Cross-Scale Coefficient Selection for Volumetric Medical Image FusionIEEE Transactions on Biomedical Engineering, 2012
- MRI and PET image fusion by combining IHS and retina-inspired modelsInformation Fusion, 2010