Hybrid Multimodal Medical Image Fusion Method Based on LatLRR and ED-D2GAN
Open Access
- 12 December 2022
- journal article
- research article
- Published by MDPI AG in Applied Sciences
- Vol. 12 (24) , 12758
Abstract
In order to better preserve the anatomical structure information of Computed Tomography (CT) source images and highlight the metabolic information of lesion regions in Positron Emission Tomography (PET) source images, a hybrid multimodal medical image fusion method (LatLRR-GAN) based on Latent low-rank representation (LatLRR) and the dual discriminators Generative Adversarial Network (ED-D2GAN) is proposed. Firstly, considering the denoising capability of LatLRR, source images were decomposed by LatLRR. Secondly, the ED-D2GAN model was put forward as the low-rank region fusion method, which can fully extract the information contained by the low-rank region images. Among them, encoder and decoder networks were used in the generator; convolutional neural networks were also used in dual discriminators. Thirdly, a threshold adaptive weighting algorithm based on the region energy ratio is proposed as the salient region fusion rule, which can improve the overall sharpness of the fused image. The experimental results show that compared with the best methods of the other six methods, this paper is effective in multiple objective evaluation metrics, including the average gradient, edge intensity, information entropy, spatial frequency and standard deviation. The results of the two experiments are improved by 35.03%, 42.42%, 4.66%, 8.59% and 11.49% on average.Keywords
Funding Information
- National Natural Science Foundation of China (62062003)
- Natural Science Foundation of Ningxia Province (2022AAC03149)
- North Minzu University Research Project of Talent Introduction (2020KYQD08)
This publication has 27 references indexed in Scilit:
- Multi-focus image fusion with a deep convolutional neural networkInformation Fusion, 2017
- Medical image fusion based on sparse representation of classified image patchesBiomedical Signal Processing and Control, 2017
- Curvelet transform with learning-based tilingSignal Processing: Image Communication, 2017
- Local energy‐based multimodal medical image fusion in curvelet domainIET Computer Vision, 2016
- A general framework for image fusion based on multi-scale transform and sparse representationInformation Fusion, 2015
- Group-Sparse Representation With Dictionary Learning for Medical Image Denoising and FusionIEEE Transactions on Biomedical Engineering, 2012
- Assessment of image fusion procedures using entropy, image quality, and multispectral classificationJournal of Applied Remote Sensing, 2008
- Information Representation for Image Fusion Evaluation2006
- Objective image fusion performance measureElectronics Letters, 2000
- Image quality measures and their performanceIEEE Transactions on Communications, 1995