MaD-DLS: Mean and Deviation of Deep and Local Similarity for Image Quality Assessment
- 1 January 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 23 (15209210) , 4037-4048
Abstract
When human visual system (HVS) looks at a scene, it extracts various features from the image about the scene to understand it. The extracted features are compared with the stored memory on the analogous scene to judge their similarity [1]. By analyzing to the similarity, HVS understands the scene presented on eyes. Based on the neurobiological basis, we propose a 2D full reference (FR) image quality assessment (IQA) method, named mean and deviation of deep and local similarity (MaD-DLS) that compares similarity between many original and distorted deep feature maps from convolutional neural networks (CNNs). MaD-DLS uses a deep learning algorithm, but since it uses the convolutional layers of a pre-trained model, it is free from training. For pooling of local quality scores within a deep similarity map, we employ two important descriptive statistics, (weighted) mean and standard deviation and name it mean and deviation (MaD) pooling. The two statistics each have the physical meaning: the weighted mean reflects effect of visual saliency on quality, whereas the standard deviation reflects effect of distortion distribution within the image on it. Experimental results show that MaD-DLS is superior or competitive to the existing methods and the MaD pooling is effective. The MATLAB source code of MaD-DLS will be available online soon (after the manuscript is accepted).Keywords
Funding Information
- National Natural Science Foundation of China (61871283)
- Foundation of Pre-Research on Equipment of China (61400010304)
- Major Civil-Military Integration Project in Tianjin, China (18ZXJMTG00170)
This publication has 31 references indexed in Scilit:
- The Application of Visual Saliency Models in Objective Image Quality Assessment: A Statistical EvaluationIEEE Transactions on Neural Networks and Learning Systems, 2015
- Going deeper with convolutions2015
- The Analysis of Image Contrast: From Quality Assessment to Automatic EnhancementIEEE Transactions on Cybernetics, 2015
- VSI: A Visual Saliency-Induced Index for Perceptual Image Quality AssessmentIEEE Transactions on Image Processing, 2014
- Return of the Devil in the Details: Delving Deep into Convolutional Nets2014
- SR-SIM: A fast and high performance IQA index based on spectral residual2012
- Full Reference Image Quality Assessment Based on Saliency Map AnalysisJournal of Imaging Science and Technology, 2010
- Multiscale structural similarity for image quality assessment2004
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- Segregation of Form, Color, Movement, and Depth: Anatomy, Physiology, and PerceptionScience, 1988