Kyohoon Sim
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Kyohoon Sim has a total of 11 co-authors. Here are the top co-authors by number of shared publications:
- Jiachen Yang
Tianjin University, Tianjin, China
4 shared publications Wen Lu
School of Electronic Engineering, Xidian University, Xian, China
3 shared publications- Wen Lu
Xidian University, Xi'an, China
2 shared publications - Xinbo Gao
Xidian University, Xi'an, China
2 shared publications - Bin Jiang
School of Electrical and Information Engineering, Tianjin Univeristy, Tianjin China
1 shared publication Hehan Liu
1 shared publication
Latest Publications
- 17 March 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 24 (15209210) , 1389-1398
Blind Stereoscopic Image Quality Evaluator Based on Binocular Semantic and Quality Channels
Human beings always evaluate the perceptual quality of an image coupled with identifying the semantic content of images. This paper addresses the correlation issue between stereoscopic image quality assessment (SIQA) and semantic recognition. In contrast to the previous SIQA methods that relied on binocular quality-aware features of a stereoscopic image, our approach also extracts binocular semantic features using a pre-trained deep convolutional neural network (DCNN) on a large dataset like ImageNet dataset, as well as the manually designed binocular quality-aware features. It can solve the problem of limited SIQA dataset size and facilitate better prediction on the quality. Experimental results demonstrate that the binocular semantic features are a good predictor for the stereoscopic image quality. The proposed method outperforms the state-of-the-art SIQA methods on four benchmark SIQA datasets. Significantly, all Spearman rank-order correlation coefficients (SROCCs) between the predicted scores and the subjective scores on the four datasets exceed 0.95. The MATLAB source code of the proposed method is available at https://github.com/kyohoonsim/Blind-Stereoscopic-Image-Quality-Evaluator-based-on-Binocular-Semantic-and-Quality-Channels https://github.com/kyohoonsim .
- 17 March 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 24 (15209210) , 1389-1398
Blind Stereoscopic Image Quality Evaluator Based on Binocular Semantic and Quality Channels
Human beings always evaluate the perceptual quality of an image coupled with identifying the semantic content of images. This paper addresses the correlation issue between stereoscopic image quality assessment (SIQA) and semantic recognition. In contrast to the previous SIQA methods that relied on binocular quality-aware features of a stereoscopic image, our approach also extracts binocular semantic features using a pre-trained deep convolutional neural network (DCNN) on a large dataset like ImageNet dataset, as well as the manually designed binocular quality-aware features. It can solve the problem of limited SIQA dataset size and facilitate better prediction on the quality. Experimental results demonstrate that the binocular semantic features are a good predictor for the stereoscopic image quality. The proposed method outperforms the state-of-the-art SIQA methods on four benchmark SIQA datasets. Significantly, all Spearman rank-order correlation coefficients (SROCCs) between the predicted scores and the subjective scores on the four datasets exceed 0.95. The MATLAB source code of the proposed method is available at https://github.com/kyohoonsim/Blind-Stereoscopic-Image-Quality-Evaluator-based-on-Binocular-Semantic-and-Quality-Channels https://github.com/kyohoonsim .
- 1 January 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 23 (15209210) , 4037-4048
MaD-DLS: Mean and Deviation of Deep and Local Similarity for Image Quality Assessment
When human visual system (HVS) looks at a scene, it extracts various features from the image about the scene to understand it. The extracted features are compared with the stored memory on the analogous scene to judge their similarity [1]. By analyzing to the similarity, HVS understands the scene presented on eyes. Based on the neurobiological basis, we propose a 2D full reference (FR) image quality assessment (IQA) method, named mean and deviation of deep and local similarity (MaD-DLS) that compares similarity between many original and distorted deep feature maps from convolutional neural networks (CNNs). MaD-DLS uses a deep learning algorithm, but since it uses the convolutional layers of a pre-trained model, it is free from training. For pooling of local quality scores within a deep similarity map, we employ two important descriptive statistics, (weighted) mean and standard deviation and name it mean and deviation (MaD) pooling. The two statistics each have the physical meaning: the weighted mean reflects effect of visual saliency on quality, whereas the standard deviation reflects effect of distortion distribution within the image on it. Experimental results show that MaD-DLS is superior or competitive to the existing methods and the MaD pooling is effective. The MATLAB source code of MaD-DLS will be available online soon (after the manuscript is accepted).
- 1 January 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 23 (15209210) , 4037-4048
MaD-DLS: Mean and Deviation of Deep and Local Similarity for Image Quality Assessment
When human visual system (HVS) looks at a scene, it extracts various features from the image about the scene to understand it. The extracted features are compared with the stored memory on the analogous scene to judge their similarity [1]. By analyzing to the similarity, HVS understands the scene presented on eyes. Based on the neurobiological basis, we propose a 2D full reference (FR) image quality assessment (IQA) method, named mean and deviation of deep and local similarity (MaD-DLS) that compares similarity between many original and distorted deep feature maps from convolutional neural networks (CNNs). MaD-DLS uses a deep learning algorithm, but since it uses the convolutional layers of a pre-trained model, it is free from training. For pooling of local quality scores within a deep similarity map, we employ two important descriptive statistics, (weighted) mean and standard deviation and name it mean and deviation (MaD) pooling. The two statistics each have the physical meaning: the weighted mean reflects effect of visual saliency on quality, whereas the standard deviation reflects effect of distortion distribution within the image on it. Experimental results show that MaD-DLS is superior or competitive to the existing methods and the MaD pooling is effective. The MATLAB source code of MaD-DLS will be available online soon (after the manuscript is accepted).
- 1 October 2019
- Published by Elsevier in Signal Processing: Image Communication
- Vol. 78, 51-61
No-reference image quality assessment focusing on human facial region
- 1 October 2019
- Published by Elsevier in Signal Processing: Image Communication
- Vol. 78, 51-61
No-reference image quality assessment focusing on human facial region
- 26 August 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Broadcasting
- Vol. 66 (2) , 279-291
3-D Visual Discomfort Assessment Considering Optical and Neural Attention Models
In this paper, we find factors affecting visual comfort along the visual pathways. The statistics data used for the 3D visual discomfort predictor (3D-VDP) model extracted from the traditional optical statistical elements derived, retinal disparity and neurophysiological elements obtained from V1 and MT areas. These physiological statistics are used to create a new feature map for 3D-VDP models, known as MT importance map. The thirteen typical MT neurons are divided into four kinds of cells: tuned-excitatory cells, tuned-inhibitory cells, far cells and near cells based on their characteristics. Disparity map is also transformed by changing the viewing distance in the formula to calculate the horizontal angular disparity factor. We propose that variation of viewing distance will make the MT cells more intense reaction. As a result, three types of angular disparity maps are obtained: tuned-near, tuned-far and original disparity maps. Preliminary results on two public available databases exhibit the performance of the proposed approach outperforms previous S3D VDP models.
- 26 August 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Broadcasting
- Vol. 66 (2) , 279-291
3-D Visual Discomfort Assessment Considering Optical and Neural Attention Models
In this paper, we find factors affecting visual comfort along the visual pathways. The statistics data used for the 3D visual discomfort predictor (3D-VDP) model extracted from the traditional optical statistical elements derived, retinal disparity and neurophysiological elements obtained from V1 and MT areas. These physiological statistics are used to create a new feature map for 3D-VDP models, known as MT importance map. The thirteen typical MT neurons are divided into four kinds of cells: tuned-excitatory cells, tuned-inhibitory cells, far cells and near cells based on their characteristics. Disparity map is also transformed by changing the viewing distance in the formula to calculate the horizontal angular disparity factor. We propose that variation of viewing distance will make the MT cells more intense reaction. As a result, three types of angular disparity maps are obtained: tuned-near, tuned-far and original disparity maps. Preliminary results on two public available databases exhibit the performance of the proposed approach outperforms previous S3D VDP models.
- 24 December 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 21 (7) , 1750-1761
Predicting Stereoscopic Image Quality via Stacked Auto-Encoders Based on Stereopsis Formation
Most of previous 2D and 3D image quality evaluators were based on shallow architectures. Their shallow architectures cannot model phenomenon occurring in human visual system (HVS) sufficiently. Disparities between left and right views have been importantly used for 3D image quality assessment (IQA), but single disparity, depth or cyclopean maps made from the disparities cannot thoroughly reflect depth sense. In this paper, we propose a blind stereoscopic image quality evaluator using Stacked Auto-Encoders (SAE). The proposed method is based on two theories on initial stages of stereopsis. One is cyclopean channel theory, the other is binocular summation/difference channels theory. Especially a cyclopean image which models the former theory is computed to consider binocular suppression, whereas summation and difference images which model the latter one are utilized to treat the depth sense. We train three SAEs for quality-aware features from the three images in an unsupervised manner. Through the SAEs, the features are transformed into more meaningful features, and they are used to train two regressors. The regressors can be used to get a final predicted score. Experimental results conducted on popular 3D IQA databases prove that the proposed algorithm outperforms state-of-the-art 3D IQA methods.
- 24 December 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 21 (7) , 1750-1761
Predicting Stereoscopic Image Quality via Stacked Auto-Encoders Based on Stereopsis Formation
Most of previous 2D and 3D image quality evaluators were based on shallow architectures. Their shallow architectures cannot model phenomenon occurring in human visual system (HVS) sufficiently. Disparities between left and right views have been importantly used for 3D image quality assessment (IQA), but single disparity, depth or cyclopean maps made from the disparities cannot thoroughly reflect depth sense. In this paper, we propose a blind stereoscopic image quality evaluator using Stacked Auto-Encoders (SAE). The proposed method is based on two theories on initial stages of stereopsis. One is cyclopean channel theory, the other is binocular summation/difference channels theory. Especially a cyclopean image which models the former theory is computed to consider binocular suppression, whereas summation and difference images which model the latter one are utilized to treat the depth sense. We train three SAEs for quality-aware features from the three images in an unsupervised manner. Through the SAEs, the features are transformed into more meaningful features, and they are used to train two regressors. The regressors can be used to get a final predicted score. Experimental results conducted on popular 3D IQA databases prove that the proposed algorithm outperforms state-of-the-art 3D IQA methods.