Min-Cheol Hong

Motion blur is a complex phenomenon caused by the relative movement between an observed object and an imaging sensor during the exposure time, resulting in degradation in the image quality. Deep-learning-based methods, particularly convolutional neural networks (CNNs), have shown promise in motion deblurring. However, the small kernel sizes of CNNs limit their ability to achieve optimal performance. Moreover, supervised deep-learning-based deblurring methods often exhibit overfitting in their training datasets. Models trained on widely used synthetic blur datasets frequently fail to generalize in other blur domains in real-world scenarios and often produce undesired artifacts. To address these challenges, we propose the Spatial Feature Selection Network (SFSNet), which incorporates a Regional Feature Extractor (RFE) module to expand the receptive field and effectively select critical spatial features in order to improve the deblurring performance. In addition, we present the BlurMix dataset, which includes diverse blur types, as well as a meta-tuning strategy for effective blur domain adaptation. Our method enables the network to rapidly adapt to novel blur distributions with minimal additional training, and thereby improve generalization. The experimental results show that the meta-tuning variant of the SFSNet eliminates unwanted artifacts and significantly improves the deblurring performance across various blur domains.

The motion of an object or camera platform makes the acquired image blurred. This degradation is a major reason to obtain a poor-quality image from an imaging sensor. Therefore, developing an efficient deep-learning-based image processing method to remove the blur artifact is desirable. Deep learning has recently demonstrated significant efficacy in image deblurring, primarily through convolutional neural networks (CNNs) and Transformers. However, the limited receptive fields of CNNs restrict their ability to capture long-range structural dependencies. In contrast, Transformers excel at modeling these dependencies, but they are computationally expensive for high-resolution inputs and lack the appropriate inductive bias. To overcome these challenges, we propose an Efficient Hybrid Network (EHNet) that employs CNN encoders for local feature extraction and Transformer decoders with a dual-attention module to capture spatial and channel-wise dependencies. This synergy facilitates the acquisition of rich contextual information for high-quality image deblurring. Additionally, we introduce the Simple Feature-Embedding Module (SFEM) to replace the pointwise and depthwise convolutions to generate simplified embedding features in the self-attention mechanism. This innovation substantially reduces computational complexity and memory usage while maintaining overall performance. Finally, through comprehensive experiments, our compact model yields promising quantitative and qualitative results for image deblurring on various benchmark datasets.

Low-light images are susceptible to poor visibility, low contrast, and additive noise. Although various techniques can enhance brightness to a reasonable extent, they inevitably amplify noise and cause color distortion, halo artifacts, and other forms of deterioration. Most low-light image enhancement (LLIE) methods utilize multiple algorithms or models to remove the corresponding deterioration types while brightening the image. However, an optimization strategy can be difficult, primarily because each sub-model has its own set of hyperparameters that must be tuned. To address this problem, we propose an end-to-end network, called CBMD-Net, which concurrently brightens and mitigates the deterioration in low-light images. Specifically, our network is developed on a U-shaped backbone with several stacked pyramid depthwise separable convolution (PDC) modules, which accurately extract meaningful spatial features from the network. Additionally, a new upsampling module is presented to efficiently restore the spatial resolution of the feature maps. Finally, we discovered that the latent representations at the network bottleneck contain relevant information regarding the input image, including deteriorated features that can have repercussions on image quality. Therefore, we introduce an intermediate loss function at the bottleneck to mitigate the additive noise and negative effects. Comprehensive experiments demonstrate that the developed network can obtain satisfactory results from both subjective and objective comparisons against state-of-the-art methods.

Image denoising is regarded as an ill-posed problem in computer vision tasks that removes additive noise from imaging sensors. Recently, several convolution neural network-based image-denoising methods have achieved remarkable advances. However, it is difficult for a simple denoising network to recover aesthetically pleasing images owing to the complexity of image content. Therefore, this study proposes a multi-branch network to improve the performance of the denoising method. First, the proposed network is designed based on a conventional autoencoder to learn multi-level contextual features from input images. Subsequently, we integrate two modules into the network, including the Pyramid Context Module (PCM) and the Residual Bottleneck Attention Module (RBAM), to extract salient information for the training process. More specifically, PCM is applied at the beginning of the network to enlarge the receptive field and successfully address the loss of global information using dilated convolution. Meanwhile, RBAM is inserted into the middle of the encoder and decoder to eliminate degraded features and reduce undesired artifacts. Finally, extensive experimental results prove the superiority of the proposed method over state-of-the-art deep-learning methods in terms of objective and subjective performances.

Facial expression recognition is crucial for understanding human emotions and nonverbal communication. With the growing prevalence of facial recognition technology and its various applications, accurate and efficient facial expression recognition has become a significant research area. However, most previous methods have focused on designing unique deep-learning architectures while overlooking the loss function. This study presents a new loss function that allows simultaneous consideration of inter- and intra-class variations to be applied to CNN architecture for facial expression recognition. More concretely, this loss function reduces the intra-class variations by minimizing the distances between the deep features and their corresponding class centers. It also increases the inter-class variations by maximizing the distances between deep features and their non-corresponding class centers, and the distances between different class centers. Numerical results from several benchmark facial expression databases, such as Cohn-Kanade Plus, Oulu-Casia, MMI, and FER2013, are provided to prove the capability of the proposed loss function compared with existing ones.

Images captured under low-light conditions are typically prone to undesirable visual phenomena, particularly color distortion and additive noise, that impede the aesthetics and performance of high-level vision tasks. Therefore, developing an enhancement method is necessary to obtain visually pleasing images. This paper proposes deep multiple networks for low-light image enhancement based on the Retinex model (defined as DMT-Net), wherein each network is configured to perform its own role in the YCbCr color space. More specifically, Decoupled-Net is presented to accurately decouple the luminance channel into reflectance and illumination components using the Retinex model. Denoising-Net is connected to Decoupled-Net to eradicate additive noise in the reflectance component. Boosting-Net enhances the illumination component and reduces halo artifacts. In addition, we propose Chrominance- Net to mitigate the distortion of the Cb and Cr channels. Our study also focuses on having a suitable norm order for the loss function based on kurtosis characteristics, such that each network is properly trained to minimize the residual in the training process. Finally, the experimental results prove that our proposed method outperforms numerous other cutting-edge methods in both qualitative and quantitative terms.

Facial expression recognition is crucial for understanding human emotions and nonverbal communication. With the growing prevalence of facial recognition technology and its various applications, accurate and efficient facial expression recognition has become a significant research area. However, most previous methods have focused on designing unique deep-learning architectures while overlooking the loss function. This study presents a new loss function that allows simultaneous consideration of inter- and intra-class variations to be applied to CNN architecture for facial expression recognition. More concretely, this loss function reduces the intra-class variations by minimizing the distances between the deep features and their corresponding class centers. It also increases the inter-class variations by maximizing the distances between deep features and their non-corresponding class centers, and the distances between different class centers. Numerical results from several benchmark facial expression databases, such as Cohn-Kanade Plus, Oulu-Casia, MMI, and FER2013, are provided to prove the capability of the proposed loss function compared with existing ones.

This study introduces a low-light image enhancement method using a hybrid deep-learning network and mixed-norm loss functions, in which the network consists of a decomposition-net, illuminance enhance-net, and chroma-net. To consider the correlation between R, G, and B channels, YCbCr channels converted from the RGB channels are used for training and restoration processes. With the luminance, the decomposition-net aims to decouple the reflectance and illuminance and to train the reflectance, leading to a more accurate feature map with noise reduction. The illumination enhance-net connected to the decomposition-net is used to enhance the illumination such that the illuminance is improved with reduced halo artifacts. In addition, the chroma-net is independently used to reduce color distortion. Moreover, a mixed-norm loss function used in the training process of each network is described to increase the stability and remove blurring in the reconstructed image by reflecting the properties of reflectance, illuminance, and chroma. The experimental results demonstrate that the proposed method leads to promising subjective and objective improvements over state-of-the-art deep-learning methods.

This paper introduces an adaptive image rendering using a parametric nonlinear mapping-function-based on the retinex model in a low-light source. For this study, only a luminance channel was used to estimate the reflectance component of an observed low-light image, therefore halo artifacts coming from the use of the multiple center/surround Gaussian filters were reduced. A new nonlinear mapping function that incorporates the statistics of the luminance and the estimated reflectance in the reconstruction process is proposed. In addition, a new method to determine the gain and offset of the mapping function is addressed to adaptively control the contrast ratio. Finally, the relationship between the estimated luminance and the reconstructed luminance is used to reconstruct the chrominance channels. The experimental results demonstrate that the proposed method leads to the promised subjective and objective improvements over state-of-the-art, scale-based retinex methods.

This paper introduces a new hole-filling method using extrapolated spatio-temporal background information to obtain a synthesized free-view. New temporal background modeling is proposed, which incorporates stationary temporal information in the hole-filling process and preserves the temporal consistency of synthesized views. A background codebook is distinguished from a non-overlapped patch-based codebook, which contributes to extracting reliable temporal background information. Furthermore, a depth-map driven spatial local background estimation is also addressed to discriminate the background holes in each disocclusion and to define two spatial BG constraints that represent the lower and upper bounds of a background candidate. Holes are filled by comparing the similarities between the temporal background information and the spatial background constraints. In addition, a depth map-based ghost removal filter is described to solve the problem of the non-fit between a color image and the corresponding depth map of a virtual view. Finally, an exemplar-based inpainting is applied to fill in the remaining holes with a priority function that includes a new depth term. The experimental results demonstrated that the proposed method led to results that promised subjective and objective improvement over state-of-the-art methods.