GLStyleNet: exquisite style transfer combining global and local pyramid features
Authors:
Open Access
- 14 October 2020
- journal article
- research article
- Published by Institution of Engineering and Technology (IET) in IET Computer Vision
- Vol. 14 (8) , 575-586
Abstract
Recent studies using deep neural networks have shown remarkable success in style transfer, especially for artistic and photo‐realistic images. However, these methods cannot solve more sophisticated problems. The approaches using global statistics fail to capture small, intricate textures and maintain correct texture scales of the artworks, and the others based on local patches are defective on global effect. To address these issues, this study presents a unified model [global and local style network (GLStyleNet)] to achieve exquisite style transfer with higher quality. Specifically, a simple yet effective perceptual loss is proposed to consider the information of global semantic‐level structure, local patch‐level style, and global channel‐level effect at the same time. This could help transfer not just large‐scale, obvious style cues but also subtle, exquisite ones, and dramatically improve the quality of style transfer. Besides, the authors introduce a novel deep pyramid feature fusion module to provide a more flexible style expression and a more efficient transfer process. This could help retain both high‐frequency pixel information and low‐frequency construct information. They demonstrate the effectiveness and superiority of their approach on numerous style transfer tasks, especially the Chinese ancient painting style transfer. Experimental results indicate that their unified approach improves image style transfer quality over previous state‐of‐the‐art methods.Keywords
All Related Versions
This publication has 41 references indexed in Scilit:
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Understanding deep image representations by inverting them2015
- Optimal Transportation for Example-Guided Color Transfer2015
- Image quilting for texture synthesis and transfer2001
- Color transfer between imagesIEEE Computer Graphics and Applications, 2001
- Fast texture synthesis using tree-structured vector quantization2000
- Texture synthesis by non-parametric sampling1999
- Algorithm 778: L-BFGS-BACM Transactions on Mathematical Software, 1997
- Multiresolution sampling procedure for analysis and synthesis of texture images1997
- Pyramid-based texture analysis/synthesis1995