Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Top Cited Papers
Preprint
Abstract
We consider image transformation problems, where an input image is transformed into an output image. Recent methods for such problems typically train feed-forward convolutional neural networks using a \emph{per-pixel} loss between the output and ground-truth images. Parallel work has shown that high-quality images can be generated by defining and optimizing \emph{perceptual} loss functions based on high-level features extracted from pretrained networks. We combine the benefits of both approaches, and propose the use of perceptual loss functions for training feed-forward networks for image transformation tasks. We show results on image style transfer, where a feed-forward network is trained to solve the optimization problem proposed by Gatys et al in real-time. Compared to the optimization-based method, our network gives similar qualitative results but is three orders of magnitude faster. We also experiment with single-image super-resolution, where replacing a per-pixel loss with a perceptual loss gives visually pleasing results.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Inverting Visual Representations with Convolutional Networks2016
- Deep Residual Learning for Image Recognition2016
- Fully Convolutional Networks for Semantic SegmentationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- Learning Deconvolution Network for Semantic Segmentation2015
- Image Super-Resolution Using Deep Convolutional NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- On Single Image Scale-Up Using Sparse-Representations2012
- Fast image/video upsamplingACM Transactions on Graphics, 2008
- Scope of validity of PSNR in image/video quality assessmentElectronics Letters, 2008
- A Statistical Evaluation of Recent Full Reference Image Quality Assessment AlgorithmsIEEE Transactions on Image Processing, 2006
- Image up-sampling using total-variation regularization with a new observation modelIEEE Transactions on Image Processing, 2005