Fast Image Processing with Fully-Convolutional Networks
Authors:
- Qifeng ChenQCQifeng Chen
- Jia XuJXJia Xu
- Vladlen KoltunVKVladlen Koltun
Top Cited Papers
- 1 October 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2516-2525
Abstract
We present an approach to accelerating a wide variety of image processing operators. Our approach uses a fully-convolutional network that is trained on input-output pairs that demonstrate the operator's action. After training, the original operator need not be run at all. The trained network operates at full resolution and runs in constant time. We investigate the effect of network architecture on approximation accuracy, runtime, and memory footprint, and identify a specific architecture that balances these considerations. We evaluate the presented approach on ten advanced image processing operators, including multiple variational models, multiscale tone and detail manipulation, photographic style transfer, nonlocal dehazing, and nonphoto-realistic stylization. All operators are approximated by the same model. Experiments demonstrate that the presented approach is significantly more accurate than prior approximation schemes. It increases approximation accuracy as measured by PSNR across the evaluated operators by 8.5 dB on the MIT-Adobe dataset (from 27.5 to 36 dB) and reduces DSSIM by a multiplicative factor of 3 compared to the most accurate prior approximation scheme, while being the fastest. We show that our models generalize across datasets and across resolutions, and investigate a number of extensions of the presented approach.Keywords
All Related Versions
This publication has 62 references indexed in Scilit:
- Domain transform for edge-aware image and video processingACM Transactions on Graphics, 2011
- Local Laplacian filtersACM Transactions on Graphics, 2011
- Edge-preserving multiscale image decomposition based on local extremaACM Transactions on Graphics, 2009
- Edge-avoiding wavelets and their applicationsACM Transactions on Graphics, 2009
- Single image dehazingACM Transactions on Graphics, 2008
- Edge-preserving decompositions for multi-scale tone and detail manipulationACM Transactions on Graphics, 2008
- Real-time edge-aware image processing with the bilateral gridACM Transactions on Graphics, 2007
- Joint bilateral upsamplingACM Transactions on Graphics, 2007
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- Fast bilateral filtering for the display of high-dynamic-range imagesACM Transactions on Graphics, 2002