Decoupled Dynamic Filter Networks
Authors:
- 1 June 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 6643-6652
Abstract
Convolution is one of the basic building blocks of CNN architectures. Despite its common use, standard convolution has two main shortcomings: Content-agnostic and Computation-heavy. Dynamic filters are content-adaptive, while further increasing the computational overhead. Depth-wise convolution is a lightweight variant, but it usually leads to a drop in CNN performance or requires a larger number of channels. In this work, we propose the Decoupled Dynamic Filter (DDF) that can simultaneously tackle both of these shortcomings. Inspired by recent advances in attention, DDF decouples a depth-wise dynamic filter into spatial and channel dynamic filters. This decomposition considerably reduces the number of parameters and limits computational costs to the same level as depth-wise convolution. Meanwhile, we observe a significant boost in performance when replacing standard convolution with DDF in classification networks. ResNet50 / 101 get improved by 1.9% and 1.3% on the top-1 accuracy, while their computational costs are reduced by nearly half. Experiments on the detection and joint upsampling networks also demonstrate the superior performance of the DDF upsampling variant (DDF-Up) in comparison with standard convolution and specialized content-adaptive layers. The project page with code is available 1 .Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China
This publication has 33 references indexed in Scilit:
- Superpixel Convolutional Networks Using Bilateral Inceptions2016
- Learning Sparse High Dimensional Filters: Image Filtering, Dense CRFs and Bilateral Neural Networks2016
- Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network2016
- Learning Deconvolution Network for Semantic Segmentation2015
- Robust image filtering using joint static and dynamic guidance2015
- Texture-Consistent Shadow Removal2008
- Joint bilateral upsamplingACM Transactions on Graphics, 2007
- Control of goal-directed and stimulus-driven attention in the brainNature Reviews Neuroscience, 2002
- The Dynamic Representation of ScenesVisual Cognition, 2000
- A model of saliency-based visual attention for rapid scene analysisIEEE Transactions on Pattern Analysis and Machine Intelligence, 1998