Squeeze-and-Excitation Networks
Top Cited Papers
- 29 April 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 42 (8) , 2011-2023
Abstract
The central building block of convolutional neural networks (CNNs) is the convolution operator, which enables networks to construct informative features by fusing both spatial and channel-wise information within local receptive fields at each layer. A broad range of prior research has investigated the spatial component of this relationship, seeking to strengthen the representational power of a CNN by enhancing the quality of spatial encodings throughout its feature hierarchy. In this work, we focus instead on the channel relationship and propose a novel architectural unit, which we term the "Squeeze-and-Excitation" (SE) block, that adaptively recalibrates channel-wise feature responses by explicitly modelling interdependencies between channels. We show that these blocks can be stacked together to form SENet architectures that generalise extremely effectively across different datasets. We further demonstrate that SE blocks bring significant improvements in performance for existing state-of-the-art CNNs at minimal additional computational cost. Squeeze-and-Excitation Networks formed the foundation of our ILSVRC 2017 classification submission which won first place and reduced the top-5 error to 2.251%, surpassing the winning entry of 2016 by a relative improvement of ∼25%. Models and code are available at https://github.com/hujie-frank/SENet.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (61632003, 61620106003, 61672502, 61571439)
- National Key R&D Program of China (2017YFB1002701)
- Macao FDCT (068/2015/A2)
- EPSRC AIMS (CDT EP/L015897/1)
This publication has 34 references indexed in Scilit:
- Going deeper with convolutions2015
- Fully convolutional networks for semantic segmentation2015
- Multi-Level Discriminative Dictionary Learning With Application to Large Scale Image ClassificationIEEE Transactions on Image Processing, 2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- DeepPose: Human Pose Estimation via Deep Neural Networks2014
- Speeding up Convolutional Neural Networks with Low Rank Expansions2014
- Image Classification with the Fisher Vector: Theory and PracticeInternational Journal of Computer Vision, 2013
- Evolving Memory Cell Structures for Sequence Learning2009
- Evolving Neural Networks through Augmenting TopologiesEvolutionary Computation, 2002
- A model of saliency-based visual attention for rapid scene analysisIEEE Transactions on Pattern Analysis and Machine Intelligence, 1998