QCA-Net: Quantum-based Channel Attention for Deep Neural Networks
Authors:
- 18 June 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-7
Abstract
The channel attention mechanism, which adaptively recalibrates each channel's weight, can enhance the performance of deep neural networks. Most channel attention modules use simple pooling operations to aggregate spatial information. The drawback is the incapability to express complex global spatial information effectively. In this paper, we propose a Quantum-based Channel Attention (QCA), which only involves a handful of parameters but brings apparent performance gain. Using quantum mechanics analogy, we utilize wave functions describing microscopic particles to generate complex global spatial information. In addition, the QCA module has no convolutional layer, making it suitable for integration with various network architectures, including transformer and multilayer perceptron (MLP). We evaluate QCA through experiments on ImageNet-1K, and we also demonstrated the effect of QCA in combination with pre-training networks on small downstream transfer learning tasks.Keywords
This publication has 16 references indexed in Scilit:
- Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based LocalizationInternational Journal of Computer Vision, 2019
- Global Second-Order Pooling Convolutional Networks2019
- Do Better ImageNet Models Transfer Better?2019
- Squeeze-and-Excitation Networks2018
- Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning2018
- A Comprehensive Survey on Domain Adaptation for Visual Applications2017
- SCA-CNN: Spatial and Channel-Wise Attention in Convolutional Networks for Image Captioning2017
- Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation2014
- A Survey on Transfer LearningIEEE Transactions on Knowledge and Data Engineering, 2009
- Introduction to Quantum MechanicsAmerican Journal of Physics, 1995