The Speed Improvement by Merging Batch Normalization into Previously Linear Layer in CNN
Authors:
- 1 July 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 67-72
Abstract
With the development of deep learning, the convolutional neural network is developing more deeply, which undoubtedly has considerable requirements for the calculative ability and storage capacity of the application environment. In order to get a better use of convolutional neural networks in mobile phones, embedded platforms and other platforms with limited calculative ability, this paper proposes to accelerate the neural network by integrating the Batch Normalization and previously Linear layers and analyzes the feasibility theoretically. The experimental results on the CPU, GPU and Raspberry Pi validate the effectiveness of this method. The experiments using Caffe framework show that the merging of Batch Normalization and previously linear layers can increase the speed of the neural network by 30% to 50%.Keywords
This publication has 4 references indexed in Scilit:
- Distributed Training of Deep Neural Networks: Theoretical and Practical Limits of Parallel Scalability2016
- A Validation of DRAM RAPL Power Measurements2016
- Anatomy of high-performance matrix multiplicationACM Transactions on Mathematical Software, 2008
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998