Deep Quaternion Networks
Authors:
- 1 July 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
Abstract
The field of deep learning has seen significant advancement in recent years. However, much of the existing work has been focused on real-valued numbers. Recent work has shown that a deep learning system using the complex numbers can be deeper for a fixed parameter budget compared to its real-valued counterpart. In this work, we explore the benefits of generalizing one step further into the hyper-complex numbers, quaternions specifically, and provide the architecture components needed to build deep quaternion networks. We develop the theoretical basis by reviewing quaternion convolutions, developing a novel quaternion weight initialization scheme, and developing novel algorithms for quaternion batch-normalization. These pieces are tested in a classification model by end-to-end training on the CIFAR -10 and CIFAR -100 data sets and a segmentation model by end-to-end training on the KITTI Road Segmentation data set. These quaternion networks show improved convergence compared to real-valued and complex-valued networks, especially on the segmentation task, while having fewer parameters.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- Feed forward neural network with random quaternionic neuronsSignal Processing, 2017
- Quaternion Neural Networks for Spoken Language Understanding2017
- Deep Residual Learning for Image Recognition2016
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification2015
- PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization2015
- A new performance measure and evaluation benchmark for road detection algorithms2013
- Quaternion color texture segmentationComputer Vision and Image Understanding, 2007
- Hypercomplex signals-a novel extension of the analytic signal to the multidimensional caseIEEE Transactions on Signal Processing, 2001
- Colour image filters based on hypercomplex convolutionIEE Proceedings - Vision, Image, and Signal Processing, 2000
- The importance of phase in signalsProceedings of the IEEE, 1981