Deep Multi-View Learning with Stochastic Decorrelation Loss
Preprint
Abstract
Multi-view learning aims to learn an embedding space where multiple views are either maximally correlated for cross-view recognition, or decorrelated for latent factor disentanglement. A key challenge for deep multi-view representation learning is scalability. To correlate or decorrelate multi-view signals, the covariance of the whole training set should be computed which does not fit well with the mini-batch based training strategy, and moreover (de)correlation should be done in a way that is free of SVD-based computation in order to scale to contemporary layer sizes. In this work, a unified approach is proposed for efficient and scalable deep multi-view learning. Specifically, a mini-batch based Stochastic Decorrelation Loss (SDL) is proposed which can be applied to any network layer to provide soft decorrelation of the layer's activations. This reveals the connection between deep multi-view learning models such as Deep Canonical Correlation Analysis (DCCA) and Factorisation Autoencoder (FAE), and allows them to be easily implemented. We further show that SDL is superior to other decorrelation losses in terms of efficacy and scalability.Keywords
All Related Versions
This publication has 21 references indexed in Scilit:
- Perceptual Losses for Real-Time Style Transfer and Super-Resolution2016
- Embedding Deep Metric for Person Re-identification: A Study Against Large Variations2016
- Gated Siamese Convolutional Neural Network Architecture for Human Re-identification2016
- A Siamese Long Short-Term Memory Architecture for Human Re-identification2016
- Learning Deep Feature Representations with Domain Guided Dropout for Person Re-identification2016
- Stochastic optimization for deep CCA via nonlinear orthogonal iterations2015
- Going deeper with convolutions2015
- An improved deep learning architecture for person re-identification2015
- DeepReID: Deep Filter Pairing Neural Network for Person Re-identification2014
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998