Exploring complementary information of self‐supervised pretext tasks for unsupervised video pre‐training
Authors:
Open Access
- 28 November 2021
- journal article
- research article
- Published by Institution of Engineering and Technology (IET) in IET Computer Vision
- Vol. 16 (3) , 255-265
Abstract
This study addresses the problem of the unsupervised pre‐training of video representation learning. The authors' focus is on two common approaches: knowledge distillation and self‐supervised learning. The insight provided is that knowledge distillation and the rapidly advancing self‐supervised technique can be mutually beneficial. Combining these two approaches, a unified framework of self‐supervised learning and image‐based distillation (SSID) for unsupervised video pre‐training is proposed. The effectiveness of SSID in comparison to both image‐based distillation methods and the existing self‐supervised pre‐training baseline is demonstrated. In particular, the authors' model leverages three signals from the unlabelled data. First, the authors distil from the classifier of a 2D pre‐trained model as a soft label. To regularize the training process, the authors then build a novel positive pair of contrastive learning on the representation of the 2D/3D model. Finally, a self‐supervised pretext task is introduced to enhance the authors' model to become aware of the temporal evolution. The authors' experiment results showed that the learnt features achieved the best performance when transferred to action recognition tasks on UCF101 and HMDB51, reaching increases of 2.4% and 1.9% compared to the existing unsupervised pre‐training model, respectively.Keywords
This publication has 41 references indexed in Scilit:
- Unsupervised Representation Learning by Sorting Sequences2017
- Places: A 10 Million Image Database for Scene RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset2017
- Shuffle and Learn: Unsupervised Learning Using Temporal Order Verification2016
- Temporal Segment Networks: Towards Good Practices for Deep Action Recognition2016
- Deep Residual Learning for Image Recognition2016
- Learning to Track for Spatio-Temporal Action Localization2015
- Learning Spatiotemporal Features with 3D Convolutional Networks2015
- Large-Scale Video Classification with Convolutional Neural Networks2014
- ImageNet: A large-scale hierarchical image database2009