Regularizing CNN Transfer Learning With Randomised Regression
Authors:
- 1 June 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 13634-13643
Abstract
This paper is about regularizing deep convolutional networks (CNNs) based on an adaptive framework for transfer learning with limited training data in the target domain. Recent advances of CNN regularization in this context are commonly due to the use of additional regularization objectives. They guide the training away from the target task using some forms of concrete tasks. Unlike those related approaches, we suggest that an objective without a concrete goal can still serve well as a regularizer. In particular, we demonstrate Pseudo-task Regularization (PtR) which dynamically regularizes a network by simply attempting to regress image representations to pseudo-regression targets during fine-tuning. That is, a CNN is efficiently regularized without additional resources of data or prior domain expertise. In sum, the proposed PtR provides: a) an alternative for network regularization without dependence on the design of concrete regularization objectives or extra annotations; b) a dynamically adjusted and maintained strength of regularization effect by balancing the gradient norms between objectives on-line. Through numerous experiments, surprisingly, the improvements on classification accuracy by PtR are shown greater or on a par to the recent state-of-the-art methods.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Borrowing Treasures from the Wealthy: Deep Transfer Learning through Selective Joint Fine-Tuning2017
- DisturbLabel: Regularizing CNN on the Loss Layer2016
- Deep Residual Learning for Image Recognition2016
- Deep Learning Face Attributes in the Wild2015
- PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization2015
- Simultaneous Deep Transfer Across Domains and Tasks2015
- Factors of Transferability for a Generic ConvNet RepresentationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- One millisecond face alignment with an ensemble of regression trees2014
- Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks2014
- Human action recognition by learning bases of action attributes and parts2011