Student Becoming the Master: Knowledge Amalgamation for Joint Scene Parsing, Depth Estimation, and More
Authors:
- 1 June 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2824-2833
Abstract
In this paper, we investigate a novel deep-model reusing task. Our goal is to train a lightweight and versatile student model, without human-labelled annotations, that amalgamates the knowledge and masters the expertise of two pre-trained teacher models working on heterogeneous problems, one on scene parsing and the other on depth estimation. To this end, we propose an innovative training strategy that learns the parameters of the student intertwined with the teachers, achieved by ``projecting'' its amalgamated features onto each teacher's domain and computing the loss. We also introduce two options to generalize the proposed training strategy to handle three or more tasks simultaneously. The proposed scheme yields very encouraging results. As demonstrated on several benchmarks, the trained student model achieves results even superior to those of the teachers in their own expertise domains and on par with the state-of-the-art fully supervised models relying on human-labelled annotations.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- Pyramid Scene Parsing Network2017
- SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image SegmentationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Joint Semantic Segmentation and Depth Estimation with Deep Convolutional Networks2016
- Deeper Depth Prediction with Fully Convolutional Residual Networks2016
- The Cityscapes Dataset for Semantic Urban Scene Understanding2016
- Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture2015
- Deep convolutional neural fields for depth estimation from a single image2015
- Fully convolutional networks for semantic segmentation2015
- Space-Time Joint Multi-layer Segmentation and Depth Estimation2012
- High-accuracy stereo depth maps using structured light2003