Implicit Label Augmentation on Partially Annotated Clips via Temporally-Adaptive Features Learning
Authors:
Preprint
Abstract
Partially annotated clips contain rich temporal contexts that can complement the sparse key frame annotations in providing supervision for model training. We present a novel paradigm called Temporally-Adaptive Features (TAF) learning that can utilize such data to learn better single frame models. By imposing distinct temporal change rate constraints on different factors in the model, TAF enables learning from unlabeled frames using context to enhance model accuracy. TAF generalizes "slow feature" learning and we present much stronger empirical evidence than prior works, showing convincing gains for the challenging semantic segmentation task over a variety of architecture designs and on two popular datasets. TAF can be interpreted as an implicit label augmentation method but is a more principled formulation compared to existing explicit augmentation techniques. Our work thus connects two promising methods that utilize partially annotated clips for single frame model training and can inspire future explorations in this direction.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Data Augmentation Using Learned Transformations for One-Shot Medical Image Segmentation2019
- Improving Semantic Segmentation via Video Propagation and Label Relaxation2019
- UPSNet: A Unified Panoptic Segmentation Network2019
- MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features2018
- Path Aggregation Network for Instance Segmentation2018
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric2018
- Learning Image Representations Tied to Egomotion from Unlabeled VideoInternational Journal of Computer Vision, 2017
- Shuffle and Learn: Unsupervised Learning Using Temporal Order Verification2016
- Perceptual Losses for Real-Time Style Transfer and Super-Resolution2016
- Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue2016