Generating the Future with Adversarial Transformers
Authors:
- Carl VondrickCVCarl Vondrick
- Antonio TorralbaATAntonio Torralba
Open Access
- 1 July 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 10636919,p. 2992-3000
Abstract
We learn models to generate the immediate future in video. This problem has two main challenges. Firstly, since the future is uncertain, models should be multi-modal, which can be difficult to learn. Secondly, since the future is similar to the past, models store low-level details, which complicates learning of high-level semantics. We propose a framework to tackle both of these challenges. We present a model that generates the future by transforming pixels in the past. Our approach explicitly disentangles the models memory from the prediction, which helps the model learn desirable invariances. Experiments suggest that this model can generate short videos of plausible futures. We believe predictive models have many applications in robotics, health-care, and video understanding.Keywords
This publication has 22 references indexed in Scilit:
- Shuffle and Learn: Unsupervised Learning Using Temporal Order Verification2016
- Learning Temporal Transformations from Time-Lapse Videos2016
- An Uncertain Future: Forecasting from Static Images Using Variational Autoencoders2016
- Context Encoders: Feature Learning by Inpainting2016
- Actions ~ Transformations2016
- Learning Temporal Embeddings for Complex Video Analysis2015
- Dense Optical Flow Prediction from a Static Image2015
- Patch to the Future: Unsupervised Visual Prediction2014
- Watching Unlabeled Video Helps Learn New Human Actions from Very Few Labeled Snapshots2013
- Deep learning from temporal coherence in video2009