Time Flies: Animating a Still Image With Time-Lapse Video As Reference
Authors:
- 1 June 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 5640-5649
Abstract
Time-lapse videos usually perform eye-catching appearances but are often hard to create. In this paper, we propose a self-supervised end-to-end model to generate the time-lapse video from a single image and a reference video. Our key idea is to extract both the style and the features of temporal variation from the reference video, and transfer them onto the input image. To ensure both the temporal consistency and realness of our resultant videos, we introduce several novel designs in our architecture, including classwise NoiseAdaIN, flow loss, and the video discriminator. In comparison to the baselines of state-of-the-art style transfer approaches, our proposed method is not only efficient in computation but also able to create more realistic and temporally smooth time-lapse video of a still image, with its temporal variation consistent to the reference.Keywords
This publication has 20 references indexed in Scilit:
- Multimodal Transfer: A Hierarchical Deep Convolutional Neural Network for Fast Artistic Style Transfer2017
- StyleBank: An Explicit Representation for Neural Image Style Transfer2017
- Real-Time Neural Style Transfer for Videos2017
- Improved Texture Networks: Maximizing Quality and Diversity in Feed-Forward Stylization and Texture Synthesis2017
- Deep Photo Style Transfer2017
- Image Style Transfer Using Convolutional Neural Networks2016
- Split and Match: Example-Based Adaptive Patch Sampling for Unsupervised Style Transfer2016
- Style transfer for headshot portraitsACM Transactions on Graphics, 2014
- Data-driven hallucination of different times of day from a single outdoor photoACM Transactions on Graphics, 2013
- Webcam clip artACM Transactions on Graphics, 2009