AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolation
Authors:
Top Cited Papers
- 1 June 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 9801-9810
Abstract
We present All-Pairs Multi-Field Transforms (AMT), a new network architecture for video frame interpolation. It is based on two essential designs. First, we build bidirectional correlation volumes for all pairs of pixels, and use the predicted bilateral flows to retrieve correlations for updating both flows and the interpolated content feature. Second, we derive multiple groups of fine-grained flow fields from one pair of updated coarse flows for performing backward warping on the input frames separately. Combining these two designs enables us to generate promising task-oriented flows and reduce the difficulties in modeling large motions and handling occluded areas during frame interpolation. These qualities promote our model to achieve state-of-the-art performance on various benchmarks with high efficiency. Moreover, our convolution-based model competes favorably compared to Transformer-based models in terms of accuracy and efficiency. Our code is available at https://github.com/MCG-NKU/AMT.Keywords
Funding Information
- NSFC (62176130)
- Fundamental Research Funds for the Central Universities (63223050)
- China Postdoctoral Science Foundation (2021M701780)
This publication has 58 references indexed in Scilit:
- End-to-End Learning of Geometry and Context for Deep Stereo Regression2017
- Video Frame Interpolation via Adaptive Convolution2017
- View Synthesis by Appearance Flow2016
- Deep Stereo: Learning to Predict New Views from the World's Imagery2016
- Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network2016
- Learning Spatiotemporal Features with 3D Convolutional Networks2015
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Fast Cost-Volume Filtering for Visual Correspondence and BeyondIEEE Transactions on Pattern Analysis and Machine Intelligence, 2012
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- Two deterministic half-quadratic regularization algorithms for computed imaging2002