Retargeting Visual Data with Deformation Fields

Authors:
  • TETim Elsner
    Tim Elsner
  • JBJulia Berger
    Julia Berger
  • TWTong Wu
    Tong Wu
  • VCVictor Czech
    Victor Czech
  • LGLin Gao
    Lin Gao
  • LKLeif Kobbelt
    Leif Kobbelt

Abstract
Seam carving is an image editing method that enable content-aware resizing, including operations like removing objects. However, the seam-finding strategy based on dynamic programming or graph-cut limits its applications to broader visual data formats and degrees of freedom for editing. Our observation is that describing the editing and retargeting of images more generally by a displacement field yields a generalisation of content-aware deformations. We propose to learn a deformation with a neural network that keeps the output plausible while trying to deform it only in places with low information content. This technique applies to different kinds of visual data, including images, 3D scenes given as neural radiance fields, or even polygon meshes. Experiments conducted on different visual data show that our method achieves better content-aware retargeting compared to previous methods.