DanceCamAnimator: Keyframe-Based Controllable 3D Dance Camera Synthesis

Authors:
  • ZWZixuan Wang
    Zixuan Wang
  • JLJiayi Li
    Jiayi Li
  • XQXiaoyu Qin
    Xiaoyu Qin
  • SSShikun Sun
    Shikun Sun
  • SZSongtao Zhou
    Songtao Zhou
  • JJJia Jia
    Jia Jia
  • JLJiebo Luo
    Jiebo Luo

Abstract
Synthesizing camera movements from music and dance is highly challenging due to the contradicting requirements and complexities of dance cinematography. Unlike human movements, which are always continuous, dance camera movements involve both continuous sequences of variable lengths and sudden drastic changes to simulate the switching of multiple cameras. However, in previous works, every camera frame is equally treated and this causes jittering and unavoidable smoothing in post-processing. To solve these problems, we propose to integrate animator dance cinematography knowledge by formulating this task as a three-stage process: keyframe detection, keyframe synthesis, and tween function prediction. Following this formulation, we design a novel end-to-end dance camera synthesis framework \textbf{DanceCamAnimator}, which imitates human animation procedures and shows powerful keyframe-based controllability with variable lengths. Extensive experiments on the DCM dataset demonstrate that our method surpasses previous baselines quantitatively and qualitatively. Code will be available at \url{https://github.com/Carmenw1203/DanceCamAnimator-Official}.
All Related Versions

This publication has 20 references indexed in Scilit: