DanceCamAnimator: Keyframe-Based Controllable 3D Dance Camera Synthesis
Authors:
- Zixuan WangZWZixuan Wang
- Jiayi LiJLJiayi Li
- Xiaoyu QinXQXiaoyu Qin
- Shikun SunSSShikun Sun
- Songtao ZhouSZSongtao Zhou
- Jia JiaJJJia Jia
- Jiebo LuoJLJiebo Luo
Preprint
Abstract
Synthesizing camera movements from music and dance is highly challenging due to the contradicting requirements and complexities of dance cinematography. Unlike human movements, which are always continuous, dance camera movements involve both continuous sequences of variable lengths and sudden drastic changes to simulate the switching of multiple cameras. However, in previous works, every camera frame is equally treated and this causes jittering and unavoidable smoothing in post-processing. To solve these problems, we propose to integrate animator dance cinematography knowledge by formulating this task as a three-stage process: keyframe detection, keyframe synthesis, and tween function prediction. Following this formulation, we design a novel end-to-end dance camera synthesis framework \textbf{DanceCamAnimator}, which imitates human animation procedures and shows powerful keyframe-based controllability with variable lengths. Extensive experiments on the DCM dataset demonstrate that our method surpasses previous baselines quantitatively and qualitatively. Code will be available at \url{https://github.com/Carmenw1203/DanceCamAnimator-Official}.Keywords
All Related Versions
- Version 1, 2024-09-23, ArXiv
- Published version: , 10200.
This publication has 20 references indexed in Scilit:
- Divide and Fuse: Body Part Mesh Recovery from Partially Visible Human Images2024
- Cinematic Behavior Transfer via NeRF-based Differentiable Filming2024
- Camera Motion Generation Method Based on Performer's Position for Performance Filming2023
- DeepDance: Music-to-Dance Motion Choreography With Adversarial LearningIEEE Transactions on Multimedia, 2020
- Monocular human pose estimation: A survey of deep learning-based methodsComputer Vision and Image Understanding, 2020
- Can a Robot Become a Movie Director? Learning Artistic Principles for Aerial Cinematography2019
- Learning to Capture a Film-Look Video with a Camera Drone2019
- An automatic system for humanoid dance creationBiologically Inspired Cognitive Architectures, 2016
- Music similarity-based approach to generating dance motion sequenceMultimedia Tools and Applications, 2012
- Learn2Dance: Learning Statistical Music-to-Dance Mappings for Choreography SynthesisIEEE Transactions on Multimedia, 2011