DanceCamAnimator: Keyframe-Based Controllable 3D Dance Camera Synthesis
Open Access
- 28 October 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 10200-10209
Abstract
Synthesizing camera movements from music and dance is highly challenging due to the contradicting requirements and complexities of dance cinematography. Unlike human movements, which are always continuous, dance camera movements involve both continuous sequences of variable lengths and sudden drastic changes to simulate the switching of multiple cameras. However, in previous works, every camera frame is equally treated and this causes jittering and unavoidable smoothing in post-processing. To solve these problems, we propose to integrate animator dance cinematography knowledge by formulating this task as a three-stage process: keyframe detection, keyframe synthesis, and tween function prediction. Following this formulation, we design a novel end-to-end dance camera synthesis framework DanceCamAnimator, which imitates human animation procedures and shows powerful keyframe-based controllability with variable lengths. Extensive experiments on the DCM dataset demonstrate that our method surpasses previous baselines quantitatively and qualitatively. Code will be available at https://github.com/Carmenw1203/DanceCamAnimator-Official.Keywords
All Related Versions
Funding Information
- National Key R&D Program of China (2024QY1400)
This publication has 42 references indexed in Scilit:
- Music-Driven Group Choreography2023
- EDGE: Editable Dance Generation From Music2023
- Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory2022
- A Brand New Dance Partner: Music-Conditioned Pluralistic Dancing Controlled by Multiple Dance Genres2022
- CamerAI: Chase Camera in a Dense Environment using a Proximal Policy Optimization-trained Neural Network2021
- Learning to Film From Professional Human Motion Videos2019
- An automatic system for humanoid dance creationBiologically Inspired Cognitive Architectures, 2016
- Real-Time Camera Planning for Navigation in Virtual Environments2008
- A Constraint-Based Approach to Camera Path PlanningLecture Notes in Computer Science, 2003
- Modeling Camera Control with Constrained Hypertubes2002