Guided Motion Diffusion for Controllable Human Motion Synthesis
Authors:
Preprint
Abstract
Denoising diffusion models have shown great promise in human motion synthesis conditioned on natural language descriptions. However, integrating spatial constraints, such as pre-defined motion trajectories and obstacles, remains a challenge despite being essential for bridging the gap between isolated human motion and its surrounding environment. To address this issue, we propose Guided Motion Diffusion (GMD), a method that incorporates spatial constraints into the motion generation process. Specifically, we propose an effective feature projection scheme that manipulates motion representation to enhance the coherency between spatial information and local poses. Together with a new imputation formulation, the generated motion can reliably conform to spatial constraints such as global motion trajectories. Furthermore, given sparse spatial constraints (e.g. sparse keyframes), we introduce a new dense guidance approach to turn a sparse signal, which is susceptible to being ignored during the reverse steps, into denser signals to guide the generated motion to the given constraints. Our extensive experiments justify the development of GMD, which achieves a significant improvement over state-of-the-art methods in text-based motion generation while allowing control of the synthesized motions with spatial constraints.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Scalable Diffusion Models with Transformers2023
- PhysDiff: Physics-Guided Human Motion Diffusion Model2023
- UDE: A Unified Driving Engine for Human Motion Generation2023
- MoFusion: A Framework for Denoising-Diffusion-Based Motion Synthesis2023
- InstructPix2Pix: Learning to Follow Image Editing Instructions2023
- EDGE: Editable Dance Generation From Music2023
- Executing your Commands via Motion Diffusion in Latent Space2023
- High-Resolution Image Synthesis with Latent Diffusion Models2022
- SRDiff: Single image super-resolution with diffusion probabilistic modelsNeurocomputing, 2022
- ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models2021