CoMo: Controllable Motion Generation through Language Guided Pose Code Editing
Authors:
Preprint
Abstract
Text-to-motion models excel at efficient human motion generation, but existing approaches lack fine-grained controllability over the generation process. Consequently, modifying subtle postures within a motion or inserting new actions at specific moments remains a challenge, limiting the applicability of these methods in diverse scenarios. In light of these challenges, we introduce CoMo, a Controllable Motion generation model, adept at accurately generating and editing motions by leveraging the knowledge priors of large language models (LLMs). Specifically, CoMo decomposes motions into discrete and semantically meaningful pose codes, with each code encapsulating the semantics of a body part, representing elementary information such as "left knee slightly bent". Given textual inputs, CoMo autoregressively generates sequences of pose codes, which are then decoded into 3D motions. Leveraging pose codes as interpretable representations, an LLM can directly intervene in motion editing by adjusting the pose codes according to editing instructions. Experiments demonstrate that CoMo achieves competitive performance in motion generation compared to state-of-the-art models while, in human studies, CoMo substantially surpasses previous work in motion editing abilities.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- TLControl: Trajectory and Language Control for Human Motion Synthesis2024
- EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generation2024
- PoseFix: Correcting 3D Human Poses with Natural Language2023
- ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model2023
- PhysDiff: Physics-Guided Human Motion Diffusion Model2023
- SINC: Spatial Composition of 3D Human Motions for Simultaneous Action Generation2023
- PoseScript: 3D Human Poses from Natural Language2022
- COUCH: Towards Controllable Human-Chair Interactions2022
- Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory2022
- The KIT Motion-Language DatasetBig Data, 2016