Learn2Dance: Learning Statistical Music-to-Dance Mappings for Choreography Synthesis
- 23 December 2011
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 14 (3) , 747-759
Abstract
We propose a novel framework for learning many-to-many statistical mappings from musical measures to dance figures towards generating plausible music-driven dance choreographies. We obtain music-to-dance mappings through use of four statistical models: 1) musical measure models, representing a many-to-one relation, each of which associates different melody patterns to a given dance figure via a hidden Markov model (HMM); 2) exchangeable figures model, which captures the diversity in a dance performance through a one-to-many relation, extracted by unsupervised clustering of musical measure segments based on melodic similarity; 3) figure transition model, which captures the intrinsic dependencies of dance figure sequences via an n-gram model; 4) dance figure models, capturing the variations in the way particular dance figures are performed, by modeling the motion trajectory of each dance figure via an HMM. Based on the first three of these statistical mappings, we define a discrete HMM and synthesize alternative dance figure sequences by employing a modified Viterbi algorithm. The motion parameters of the dance figures in the synthesized choreography are then computed using the dance figure models. Finally, the generated motion parameters are animated synchronously with the musical audio using a 3-D character model. Objective and subjective evaluation results demonstrate that the proposed framework is able to produce compelling music-driven choreographies.Keywords
This publication has 36 references indexed in Scilit:
- Multi-modal analysis of dance performances for music-driven choreography synthesis2010
- Perceptually motivated automatic dance motion generation for musicComputer Animation and Virtual Worlds, 2009
- A robust harmony structure modeling scheme for classical music opus identification2009
- An audio-driven dancing avatarJournal on Multimodal User Interfaces, 2008
- Multicamera Audio-Visual Analysis of Dance Figures2007
- Dancing‐to‐Music Character AnimationComputer Graphics Forum, 2006
- Silhouettes: A graphical aid to the interpretation and validation of cluster analysisJournal of Computational and Applied Mathematics, 1987
- Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentencesIEEE Transactions on Acoustics, Speech, and Signal Processing, 1980
- Dynamic programming algorithm optimization for spoken word recognitionIEEE Transactions on Acoustics, Speech, and Signal Processing, 1978
- Circularity in Judgments of Relative PitchThe Journal of the Acoustical Society of America, 1964