BeatDance: A Beat-Based Model-Agnostic Contrastive Learning Framework for Music-Dance Retrieval
Open Access
- 30 May 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 11-19
Abstract
Dance and music are closely related forms of expression, with mutual retrieval between dance videos and music being a fundamental task in various fields like education, art, and sports. However, existing methods often suffer from unnatural generation effects or fail to fully explore the correlation between music and dance. To overcome these challenges, we propose BeatDance, a novel beat-based model-agnostic contrastive learning framework. BeatDance incorporates a Beat-Aware Music-Dance InfoExtractor, a Trans-Temporal Beat Blender, and a Beat-Enhanced Hubness Reducer to improve Music-Dance retrieval performance by utilizing the alignment between music beats and dance movements. We also introduce the Music-Dance (M-D) dataset, a large-scale collection of over 10,000 Music-Dance video pairs for training and testing. Experimental results on the M-D dataset demonstrate the superiority of our method over existing baselines, achieving state-of-the-art performance. The code and dataset are available at https://github.com/XulongT/BeatDance.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62072459, 62172421)
This publication has 28 references indexed in Scilit:
- Language-Guided Music Recommendation for Video via Prompt Analogies2023
- Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory2022
- X-Pool: Cross-Modal Language-Video Attention for Text-Video Retrieval2022
- A Brand New Dance Partner: Music-Conditioned Pluralistic Dancing Controlled by Multiple Dance Genres2022
- Cross Modal Retrieval with Querybank Normalisation2022
- It's Time for Artistic Correspondence in Music and Video2022
- Learning to dance: A graph convolutional adversarial network to generate realistic dance motions from audioComputers & Graphics, 2021
- Deep learning-based late fusion of multimodal information for emotion classification of music videoMultimedia Tools and Applications, 2020
- Query-by-Dancing: A Dance Music Retrieval System Based on Body-Motion Similarity2018
- Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields2017