CVSM: Contrastive Vocal Similarity Modeling
Preprint
Abstract
The availability of large, unlabeled datasets across various domains has contributed to the development of a plethora of methods that learn representations for multiple target (downstream) tasks through self-supervised pre-training. In this work, we introduce CVSM (Contrastive Vocal Similarity Modeling), a contrastive self-supervised procedure for music signal representation learning in the audio domain that can be utilized for musical and vocal similarity modeling. Our method operates under a contrastive framework, maximizing the similarity between vocal excerpts and musical mixtures containing the same vocals; we devise both a label-informed protocol, leveraging artist identity information to sample the contrastive pairs, and a label-agnostic scheme, involving artificial mixture creation from randomly sampled vocal and accompaniment excerpts, which are paired with vocals from the same audio segment. We evaluate our proposed method in measuring vocal similarity both objectively, through linear probing on a suite of appropriate downstream tasks, and subjectively, via conducting a user study consisting of pairwise comparisons between different models in a recommendation-by-query setting. Our results indicate that the representations learned through CVSM are effective in musical and vocal similarity modeling, outperforming numerous baselines across both isolated vocals and complete musical mixtures. Moreover, while the availability of artist identity labels during pre-training leads to overall more consistent performance both in the evaluated downstream tasks and the user study, a label-agnostic CVSM variant incorporating hybrid pre-training with real and artificial mixtures achieves comparable performance to the label-informed one in artist identification and perceived vocal similarity.Keywords
All Related Versions
This publication has 26 references indexed in Scilit:
- Evaluating Contrastive Methodologies for Music Representation Learning Using Playlist Data2025
- On The Effect Of Data-Augmentation On Local Embedding Properties In The Contrastive Learning Of Music Audio Representations2024
- Content-driven music recommendation: Evolution, state of the art, and challengesComputer Science Review, 2024
- Toward Leveraging Pre-Trained Self-Supervised Frontends for Automatic Singing Voice Understanding Tasks: Three Case Studies2023
- Pre-Training Strategies Using Contrastive Learning and Playlist Information for Music Classification and Similarity2023
- Towards Proper Contrastive Self-Supervised Learning Strategies for Music Audio Representation2022
- Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization2018
- Improving music source separation based on deep neural networks through data augmentation and network blending2017
- Communication of emotions in vocal expression and music performance: Different channels, same code?Psychological Bulletin, 2003
- Silhouettes: A graphical aid to the interpretation and validation of cluster analysisJournal of Computational and Applied Mathematics, 1987