Musical Word Embedding for Music Tagging and Retrieval
- 6 June 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 33, 2377-2387
Abstract
Word embedding has become an essential means for text-based information retrieval. Typically, word embeddings are learned from large quantities of general and unstructured text data. However, in the domain of music, word embedding may have difficulty understanding musical contexts or recognizing music-related entities like artists and tracks. To address this issue, we propose a new approach called Musical Word Embedding (MWE), which involves learning from various types of texts, including both everyday and music-related vocabulary. We integrate MWE into an audio-word joint representation framework for tagging and retrieving music, using words like tag, artist, and track that have different levels of musical specificity. Through extensive experiments, we demonstrate that the effectiveness of musical supervision varies by task - specifically, tag-level supervision improves tagging performance while track-level supervision enhances retrieval performance. This finding suggests that the choice of musical supervision in representation learning needs to be carefully considered based on the target task. To balance this compromise, we suggest multi-prototype training that uses words with different levels of musical specificity jointly. We evaluate both word embedding and audio-word joint embedding on four tasks (tag rank prediction, music tagging, query-by-tag, and query-by-track) across two datasets (Million Song Dataset and MTG-Jamendo). The results show that the suggested MWE is more efficient and effective for both in-domain and out-of-domain datasets compared to conventional word embedding.Keywords
All Related Versions
Funding Information
- Ministry of Education
- National Research Foundation of Korea (2024S1A5C3A03046168)
This publication has 13 references indexed in Scilit:
- Toward Universal Text-To-Music Retrieval2023
- Multimodal Metric Learning for Tag-Based Music Retrieval2021
- Disentangled Multidimensional Metric Learning for Music Similarity2020
- Data-Driven Harmonic Filters for Audio Representation Learning2020
- Multi-Task Music Representation Learning from Multi-Label Embeddings2019
- BioWordVec, improving biomedical word embeddings with subword information and MeSHScientific Data, 2019
- Deep Learning for Audio-Based Music Classification and Tagging: Teaching Computers to Distinguish Rock from BachIEEE Signal Processing Magazine, 2018
- Zero-Shot Learning — The Good, the Bad and the Ugly2017
- Ups and Downs2016
- Semantic Annotation and Retrieval of Music and Sound EffectsIEEE Transactions on Audio, Speech, and Language Processing, 2008