Rebetiko Singer Identification: Fine-tuning and explaining deep pretrained transformer models
Open Access
- 18 September 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 285-291
Abstract
In Rebetiko music, the distinctive vocal qualities of individual singers, especially renowned singers, are often at the centre of attention, as their unique timbre and ornamentations play a crucial role in defining the genre’s characteristics. Pre-trained deep learning models on human voice data offer a good starting point for fine-tuning to classify small-sized voice-related datasets. While these fine-tuned models are "black boxes" that do not provide direct explanations for their classifications, their internal latent representations can be explored through visualizations. This study analyzes the performance and possibilities for explainability of pre-trained voice models fine-tuned for identifying Rebetiko singers using separated voice signals from original recordings. The selected singers exhibit diverse timbral and delivery styles, which can be numerically represented through perceptually relevant audio features (e.g., fundamental frequency, spectral centroid, vibrato rate and depth, inharmonicity, etc.). By comparing extracted audio features with visualizations from the "black box" model, valuable insights into the model’s learning process are gained. This study not only demonstrates the effectiveness of fine-tuning for Rebetiko singer identification but also enhances explainability, which is crucial for better understanding the characteristics and trends of Rebetiko singing.Keywords
Funding Information
- European Regional Development Fund of the European Union and Greek national funds (MIS 5149205)
This publication has 14 references indexed in Scilit:
- Hybrid Transformers for Music Source Separation2023
- Distilhubert: Speech Representation Learning by Layer-Wise Distillation of Hidden-Unit Bert2022
- HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden UnitsIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021
- The effects of explainability and causability on perception, trust, and acceptance: Implications for explainable AIInternational Journal of Human-Computer Studies, 2021
- Singer Identification from Smaller Snippets of Audio Clips Using Acoustic Features and DNNs2018
- A Formant Range Profile for SingersJournal of Voice, 2017
- Singer identification in Indian Hindi songs using MFCC and spectral features2016
- Classification of vocal and non-vocal regions from audio songs using spectral features and pitch variations2015
- Combining Evidences from Mel Cepstral Features and Cepstral Mean Subtracted Features for Singer Identification2012
- Perceptual aspects of singingJournal of Voice, 1994