Pretext Tasks Selection for Multitask Self-Supervised Audio Representation Learning
- 1 August 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Signal Processing
- Vol. 16 (6) , 1439-1453
Abstract
Through solving pretext tasks, self-supervised learning leverages unlabeled data to extract useful latent representations replacing traditional input features in the downstream task. In audio and speech signal processing, a wide range of features were engineered through decades of research efforts. As it turns out, learning to predict such features has proven to be a particularly relevant pretext task, leading to useful self-supervised representations which prove to be effective for downstream tasks. However, methods and common practices for combining such pretext tasks for better performance on the downstream task have not been explored and understood properly. In fact, the process relies almost exclusively on a computationally heavy experimental procedure, which becomes intractable with the increase of the number of pretext tasks. This paper introduces a method to select a group of pretext tasks among a set of candidates. The method we propose estimates calibrated weights for the partial losses corresponding to the considered pretext tasks during the self-supervised training process. The experiments conducted on automatic speech recognition, speaker and emotion recognition and instrument classification validate our approach as the groups selected and weighted with our method perform better than classic baselines, thus facilitating the selection and combination of relevant pretext-task labels for self-supervised representation learning.Keywords
All Related Versions
This publication has 66 references indexed in Scilit:
- Unsupervised Visual Representation Learning by Context Prediction2015
- Time delay deep neural network-based universal background models for speaker recognition2015
- Long-Term Statistical Feature Extraction from Speech Signal and Its Application in Emotion Recognition2015
- Librispeech: An ASR corpus based on public domain audio books2015
- Connectionist Temporal Classification2012
- IEMOCAP: interactive emotional dyadic motion capture databaseLanguage Resources and Evaluation, 2008
- Effects of vocal loudness variation on spectrum balance as reflected by the alpha measure of long-term-average spectra of speechThe Journal of the Acoustical Society of America, 2006
- Probabilistic Linear Discriminant Analysis2006
- Model Selection and Estimation in Regression with Grouped VariablesJournal of the Royal Statistical Society Series B: Statistical Methodology, 2005
- Cepstrum-Based Harmonics-to-Noise Ratio Measurement in Voiced SpeechLecture Notes in Computer Science, 2005