Contrastive Learning of General-Purpose Audio Representations
Authors:
Top Cited Papers
- 6 June 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3875-3879
Abstract
We introduce COLA, a self-supervised pre-training approach for learning a general-purpose representation of audio. Our approach is based on contrastive learning: it learns a representation which assigns high similarity to audio segments extracted from the same recording while assigning lower similarity to segments from different recordings. We build on top of recent advances in contrastive learning for computer vision and reinforcement learning to design a lightweight, easy-to-implement self-supervised model of audio. We pre-train embeddings on the large-scale Audioset database and transfer these representations to 9 diverse classification tasks, including speech, music, animal sounds, and acoustic scenes. We show that despite its simplicity, our method significantly outperforms previous self-supervised systems. We furthermore conduct ablation studies to identify key design choices and release a library 1 to pre-train and fine-tune COLA models.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Learning Problem-Agnostic Speech Representations from Multiple Self-Supervised Tasks2019
- Automatic acoustic detection of birds through deep learning: The first Bird Audio Detection challengeMethods in Ecology and Evolution, 2018
- Unsupervised Learning of Semantic Audio Representations2018
- Google’s Next-Generation Real-Time Unit-Selection Synthesizer Using Sequence-to-Sequence LSTM-Based Autoencoders2017
- VoxCeleb: A Large-Scale Speaker Identification Dataset2017
- Audio Set: An ontology and human-labeled dataset for audio events2017
- Audio Word2Vec: Unsupervised Learning of Audio Segment Representations Using Sequence-to-Sequence Autoencoder2016
- Audio enhancing with DNN autoencoder for speaker recognition2016
- Unsupervised Learning of Visual Representations Using Videos2015
- Librispeech: An ASR corpus based on public domain audio books2015