Guided Generative Adversarial Neural Network for Representation Learning and Audio Generation Using Fewer Labelled Audio Data
Open Access
- 21 July 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 29 (23299290) , 2575-2590
Abstract
The Generation power of Generative Adversarial Neural Networks (GANs) has shown great promise to learn representations from unlabelled data while guided by a small amount of labelled data. We aim to utilise the generation power of GANs to learn Audio Representations. Most existing studies are, however, focused on images. Some studies use GANs for speech generation, but they are conditioned on text or acoustic features, limiting their use for other audio, such as instruments, and even for speech where transcripts are limited. This paper proposes a novel GAN-based model that we named Guided Generative Adversarial Neural Network (GGAN), which can learn powerful representations and generate good-quality samples using a small amount of labelled data as guidance. Experimental results based on a speech [Speech Command Dataset (S09)] and a non-speech [Musical Instrument Sound dataset (Nsyth)] dataset demonstrate that using only 5% of labelled data as guidance, GGAN learns significantly better representations than the state-of-the-art models.Keywords
All Related Versions
This publication has 38 references indexed in Scilit:
- Adversarial Network Bottleneck Features for Noise Robust Speaker Verification2017
- Improved Audio Scene Classification Based on Label-Tree Embeddings and Convolutional Neural NetworksIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2017
- Unsupervised Feature Learning Based on Deep Models for Environmental Audio TaggingIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2017
- Learning representations of emotional speech with deep convolutional generative adversarial networks2017
- Learning Representations for Nonspeech Audio Events Through Their Similarities to Speech PatternsIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2016
- Deep Learning Face Attributes in the Wild2015
- Going deeper with convolutions2015
- Librispeech: An ASR corpus based on public domain audio books2015
- Signal estimation from modified short-time Fourier transformIEEE Transactions on Acoustics, Speech, and Signal Processing, 1984
- The Fréchet distance between multivariate normal distributionsJournal of Multivariate Analysis, 1982