Spectrogram Transformers for Audio Classification
Authors:
- 21 June 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Abstract
Audio classification is an important task in the machine learning field with a wide range of applications. Since the last decade, deep learning based methods have been widely used and the transformer-based models are becoming new paradigm for audio classification. In this paper, we present Spectrogram Transformers, which are a group of transformer-based models for audio classification. Based on the fundamental semantics of audio spectrogram, we design two mechanisms to extract temporal and frequency features from audio spectrogram, named time-dimension sampling and frequency-dimension sampling. These discriminative representations are then enhanced by various combinations of attention block architectures, including Tempo-ral Only (TO) attention, Temporal-Frequency sequential (TFS) attention, Temporal-Frequency Parallel (TFP) attention, and Two-stream Temporal-Frequency (TSTF) attention, to extract the sound record signatures to serve the classification task. Our experiments demonstrate that these Transformer models outper-form the state-of-the-art methods on ESC-50 dataset without pre-training stage. Furthermore, our method also shows great efficiency compared with other leading methods.Keywords
Funding Information
- China Scholarship Council
- Loughborough University
This publication has 18 references indexed in Scilit:
- A neural network constructed by deep learning technique and its application to intelligent fault diagnosis of machinesNeurocomputing, 2018
- Dilated convolution neural network with LeakyReLU for environmental sound classification2017
- Novel TEO-based Gammatone features for environmental sound classification2017
- CNN architectures for large-scale audio classification2017
- TUT database for acoustic scene classification and sound event detection2016
- Bird detection in audio: A survey and a challenge2016
- An overview of applications and advancements in automatic sound recognitionNeurocomputing, 2016
- ESC2015
- End-to-end learning for music audio2014
- Automatically extracting highlights for TV Baseball programs2000