Investigating the Effects of Training Set Synthesis for Audio Segmentation of Radio Broadcast
Open Access
- 31 March 2021
- journal article
- research article
- Published by MDPI AG in Electronics
- Vol. 10 (7) , 827
Abstract
Music and speech detection provides us valuable information regarding the nature of content in broadcast audio. It helps detect acoustic regions that contain speech, voice over music, only music, or silence. In recent years, there have been developments in machine learning algorithms to accomplish this task. However, broadcast audio is generally well-mixed and copyrighted, which makes it challenging to share across research groups. In this study, we address the challenges encountered in automatically synthesising data that resembles a radio broadcast. Firstly, we compare state-of-the-art neural network architectures such as CNN, GRU, LSTM, TCN, and CRNN. Later, we investigate how audio ducking of background music impacts the precision and recall of the machine learning algorithm. Thirdly, we examine how the quantity of synthetic training data impacts the results. Finally, we evaluate the effectiveness of synthesised, real-world, and combined approaches for training models, to understand if the synthetic data presents any additional value. Amongst the network architectures, CRNN was the best performing network. Results also show that the minimum level of audio ducking preferred by the machine learning algorithm was similar to that of human listeners. After testing our model on in-house and public datasets, we observe that our proposed synthesis technique outperforms real-world data in some cases and serves as a promising alternative.Keywords
Funding Information
- Engineering and Physical Sciences Research Council (EP/S026991/1)
This publication has 25 references indexed in Scilit:
- Multiclass audio segmentation based on recurrent neural networks for broadcast domain dataEURASIP Journal on Audio, Speech, and Music Processing, 2020
- A CNN-Assisted Enhanced Audio Signal Processing for Speech Emotion RecognitionSensors, 2019
- Open Broadcast Media Audio from TV: A Dataset of TV Broadcast Audio with Relative Music Loudness AnnotationsTransactions of the International Society for Music Information Retrieval, 2019
- Convolutional Recurrent Neural Networks for Polyphonic Sound Event DetectionIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2017
- Metrics for Polyphonic Sound Event DetectionApplied Sciences, 2016
- An Overview of Automatic Audio SegmentationInternational Journal of Information Technology and Computer Science, 2014
- Audio segmentation of broadcast news in the Albayzin-2010 evaluation: overview, results, and discussionEURASIP Journal on Audio, Speech, and Music Processing, 2011
- Advances in unsupervised audio classification and segmentation for the broadcast news and NGSW corporaIEEE Transactions on Audio, Speech, and Language Processing, 2006
- Musical genre classification of audio signalsIEEE Transactions on Speech and Audio Processing, 2002
- MARSYAS: a framework for audio analysisOrganised Sound, 2000