Audio Caption: Listen and Tell
Authors:
- 1 May 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 830-834
Abstract
Increasing amount of research has shed light on machine perception of audio events, most of which concerns detection and classification tasks. However, human-like perception of audio scenes involves not only detecting and classifying audio sounds, but also summarizing the relationship between different audio events. Comparable research such as image caption has been conducted, yet the audio field is still quite barren. This paper introduces a manually-annotated dataset for audio caption. The purpose is to automatically generate natural sentences for audio scene description and to bridge the gap between machine perception of audio and image. The whole dataset is labelled in Mandarin and we also include translated English annotations. A baseline encoder-decoder model is provided for both English and Mandarin. Similar BLEU scores are derived for both languages: our model can generate understandable and data-related captions based on the dataset.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- CNN architectures for large-scale audio classification2017
- TUT database for acoustic scene classification and sound event detection2016
- Deep Visual-Semantic Alignments for Generating Image DescriptionsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- Adding Chinese Captions to Images2016
- Jointly Modeling Embedding and Translation to Bridge Video and Language2016
- Show and tell: A neural image caption generator2015
- A Dataset and Taxonomy for Urban Sound Research2014
- The Stanford CoreNLP Natural Language Processing Toolkit2014
- A Systematic Comparison of Smoothing Techniques for Sentence-Level BLEU2014
- Measuring nominal scale agreement among many raters.Psychological Bulletin, 1971