Yun Wang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Yun Wang has a total of 31 co-authors. Here are the top co-authors by number of shared publications:
Florian Metze
Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA, U.S.A.
10 shared publicationsJuncheng Li
Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA, U.S.A.
3 shared publicationsDi Xu
Language Technologies Institute, Carnegie Mellon University; Pittsburgh, PA/ Moffett Field, CA; U.S.A
2 shared publications- Florian Metze
Carnegie Mellon University, Pittsburgh, PA, USA
2 shared publications Joseph Szurley
2 shared publicationsSamarjit Das
2 shared publications
Latest Publications
- 1 May 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 745-749
Connectionist Temporal Localization for Sound Event Detection with Sequential Labeling
Research on sound event detection (SED) with weak labeling has mostly focused on presence/absence labeling, which provides no temporal information at all about the event occurrences. In this paper, we consider SED with sequential labeling, which specifies the temporal order of the event boundaries. The conventional connectionist temporal classification (CTC) framework, when applied to SED with sequential labeling, does not localize long events well due to a "peak clustering" problem. We adapt the CTC framework and propose connectionist temporal localization (CTL), which successfully solves the problem. Evaluation on a subset of Audio Set shows that CTL closes a third of the gap between presence/ absence labeling and strong labeling, demonstrating the usefulness of the extra temporal information in sequential labeling. CTL also makes it easy to combine sequential labeling with presence/absence labeling and strong labeling.
- 1 May 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 745-749
Connectionist Temporal Localization for Sound Event Detection with Sequential Labeling
Research on sound event detection (SED) with weak labeling has mostly focused on presence/absence labeling, which provides no temporal information at all about the event occurrences. In this paper, we consider SED with sequential labeling, which specifies the temporal order of the event boundaries. The conventional connectionist temporal classification (CTC) framework, when applied to SED with sequential labeling, does not localize long events well due to a "peak clustering" problem. We adapt the CTC framework and propose connectionist temporal localization (CTL), which successfully solves the problem. Evaluation on a subset of Audio Set shows that CTL closes a third of the gap between presence/ absence labeling and strong labeling, demonstrating the usefulness of the extra temporal information in sequential labeling. CTL also makes it easy to combine sequential labeling with presence/absence labeling and strong labeling.
- Top Cited Papers
- 1 May 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 31-35
A Comparison of Five Multiple Instance Learning Pooling Functions for Sound Event Detection with Weak Labeling
Sound event detection (SED) entails two subtasks: recognizing what types of sound events are present in an audio stream (audio tagging), and pinpointing their onset and offset times (localization). In the popular multiple instance learning (MIL) framework for SED with weak labeling, an important component is the pooling function. This paper compares five types of pooling functions both theoretically and experimentally, with special focus on their performance of localization. Although the attention pooling function is currently receiving the most attention, we find the linear softmax pooling function to perform the best among the five. Using this pooling function, we build a neural network called TALNet. It is the first system to reach state-of-the-art audio tagging performance on Audio Set, while exhibiting strong localization performance on the DCASE 2017 challenge at the same time.
- 1 May 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 31-35
Top Cited PapersA Comparison of Five Multiple Instance Learning Pooling Functions for Sound Event Detection with Weak Labeling
Sound event detection (SED) entails two subtasks: recognizing what types of sound events are present in an audio stream (audio tagging), and pinpointing their onset and offset times (localization). In the popular multiple instance learning (MIL) framework for SED with weak labeling, an important component is the pooling function. This paper compares five types of pooling functions both theoretically and experimentally, with special focus on their performance of localization. Although the attention pooling function is currently receiving the most attention, we find the linear softmax pooling function to perform the best among the five. Using this pooling function, we build a neural network called TALNet. It is the first system to reach state-of-the-art audio tagging performance on Audio Set, while exhibiting strong localization performance on the DCASE 2017 challenge at the same time.
- 1 March 2019
- Published by Elsevier in Expert Systems with Applications
- Vol. 117, 29-41
Assessing the performances of different neural network architectures for the detection of screams and shouts in public transportation
- 1 March 2019
- Published by Elsevier in Expert Systems with Applications
- Vol. 117, 29-41
Assessing the performances of different neural network architectures for the detection of screams and shouts in public transportation
- 2 September 2018
- Published by International Speech Communication Association
- p. 1383-1387
The ACLEW DiViMe: An Easy-to-use Diarization Tool
- 2 September 2018
- Published by International Speech Communication Association
- p. 1383-1387
The ACLEW DiViMe: An Easy-to-use Diarization Tool
- Open Access
- 2 September 2018
- Published by International Speech Communication Association
- p. 1339-1343
Comparing the Max and Noisy-Or Pooling Functions in Multiple Instance Learning for Weakly Supervised Sequence Learning Tasks
- 2 September 2018
- Published by International Speech Communication Association
- p. 1339-1343
Open AccessComparing the Max and Noisy-Or Pooling Functions in Multiple Instance Learning for Weakly Supervised Sequence Learning Tasks
- 1 April 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 6832-6836
A Light-Weight Multimodal Framework for Improved Environmental Audio Tagging
The lack of strong labels has severely limited the state-of-the-art fully supervised audio tagging systems to be scaled to larger dataset. Meanwhile, audio-visual learning models based on unlabeled videos have been successfully applied to audio tagging, but they are inevitably resource hungry and require a long time to train. In this work, we propose a light-weight, multimodal framework for environmental audio tagging. The audio branch of the framework is a convolutional and recurrent neural network (CRNN) based on multiple instance learning (MIL). It is trained with the audio tracks of a large collection of weakly labeled YouTube video excerpts; the video branch uses pretrained state-of-the-art image recognition networks and word embeddings to extract information from the video track and to map visual objects to sound events. Experiments on the audio tagging task of the DCASE 2017 challenge show that the incorporation of video information improves a strong baseline audio tagging system by 5.3% in terms of F 1 score. The entire system can be trained within 6 hours on a single GPU, and can be easily carried over to other audio tasks such as speech sentimental analysis.
- 1 April 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 6832-6836
A Light-Weight Multimodal Framework for Improved Environmental Audio Tagging
The lack of strong labels has severely limited the state-of-the-art fully supervised audio tagging systems to be scaled to larger dataset. Meanwhile, audio-visual learning models based on unlabeled videos have been successfully applied to audio tagging, but they are inevitably resource hungry and require a long time to train. In this work, we propose a light-weight, multimodal framework for environmental audio tagging. The audio branch of the framework is a convolutional and recurrent neural network (CRNN) based on multiple instance learning (MIL). It is trained with the audio tracks of a large collection of weakly labeled YouTube video excerpts; the video branch uses pretrained state-of-the-art image recognition networks and word embeddings to extract information from the video track and to map visual objects to sound events. Experiments on the audio tagging task of the DCASE 2017 challenge show that the incorporation of video information improves a strong baseline audio tagging system by 5.3% in terms of F 1 score. The entire system can be trained within 6 hours on a single GPU, and can be easily carried over to other audio tasks such as speech sentimental analysis.
- 1 March 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2986-2990
A first attempt at polyphonic sound event detection using connectionist temporal classification
Sound event detection is the task of detecting the type, starting time, and ending time of sound events in audio streams. Recently, recurrent neural networks (RNNs) have become the mainstream solution for sound event detection. Because RNNs make a prediction at every frame, it is necessary to provide exact starting and ending times of the sound events in the training data, making data annotation an extremely time-consuming process. Connectionist temporal classification (CTC), as a sequence-to-sequence model, can relax this constraint, because it suffices to provide ordered sequences of sound events without exact starting and ending times. This paper presents a first attempt at using CTC for sound event detection. In the polyphonic situation, sound events may overlap with each other, making it hard to define ordered sequences of sound events. We propose to use the boundaries (i.e. starts and ends) of the sound events as tokens for CTC. We show that CTC is able to locate the boundaries of sound events on a very noisy corpus of consumer generated content with rough hints about their positions. The CTC approach seems to be particularly suited to detecting short and transient sounds, which have traditionally been hardest to detect.
- 1 March 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2986-2990
A first attempt at polyphonic sound event detection using connectionist temporal classification
Sound event detection is the task of detecting the type, starting time, and ending time of sound events in audio streams. Recently, recurrent neural networks (RNNs) have become the mainstream solution for sound event detection. Because RNNs make a prediction at every frame, it is necessary to provide exact starting and ending times of the sound events in the training data, making data annotation an extremely time-consuming process. Connectionist temporal classification (CTC), as a sequence-to-sequence model, can relax this constraint, because it suffices to provide ordered sequences of sound events without exact starting and ending times. This paper presents a first attempt at using CTC for sound event detection. In the polyphonic situation, sound events may overlap with each other, making it hard to define ordered sequences of sound events. We propose to use the boundaries (i.e. starts and ends) of the sound events as tokens for CTC. We show that CTC is able to locate the boundaries of sound events on a very noisy corpus of consumer generated content with rough hints about their positions. The CTC approach seems to be particularly suited to detecting short and transient sounds, which have traditionally been hardest to detect.
- Open Access
- 6 June 2016
- Published by Association for Computing Machinery (ACM)
- p. 265-269
Recurrent Support Vector Machines for Audio-Based Multimedia Event Detection
Multimedia event detection (MED) is the task of detecting given events (e.g. parade, birthday party) in a large collection of video clips. While the most useful information comes from visual features and speech recognition, a lot can also be inferred from the non-speech audio content, either alone or in conjunction with visual and speech cues. This paper studies MED with non-speech audio information only. MED is usually performed in two stages. The first stage generates a representation for each clip in the form of either a single vector or a sequence of vectors, often by aggregating frame-level features; the second stage performs binary or multi-class classification to decide whether each target event occurs in each clip. Common classifiers used for the second stage include support vector machines (SVMs), feed-forward deep neural networks (DNNs), and recurrent neural networks (RNNs). In this paper, we propose to classify clips for events using \"recurrent SVMs\". These models combine the kernel mapping and the large-margin optimization criterion of SVMs, and the ability to process sequences of variable lengths of RNNs. Reinforced with data augmentation, recurrent SVMs have achieved higher mean average precision (MAP) on the TRECVID 2011 MED task than both SVMs and RNNs.
- 6 June 2016
- Published by Association for Computing Machinery (ACM)
- p. 265-269
Open AccessRecurrent Support Vector Machines for Audio-Based Multimedia Event Detection
Multimedia event detection (MED) is the task of detecting given events (e.g. parade, birthday party) in a large collection of video clips. While the most useful information comes from visual features and speech recognition, a lot can also be inferred from the non-speech audio content, either alone or in conjunction with visual and speech cues. This paper studies MED with non-speech audio information only. MED is usually performed in two stages. The first stage generates a representation for each clip in the form of either a single vector or a sequence of vectors, often by aggregating frame-level features; the second stage performs binary or multi-class classification to decide whether each target event occurs in each clip. Common classifiers used for the second stage include support vector machines (SVMs), feed-forward deep neural networks (DNNs), and recurrent neural networks (RNNs). In this paper, we propose to classify clips for events using \"recurrent SVMs\". These models combine the kernel mapping and the large-margin optimization criterion of SVMs, and the ability to process sequences of variable lengths of RNNs. Reinforced with data augmentation, recurrent SVMs have achieved higher mean average precision (MAP) on the TRECVID 2011 MED task than both SVMs and RNNs.