Haoyu Song
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Haoyu Song has a total of 10 co-authors. Here are the top co-authors by number of shared publications:
- Ian McLoughlin
Singapore Institute of Technology
4 shared publications - Nan Jiang
National Engineering Research Center of Speech and Language Information Processing (NERC-SLIP), The University of Science and Technology of China, Hefei 230026, China
3 shared publications - Qing Gu
National Engineering Research Center of Speech and Language Information Processing (NERC-SLIP), The University of Science and Technology of China, Hefei 230026, China
3 shared publications - Yan Song
University of Science and Technology of China
3 shared publications - Pengfei Cai
National Engineering Research Center of Speech and Language Information Processing (NERC-SLIP), The University of Science and Technology of China, Hefei 230026, China
2 shared publications - Donny Soh
ICT Cluster, Singapore Institute of Technology, Singapore 828608, Singapore
1 shared publication
Latest Publications
- Open Access
- 6 November 2026
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 85 (11) , 836
Assessing gender–object association bias in text-to-image generation
- 6 November 2026
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 85 (11) , 836
Open AccessAssessing gender–object association bias in text-to-image generation
- Top Cited PapersOpen Access
- 6 January 2026
- Published by MDPI AG in Applied Sciences
- Vol. 16 (2) , 572
Spectrogram Features for Audio and Speech Analysis
Spectrogram-based representations have grown to dominate the feature space for deep learning audio analysis systems, and are often adopted for speech analysis also. Initially, the primary motivation behind spectrogram-based representations was their ability to present sound as a two-dimensional signal in the time–frequency plane, which not only provides an interpretable physical basis for analysing sound, but also unlocks the use of a range of machine learning techniques such as convolutional neural networks, which had been developed for image processing. A spectrogram is a matrix characterised by the resolution and span of its dimensions, as well as by the representation and scaling of each element. Many possibilities for these three characteristics have been explored by researchers across numerous application areas, with different settings showing affinity for various tasks. This paper reviews the use of spectrogram-based representations and surveys the state-of-the-art to question how front-end feature representation choice allies with back-end classifier architecture for different tasks.
- 6 January 2026
- Published by MDPI AG in Applied Sciences
- Vol. 16 (2) , 572
Top Cited PapersOpen AccessSpectrogram Features for Audio and Speech Analysis
Spectrogram-based representations have grown to dominate the feature space for deep learning audio analysis systems, and are often adopted for speech analysis also. Initially, the primary motivation behind spectrogram-based representations was their ability to present sound as a two-dimensional signal in the time–frequency plane, which not only provides an interpretable physical basis for analysing sound, but also unlocks the use of a range of machine learning techniques such as convolutional neural networks, which had been developed for image processing. A spectrogram is a matrix characterised by the resolution and span of its dimensions, as well as by the representation and scaling of each element. Many possibilities for these three characteristics have been explored by researchers across numerous application areas, with different settings showing affinity for various tasks. This paper reviews the use of spectrogram-based representations and surveys the state-of-the-art to question how front-end feature representation choice allies with back-end classifier architecture for different tasks.
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 582-591
Detect Any Sound: Open-Vocabulary Sound Event Detection with Multi-Modal Queries
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 582-591
Open AccessDetect Any Sound: Open-Vocabulary Sound Event Detection with Multi-Modal Queries
- Open Access
An Efficient Transfer Learning Method Based on Adapter with Local Attributes for Speech Emotion Recognition
Existing speech emotion recognition (SER) methods commonly suffer from the lack of high-quality large-scale corpus, partly due to the complex, psychological nature of emotion which makes accurate labeling difficult and time consuming. Recently, transfer learning based methods that exploit the encoders pretrained on large-scale speech corpus (e.g., Wav2Vec2.0 and HuBERT) have shown strong potential for downstream SER tasks. However, task-specific fine-tuning remains necessary for various conversational scenarios of different topics, speakers and languages to achieve satisfactory performance. It generally requires costly encoder retraining for individual SER tasks. To address this issue, we propose to train an adapter with local attributes for efficient transfer learning. Specifically, a weighted average pooling-Transformer (WAP-Transformer) is proposed as a lightweight backbone to enrich the frame-level representation. An adapter with teacher-student branches is exploited for task-agnostic transfer learning, where the student branch is jointly optimized via mask prediction and self-distillation objectives, and the teacher branch is obtained online from the student via exponential moving average (EMA). Meanwhile, local attributes are learned from the teacher branch via unsupervised clustering, which aims to act as a universal model that provides additional semantic-rich supervisions. A statistical attentive pooling (SAP) module is proposed to obtain utterance representation for fine-tuning. To evaluate the effectiveness of the proposed adapter with local attributes, extensive experiments have been conducted on IEMOCAP. Superior performance has been reported, compared to the previous state-of-the-art methods in similar settings.
Open AccessAn Efficient Transfer Learning Method Based on Adapter with Local Attributes for Speech Emotion Recognition
Existing speech emotion recognition (SER) methods commonly suffer from the lack of high-quality large-scale corpus, partly due to the complex, psychological nature of emotion which makes accurate labeling difficult and time consuming. Recently, transfer learning based methods that exploit the encoders pretrained on large-scale speech corpus (e.g., Wav2Vec2.0 and HuBERT) have shown strong potential for downstream SER tasks. However, task-specific fine-tuning remains necessary for various conversational scenarios of different topics, speakers and languages to achieve satisfactory performance. It generally requires costly encoder retraining for individual SER tasks. To address this issue, we propose to train an adapter with local attributes for efficient transfer learning. Specifically, a weighted average pooling-Transformer (WAP-Transformer) is proposed as a lightweight backbone to enrich the frame-level representation. An adapter with teacher-student branches is exploited for task-agnostic transfer learning, where the student branch is jointly optimized via mask prediction and self-distillation objectives, and the teacher branch is obtained online from the student via exponential moving average (EMA). Meanwhile, local attributes are learned from the teacher branch via unsupervised clustering, which aims to act as a universal model that provides additional semantic-rich supervisions. A statistical attentive pooling (SAP) module is proposed to obtain utterance representation for fine-tuning. To evaluate the effectiveness of the proposed adapter with local attributes, extensive experiments have been conducted on IEMOCAP. Superior performance has been reported, compared to the previous state-of-the-art methods in similar settings.