Xi Shao
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Xi Shao has a total of 56 co-authors. Here are the top co-authors by number of shared publications:
Bing-Kun Bao
Nanjing University of Posts and Telecommunications
4 shared publications- Bin Hu
Key Laboratory of Brain Health Intelligent Evaluation and Intervention, Ministry of Education, Beijing Institute of Technology, Beijing, China
2 shared publications - Björn Schuller
Technische Universität München, Munich, Germany
2 shared publications Changsheng Xu
Institute for Infocomm Research, Singapore
2 shared publications- Kun Qian
Beijing Institute of Technology, Beijing, China
2 shared publications Qi Tian
Institute for Infocomm Research, Singapore
2 shared publications
Latest Publications
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 34, 4482-4494
Temporal Prediction Alignment for Short-Duration Acoustic Scene Classification
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 34, 4482-4494
Temporal Prediction Alignment for Short-Duration Acoustic Scene Classification
- 29 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 33, 4527-4537
Learning Time-Graph Frequency Representation for Monaural Speech Enhancement
The Graph Fourier Transform (GFT) has recently demonstrated promising results in speech enhancement. However, existing GFT-based speech enhancement approaches often employ fixed graph topologies to build the graph Fourier basis, whose the representation lacks the adaptively and flexibility. In addition, they suffer from the numerical errors and instability introduced by matrix inversion in GFT based on both Singular Value Decomposition (GFT-SVD) and Eigen Vector Decomposition (GFT-EVD). Motivated by these limitations, this paper propose a simple yet effective learnable GFT-SVD framework for speech enhancement. Specifically, we leverage graph shift operators to construct a learnable graph topology and define a learnable graph Fourier basis by the singular value matrices using 1-D convolution (Conv-1D) neural layer. This eliminates the need for matrix inversion, thereby avoiding the associated numerical errors and stability problem. In contrast to complex-valued representation, our proposed learnable Fourier basis provides a real-valued time-graph representation, enabling better magnitude–phase alignment in speech enhancement. Comprehensive evaluations on the VCTK+DEMAND and DNS-2020 benchmarks demonstrate the consistent performance superiority of our learnable GFT-SVD over fixed STFT, GFT-EVD, and GFT-SVD within existing neural speech enhancement frameworks. Source code is available at:https://github.com/Wangfighting0015/GFT_project.
- 29 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 33, 4527-4537
Learning Time-Graph Frequency Representation for Monaural Speech Enhancement
The Graph Fourier Transform (GFT) has recently demonstrated promising results in speech enhancement. However, existing GFT-based speech enhancement approaches often employ fixed graph topologies to build the graph Fourier basis, whose the representation lacks the adaptively and flexibility. In addition, they suffer from the numerical errors and instability introduced by matrix inversion in GFT based on both Singular Value Decomposition (GFT-SVD) and Eigen Vector Decomposition (GFT-EVD). Motivated by these limitations, this paper propose a simple yet effective learnable GFT-SVD framework for speech enhancement. Specifically, we leverage graph shift operators to construct a learnable graph topology and define a learnable graph Fourier basis by the singular value matrices using 1-D convolution (Conv-1D) neural layer. This eliminates the need for matrix inversion, thereby avoiding the associated numerical errors and stability problem. In contrast to complex-valued representation, our proposed learnable Fourier basis provides a real-valued time-graph representation, enabling better magnitude–phase alignment in speech enhancement. Comprehensive evaluations on the VCTK+DEMAND and DNS-2020 benchmarks demonstrate the consistent performance superiority of our learnable GFT-SVD over fixed STFT, GFT-EVD, and GFT-SVD within existing neural speech enhancement frameworks. Source code is available at:https://github.com/Wangfighting0015/GFT_project.
- 23 July 2025
- Published by Springer Nature
- p. 69-82
Accent Recognition with Auxiliary Task and Contrastive Learning
- 23 July 2025
- Published by Springer Nature
- p. 69-82
Accent Recognition with Auxiliary Task and Contrastive Learning
- 7 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 16 (2) , 861-874
STAA-Net: A Sparse and Transferable Adversarial Attack for Speech Emotion Recognition
Speech contains rich information on the emotions of humans, and Speech Emotion Recognition (SER) has been an important topic in the area of human-computer interaction. The robustness of SER models is crucial, particularly in privacy-sensitive and reliability-demanding domains like private healthcare. Recently, the vulnerability of deep neural networks in the audio domain to adversarial attacks has become a popular area of research. However, prior works on adversarial attacks in the audio domain primarily rely on iterative gradient-based techniques, which are time-consuming and prone to overfitting the specific threat model. Furthermore, the exploration of sparse perturbations, which have the potential for better stealthiness, remains limited in the audio domain. To address these challenges, we propose a generator-based attack method to generate sparse and transferable adversarial examples to deceive SER models in an end-to-end and efficient manner. We evaluate our method on two widely-used SER datasets, Database of Elicited Mood in Speech (DEMoS) and Interactive Emotional dyadic MOtion CAPture (IEMOCAP), and demonstrate its ability to generate successful sparse adversarial examples in an efficient manner. Moreover, our generated adversarial examples exhibit model-agnostic transferability, enabling effective adversarial attacks on advanced victim models. The source code for this project is available at https://github.com/glam-imperial/STAA-Net-SER .
- 7 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 16 (2) , 861-874
STAA-Net: A Sparse and Transferable Adversarial Attack for Speech Emotion Recognition
Speech contains rich information on the emotions of humans, and Speech Emotion Recognition (SER) has been an important topic in the area of human-computer interaction. The robustness of SER models is crucial, particularly in privacy-sensitive and reliability-demanding domains like private healthcare. Recently, the vulnerability of deep neural networks in the audio domain to adversarial attacks has become a popular area of research. However, prior works on adversarial attacks in the audio domain primarily rely on iterative gradient-based techniques, which are time-consuming and prone to overfitting the specific threat model. Furthermore, the exploration of sparse perturbations, which have the potential for better stealthiness, remains limited in the audio domain. To address these challenges, we propose a generator-based attack method to generate sparse and transferable adversarial examples to deceive SER models in an end-to-end and efficient manner. We evaluate our method on two widely-used SER datasets, Database of Elicited Mood in Speech (DEMoS) and Interactive Emotional dyadic MOtion CAPture (IEMOCAP), and demonstrate its ability to generate successful sparse adversarial examples in an efficient manner. Moreover, our generated adversarial examples exhibit model-agnostic transferability, enabling effective adversarial attacks on advanced victim models. The source code for this project is available at https://github.com/glam-imperial/STAA-Net-SER .
- Open Access
Leveraging Self-supervised Audio Representations for Data-Efficient Acoustic Scene Classification
Acoustic scene classification (ASC) predominantly relies on supervised approaches. However, acquiring labeled data for training ASC models is often costly and time-consuming. Recently, self-supervised learning (SSL) has emerged as a powerful method for extracting features from unlabeled audio data, benefiting many downstream audio tasks. This paper proposes a data-efficient and low-complexity ASC system by leveraging self-supervised audio representations extracted from general-purpose audio datasets. We introduce BEATs, an audio SSL pre-trained model, to extract the general representations from AudioSet. Through extensive experiments, it has been demonstrated that the self-supervised audio representations can help to achieve high ASC accuracy with limited labeled fine-tuning data. Furthermore, we find that ensembling the SSL models fine-tuned with different strategies contributes to a further performance improvement. To meet low-complexity requirements, we use knowledge distillation to transfer the self-supervised knowledge from large teacher models to an efficient student model. The experimental results suggest that the self-supervised teachers effectively improve the classification accuracy of the student model. Our best-performing system obtains an average accuracy of 56.7%.
Open AccessLeveraging Self-supervised Audio Representations for Data-Efficient Acoustic Scene Classification
Acoustic scene classification (ASC) predominantly relies on supervised approaches. However, acquiring labeled data for training ASC models is often costly and time-consuming. Recently, self-supervised learning (SSL) has emerged as a powerful method for extracting features from unlabeled audio data, benefiting many downstream audio tasks. This paper proposes a data-efficient and low-complexity ASC system by leveraging self-supervised audio representations extracted from general-purpose audio datasets. We introduce BEATs, an audio SSL pre-trained model, to extract the general representations from AudioSet. Through extensive experiments, it has been demonstrated that the self-supervised audio representations can help to achieve high ASC accuracy with limited labeled fine-tuning data. Furthermore, we find that ensembling the SSL models fine-tuned with different strategies contributes to a further performance improvement. To meet low-complexity requirements, we use knowledge distillation to transfer the self-supervised knowledge from large teacher models to an efficient student model. The experimental results suggest that the self-supervised teachers effectively improve the classification accuracy of the student model. Our best-performing system obtains an average accuracy of 56.7%.
- Open Access
STAA-Net: A Sparse and Transferable Adversarial Attack for Speech Emotion Recognition
Speech contains rich information on the emotions of humans, and Speech Emotion Recognition (SER) has been an important topic in the area of human-computer interaction. The robustness of SER models is crucial, particularly in privacy-sensitive and reliability-demanding domains like private healthcare. Recently, the vulnerability of deep neural networks in the audio domain to adversarial attacks has become a popular area of research. However, prior works on adversarial attacks in the audio domain primarily rely on iterative gradient-based techniques, which are time-consuming and prone to overfitting the specific threat model. Furthermore, the exploration of sparse perturbations, which have the potential for better stealthiness, remains limited in the audio domain. To address these challenges, we propose a generator-based attack method to generate sparse and transferable adversarial examples to deceive SER models in an end-to-end and efficient manner. We evaluate our method on two widely-used SER datasets, Database of Elicited Mood in Speech (DEMoS) and Interactive Emotional dyadic MOtion CAPture (IEMOCAP), and demonstrate its ability to generate successful sparse adversarial examples in an efficient manner. Moreover, our generated adversarial examples exhibit model-agnostic transferability, enabling effective adversarial attacks on advanced victim models.
Open AccessSTAA-Net: A Sparse and Transferable Adversarial Attack for Speech Emotion Recognition
Speech contains rich information on the emotions of humans, and Speech Emotion Recognition (SER) has been an important topic in the area of human-computer interaction. The robustness of SER models is crucial, particularly in privacy-sensitive and reliability-demanding domains like private healthcare. Recently, the vulnerability of deep neural networks in the audio domain to adversarial attacks has become a popular area of research. However, prior works on adversarial attacks in the audio domain primarily rely on iterative gradient-based techniques, which are time-consuming and prone to overfitting the specific threat model. Furthermore, the exploration of sparse perturbations, which have the potential for better stealthiness, remains limited in the audio domain. To address these challenges, we propose a generator-based attack method to generate sparse and transferable adversarial examples to deceive SER models in an end-to-end and efficient manner. We evaluate our method on two widely-used SER datasets, Database of Elicited Mood in Speech (DEMoS) and Interactive Emotional dyadic MOtion CAPture (IEMOCAP), and demonstrate its ability to generate successful sparse adversarial examples in an efficient manner. Moreover, our generated adversarial examples exhibit model-agnostic transferability, enabling effective adversarial attacks on advanced victim models.
- 29 October 2023
- Published by Springer Nature
- p. 72-83
Multi-level Semantic Extraction Using Graph Pooling Network for Text Representation
- 29 October 2023
- Published by Springer Nature
- p. 72-83
Multi-level Semantic Extraction Using Graph Pooling Network for Text Representation
- 9 January 2022
- Published by Springer Nature in Multimedia Systems
- Vol. 28 (3) , 749-759
Unbiased feature enhancement framework for cross-modality person re-identification
- 9 January 2022
- Published by Springer Nature in Multimedia Systems
- Vol. 28 (3) , 749-759
Unbiased feature enhancement framework for cross-modality person re-identification
- Open Access
- 1 December 2020
- Published by MDPI AG in Electronics
- Vol. 9 (12) , 2038
Scene Recognition Based on Recurrent Memorized Attention Network
We propose a new end-to-end scene recognition framework, called a Recurrent Memorized Attention Network (RMAN) model, which performs object-based scene classification by recurrently locating and memorizing objects in the image. Based on the proposed framework, we introduce a multi-task mechanism that contiguously attends on the different essential objects in a scene image and recurrently performs memory fusion of the features of object focused by an attention model to improve the scene recognition accuracy. The experimental results show that the RMAN model has achieved better classification performance on the constructed dataset and two public scene datasets, surpassing state-of-the-art image scene recognition approaches.
- 1 December 2020
- Published by MDPI AG in Electronics
- Vol. 9 (12) , 2038
Open AccessScene Recognition Based on Recurrent Memorized Attention Network
We propose a new end-to-end scene recognition framework, called a Recurrent Memorized Attention Network (RMAN) model, which performs object-based scene classification by recurrently locating and memorizing objects in the image. Based on the proposed framework, we introduce a multi-task mechanism that contiguously attends on the different essential objects in a scene image and recurrently performs memory fusion of the features of object focused by an attention model to improve the scene recognition accuracy. The experimental results show that the RMAN model has achieved better classification performance on the constructed dataset and two public scene datasets, surpassing state-of-the-art image scene recognition approaches.
- Open Access
- 13 August 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 7, 113043-113052
Personalized Travel Recommendation Based on Sentiment-Aware Multimodal Topic Model
In this paper, we try to solve the personalized travel recommendation problem by exploiting the multi-modal data available from the real world social media, and a probabilistic graph model so called Sentiment-aware Multi-modal Topic Model (SMTM) is proposed to mine the latent semantics of the multi-modal data on the online travel website. Distinguished from previous approaches, our proposed approach try to mine the topics from tourist and attraction domains separately for disclosing semantics for tourist topics and attraction themes. In addition, we analyze tourist’s sentiments on attractions to further obtain the tourist’s attitude over attractions and recommend the attraction with proper sentiment on the related attraction themes accordingly. Based on the proposed SMTM model, the documents in tourist domain and in attraction domain can be compared with each other after they were projected into the mutual topic space, and this latent space projection scheme can be further applied to two personalized traveling recommendations, that is, the single platform traveling recommendation and the inter-platform traveling recommendation. Evaluation results based on the real world online travel website have shown the improved performance of our method.
- 13 August 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 7, 113043-113052
Open AccessPersonalized Travel Recommendation Based on Sentiment-Aware Multimodal Topic Model
In this paper, we try to solve the personalized travel recommendation problem by exploiting the multi-modal data available from the real world social media, and a probabilistic graph model so called Sentiment-aware Multi-modal Topic Model (SMTM) is proposed to mine the latent semantics of the multi-modal data on the online travel website. Distinguished from previous approaches, our proposed approach try to mine the topics from tourist and attraction domains separately for disclosing semantics for tourist topics and attraction themes. In addition, we analyze tourist’s sentiments on attractions to further obtain the tourist’s attitude over attractions and recommend the attraction with proper sentiment on the related attraction themes accordingly. Based on the proposed SMTM model, the documents in tourist domain and in attraction domain can be compared with each other after they were projected into the mutual topic space, and this latent space projection scheme can be further applied to two personalized traveling recommendations, that is, the single platform traveling recommendation and the inter-platform traveling recommendation. Evaluation results based on the real world online travel website have shown the improved performance of our method.