Chengwei Huang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Chengwei Huang has a total of 117 co-authors. Here are the top co-authors by number of shared publications:
Li Zhao
School of Information Science and Engineering, Southeast University, Nanjing, China
12 shared publications- Björn Schuller
Technische Universität München, Munich, Germany
6 shared publications Yun Jin
6 shared publications- Jiaming Cheng
School of Information Science and Engineering, Southeast University, Nanjing, China
4 shared publications - Ruiyu Liang
School of Information and Communication Engineering, Nanjing Institute of Technology, Nanjing, China
4 shared publications Yan Zhao
4 shared publications
Latest Publications
- 13 September 2025
- Published by Elsevier in Information Fusion
- Vol. 127
Affine modulation-based audiogram fusion network for joint noise reduction and hearing loss compensation
- 13 September 2025
- Published by Elsevier in Information Fusion
- Vol. 127
Affine modulation-based audiogram fusion network for joint noise reduction and hearing loss compensation
- Open Access
Affine Modulation-based Audiogram Fusion Network for Joint Noise Reduction and Hearing Loss Compensation
Hearing aids (HAs) are widely used to provide personalized speech enhancement (PSE) services, improving the quality of life for individuals with hearing loss. However, HA performance significantly declines in noisy environments as it treats noise reduction (NR) and hearing loss compensation (HLC) as separate tasks. This separation leads to a lack of systematic optimization, overlooking the interactions between these two critical tasks, and increases the system complexity. To address these challenges, we propose a novel audiogram fusion network, named AFN-HearNet, which simultaneously tackles the NR and HLC tasks by fusing cross-domain audiogram and spectrum features. We propose an audiogram-specific encoder that transforms the sparse audiogram profile into a deep representation, addressing the alignment problem of cross-domain features prior to fusion. To incorporate the interactions between NR and HLC tasks, we propose the affine modulation-based audiogram fusion frequency-temporal Conformer that adaptively fuses these two features into a unified deep representation for speech reconstruction. Furthermore, we introduce a voice activity detection auxiliary training task to embed speech and non-speech patterns into the unified deep representation implicitly. We conduct comprehensive experiments across multiple datasets to validate the effectiveness of each proposed module. The results indicate that the AFN-HearNet significantly outperforms state-of-the-art in-context fusion joint models regarding key metrics such as HASQI and PESQ, achieving a considerable trade-off between performance and efficiency. The source code and data will be released at https://github.com/deepnetni/AFN-HearNet.
Open AccessAffine Modulation-based Audiogram Fusion Network for Joint Noise Reduction and Hearing Loss Compensation
Hearing aids (HAs) are widely used to provide personalized speech enhancement (PSE) services, improving the quality of life for individuals with hearing loss. However, HA performance significantly declines in noisy environments as it treats noise reduction (NR) and hearing loss compensation (HLC) as separate tasks. This separation leads to a lack of systematic optimization, overlooking the interactions between these two critical tasks, and increases the system complexity. To address these challenges, we propose a novel audiogram fusion network, named AFN-HearNet, which simultaneously tackles the NR and HLC tasks by fusing cross-domain audiogram and spectrum features. We propose an audiogram-specific encoder that transforms the sparse audiogram profile into a deep representation, addressing the alignment problem of cross-domain features prior to fusion. To incorporate the interactions between NR and HLC tasks, we propose the affine modulation-based audiogram fusion frequency-temporal Conformer that adaptively fuses these two features into a unified deep representation for speech reconstruction. Furthermore, we introduce a voice activity detection auxiliary training task to embed speech and non-speech patterns into the unified deep representation implicitly. We conduct comprehensive experiments across multiple datasets to validate the effectiveness of each proposed module. The results indicate that the AFN-HearNet significantly outperforms state-of-the-art in-context fusion joint models regarding key metrics such as HASQI and PESQ, achieving a considerable trade-off between performance and efficiency. The source code and data will be released at https://github.com/deepnetni/AFN-HearNet.
- 1 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 32 (23299290) , 2680-2691
Residual Fusion Probabilistic Knowledge Distillation for Speech Enhancement
In recent years, a great deal of research has focused on in developing neural network (NN)-based speech enhancement (SE) models, which have achieved promising results. However, NN-based models typically require expensive computations to achieve remarkable performance, constraining their deployment in real-world scenarios, especially when hardware resources are limited or when latency requirements are strict. To reduce this computational burden, we propose a unified residual fusion probabilistic knowledge distillation (KD) method for the SE task, in which knowledge is transferred from a deep teacher to a shallower student model. Previous KD approaches commonly focused on narrowing the output distances between teachers and students, but research on the intermediate representation of these models is lacking. In this paper, we first study the cross-layer residual feature fusion strategy, which enables the student model to distill knowledge contained in multiple teacher layers from shallow to deep. Second, a frame weighting probabilistic distillation loss is proposed to assign more emphasis to frames containing essential information and preserve pairwise probabilistic similarities in the representation space. The proposed distillation framework is applied to the dual-path dilated convolutional recurrent network (DPDCRN), which won the championship of the SE track in the L3DAS23 challenge. Extensive experiments are conducted on single-channel and multichannel SE datasets. Objective evaluations show that the proposed KD strategy outperforms other distillation methods and considerably improves the enhancement effect of the low-complexity student model (with only 17% of the teacher's parameters).
- 1 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 32 (23299290) , 2680-2691
Residual Fusion Probabilistic Knowledge Distillation for Speech Enhancement
In recent years, a great deal of research has focused on in developing neural network (NN)-based speech enhancement (SE) models, which have achieved promising results. However, NN-based models typically require expensive computations to achieve remarkable performance, constraining their deployment in real-world scenarios, especially when hardware resources are limited or when latency requirements are strict. To reduce this computational burden, we propose a unified residual fusion probabilistic knowledge distillation (KD) method for the SE task, in which knowledge is transferred from a deep teacher to a shallower student model. Previous KD approaches commonly focused on narrowing the output distances between teachers and students, but research on the intermediate representation of these models is lacking. In this paper, we first study the cross-layer residual feature fusion strategy, which enables the student model to distill knowledge contained in multiple teacher layers from shallow to deep. Second, a frame weighting probabilistic distillation loss is proposed to assign more emphasis to frames containing essential information and preserve pairwise probabilistic similarities in the representation space. The proposed distillation framework is applied to the dual-path dilated convolutional recurrent network (DPDCRN), which won the championship of the SE track in the L3DAS23 challenge. Extensive experiments are conducted on single-channel and multichannel SE datasets. Objective evaluations show that the proposed KD strategy outperforms other distillation methods and considerably improves the enhancement effect of the low-complexity student model (with only 17% of the teacher's parameters).
- Open Access
- 28 December 2023
- Published by MDPI AG in Electronics
- Vol. 13 (1) , 146
Fidgety Speech Emotion Recognition for Learning Process Modeling
In this paper, the recognition of fidgety speech emotion is studied, and real-world speech emotions are collected to enhance emotion recognition in practical scenarios, especially for cognitive tasks. We first focused on eliciting fidgety emotions and data acquisition for general math learning. Students practice mathematics by performing operations, solving problems, and orally responding to questions, all of which are recorded as audio data. Subsequently, the teacher evaluates the accuracy of these mathematical exercises by scoring, which reflects the cognitive outcomes of the students. Secondly, we propose an end-to-end speech emotion model based on a multi-scale one-dimensional (1-D) residual convolutional neural network. Finally, we conducted an experiment to recognize fidgety speech emotions by testing various classifiers, including SVM, LSTM, 1-D CNN, and the proposed multi-scale 1-D CNN. The experimental results show that the classifier we constructed can identify fidgety emotion well. After conducting a thorough analysis of fidgety emotions and their influence on the learning process, a clear relationship between the two was apparent. The automatic recognition of fidgety emotions is valuable for assisting on-line math teaching.
- 28 December 2023
- Published by MDPI AG in Electronics
- Vol. 13 (1) , 146
Open AccessFidgety Speech Emotion Recognition for Learning Process Modeling
In this paper, the recognition of fidgety speech emotion is studied, and real-world speech emotions are collected to enhance emotion recognition in practical scenarios, especially for cognitive tasks. We first focused on eliciting fidgety emotions and data acquisition for general math learning. Students practice mathematics by performing operations, solving problems, and orally responding to questions, all of which are recorded as audio data. Subsequently, the teacher evaluates the accuracy of these mathematical exercises by scoring, which reflects the cognitive outcomes of the students. Secondly, we propose an end-to-end speech emotion model based on a multi-scale one-dimensional (1-D) residual convolutional neural network. Finally, we conducted an experiment to recognize fidgety speech emotions by testing various classifiers, including SVM, LSTM, 1-D CNN, and the proposed multi-scale 1-D CNN. The experimental results show that the classifier we constructed can identify fidgety emotion well. After conducting a thorough analysis of fidgety emotions and their influence on the learning process, a clear relationship between the two was apparent. The automatic recognition of fidgety emotions is valuable for assisting on-line math teaching.
- Open Access
- 26 December 2023
- Published by BON VIEW PUBLISHING PTE in Journal of Data Science and Intelligent Systems
- Vol. 3 (4) , 291-303
Assistive Learning Intelligence Navigator (ALIN) Dataset: Predicting Test Results from Learning Data
Data mining techniques have garnered significant attention within the realm of education. In this paper, we present two public available datasets for adaptive learning and studied predicting algorithms for learning results. First, we present a student dataset characterized by its size and distinctive attributes. This dataset encompasses various task-related topics interconnected through a learning pathway, thereby enabling researchers to delve into the data from novel perspectives. Moreover, it encompasses extensive longitudinal student behavioral data, a rarity that adds substantial value. Spanning the years from 2010 to 2021, our dataset comprises a cohort of 7933 students, 64,344 test scores, and 183,390 behavior records, solidifying its status as a valuable resource for educational research. Second, we proposed methods for predicting the testing results with and without practice tests. Novel learning features are constructed and various machine learning algorithms are compared. Finally, in our experiments, we achieved precision rate of 0.703 and recall rate of 0.734 in the prediction of students’ test outcomes based on behavioral learning data. The robustness of our dataset makes it well-suited for examining the connection between student behavior and academic performance, developing tailored learning recommendations, and exploring diverse research avenues. Received: 8 September 2023 | Revised: 27 November 2023 | Accepted: 25 December 2023 Conflicts of Interest The authors declare that they have no conflicts of interest to this work. Data Availability Statement The data that support the findings of this study are openly available in Google at https://sites.google.com/site/assistmentsdata/home/2009-2010-assistment-data, in Github at https://github.com/AdaptiveLearning2022/DataSetALIN2022, and in IEEE Dataport at https://ieee-dataport.org/documents/alin-open-dataset-math-adaptive-learning. Author Contribution Statement Guijia He: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization. Chengwei Huang: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization, Supervision, Project administration, Funding acquisition. Steven Yang: Conceptualization, Methodology, Software, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization. Kelvin Lwin: Conceptualization, Methodology, Software, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization, Supervision, Project administration. Eng Lieh Ouh: Conceptualization, Methodology, Formal analysis, Investigation, Resources, Writing – original draft, Writing – review & editing, Visualization, Project administration. Ran Ju: Methodology, Software, Formal analysis, Investigation, Data curation, Writing – original draft, Writing – review & editing, Visualization. Xiaoming Zhu: Methodology, Validation, Investigation, Resources, Writing – original draft, Writing – review & editing, Visualization, Supervision, Funding acquisition.
- 26 December 2023
- Published by BON VIEW PUBLISHING PTE in Journal of Data Science and Intelligent Systems
- Vol. 3 (4) , 291-303
Open AccessAssistive Learning Intelligence Navigator (ALIN) Dataset: Predicting Test Results from Learning Data
Data mining techniques have garnered significant attention within the realm of education. In this paper, we present two public available datasets for adaptive learning and studied predicting algorithms for learning results. First, we present a student dataset characterized by its size and distinctive attributes. This dataset encompasses various task-related topics interconnected through a learning pathway, thereby enabling researchers to delve into the data from novel perspectives. Moreover, it encompasses extensive longitudinal student behavioral data, a rarity that adds substantial value. Spanning the years from 2010 to 2021, our dataset comprises a cohort of 7933 students, 64,344 test scores, and 183,390 behavior records, solidifying its status as a valuable resource for educational research. Second, we proposed methods for predicting the testing results with and without practice tests. Novel learning features are constructed and various machine learning algorithms are compared. Finally, in our experiments, we achieved precision rate of 0.703 and recall rate of 0.734 in the prediction of students’ test outcomes based on behavioral learning data. The robustness of our dataset makes it well-suited for examining the connection between student behavior and academic performance, developing tailored learning recommendations, and exploring diverse research avenues. Received: 8 September 2023 | Revised: 27 November 2023 | Accepted: 25 December 2023 Conflicts of Interest The authors declare that they have no conflicts of interest to this work. Data Availability Statement The data that support the findings of this study are openly available in Google at https://sites.google.com/site/assistmentsdata/home/2009-2010-assistment-data, in Github at https://github.com/AdaptiveLearning2022/DataSetALIN2022, and in IEEE Dataport at https://ieee-dataport.org/documents/alin-open-dataset-math-adaptive-learning. Author Contribution Statement Guijia He: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization. Chengwei Huang: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization, Supervision, Project administration, Funding acquisition. Steven Yang: Conceptualization, Methodology, Software, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization. Kelvin Lwin: Conceptualization, Methodology, Software, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization, Supervision, Project administration. Eng Lieh Ouh: Conceptualization, Methodology, Formal analysis, Investigation, Resources, Writing – original draft, Writing – review & editing, Visualization, Project administration. Ran Ju: Methodology, Software, Formal analysis, Investigation, Data curation, Writing – original draft, Writing – review & editing, Visualization. Xiaoming Zhu: Methodology, Validation, Investigation, Resources, Writing – original draft, Writing – review & editing, Visualization, Supervision, Funding acquisition.
- Open Access
- 20 December 2023
- Published by MDPI AG in Preprints.org
Predicting Students' Progress in Intelligent Tutoring Systems
Intelligent Tutoring Systems (ITS) are increasingly popular for online learning. These systems use adaptive algorithms to recommend relevant content based on students' profiles. However, instructors need to periodically assess students' performance to ensure learning outcomes and adjust strategies accordingly. Our objective is to predict students' progress in advance, enabling teachers to make quicker decisions and facilitating the iterative process of adaptive algorithms. For this study, we collected a dataset from ALIN, an online learning platform, consisting of over 5,000 students' learning records and test results. Using this dataset, we conducted experiments employing various machine learning algorithms. The results indicate that learning behavior contributes to improving forecast performance, while students' progress strongly correlates with their previous test results. Additionally, we discovered that students' progress can be indirectly predicted by forecasting their scores. Furthermore, by breaking down overall scores into several distinct components and predicting individual scores for each component, the accuracy of the forecasts can be improved.
- 20 December 2023
- Published by MDPI AG in Preprints.org
Open AccessPredicting Students' Progress in Intelligent Tutoring Systems
Intelligent Tutoring Systems (ITS) are increasingly popular for online learning. These systems use adaptive algorithms to recommend relevant content based on students' profiles. However, instructors need to periodically assess students' performance to ensure learning outcomes and adjust strategies accordingly. Our objective is to predict students' progress in advance, enabling teachers to make quicker decisions and facilitating the iterative process of adaptive algorithms. For this study, we collected a dataset from ALIN, an online learning platform, consisting of over 5,000 students' learning records and test results. Using this dataset, we conducted experiments employing various machine learning algorithms. The results indicate that learning behavior contributes to improving forecast performance, while students' progress strongly correlates with their previous test results. Additionally, we discovered that students' progress can be indirectly predicted by forecasting their scores. Furthermore, by breaking down overall scores into several distinct components and predicting individual scores for each component, the accuracy of the forecasts can be improved.
- Open Access
- 31 October 2023
- Published by MDPI AG in Preprints.org
Assistive Learning Intelligence Navigator (ALIN) Dataset: Predicting Test Results from Learning Data
Data mining techniques have garnered significant attention within the realm of education. However, the procurement of ample student data poses a formidable challenge. In response to this challenge, we present a student dataset characterized by its size and distinctive attributes. This dataset encompasses various task-related topics interconnected through a learning pathway, thereby enabling researchers to delve into the data from novel perspectives. Moreover, it encompasses extensive longitudinal student behavioral data, a rarity that adds substantial value. Spanning the years from 2010 to 2021, our dataset comprises a cohort of 7,933 students, 64,344 test scores, and 183,390 behavior records, solidifying its status as a valuable resource for educational research. In our experiments, we achieved successful predictions of students' test outcomes based on behavioral learning data. The strengths of our dataset render it apt for analyzing the nexus between student conduct and academic performance, crafting personalized learning recommendations, and pursuing various other research pursuits.
- 31 October 2023
- Published by MDPI AG in Preprints.org
Open AccessAssistive Learning Intelligence Navigator (ALIN) Dataset: Predicting Test Results from Learning Data
Data mining techniques have garnered significant attention within the realm of education. However, the procurement of ample student data poses a formidable challenge. In response to this challenge, we present a student dataset characterized by its size and distinctive attributes. This dataset encompasses various task-related topics interconnected through a learning pathway, thereby enabling researchers to delve into the data from novel perspectives. Moreover, it encompasses extensive longitudinal student behavioral data, a rarity that adds substantial value. Spanning the years from 2010 to 2021, our dataset comprises a cohort of 7,933 students, 64,344 test scores, and 183,390 behavior records, solidifying its status as a valuable resource for educational research. In our experiments, we achieved successful predictions of students' test outcomes based on behavioral learning data. The strengths of our dataset render it apt for analyzing the nexus between student conduct and academic performance, crafting personalized learning recommendations, and pursuing various other research pursuits.
- Open Access
- 30 October 2023
- Published by MDPI AG in Preprints.org
Knowledge Point Entity and Relation Extraction based on Pre-training Model in Education Resources
Based on the teaching material of "Robotics" course, this paper studied the automatic knowledge graph construction, including knowledge points extraction and knowledge point relation extraction. We proposed a new method of extracting the first-level, second-level, and third-level knowledge points as well as their prerequisite relations of knowledge points in the textbook. For the problem of insufficient knowledge of the pre-trained language model in the specific field, methods such as incremental pre-training and optimization of cost functions are employed to integrate subject knowledge into the pre-trained language model, thus improving its effectiveness. To overcome the problem that the traditional method of relationship extraction can not be applied directly to the extraction of teaching materials, a new scheme for knowledge point relationship extraction based on keyword relationship is proposed. The experimental data from textbooks shows that the F1 score of knowledge point extraction reaches 93%, considerably improved compared to the traditional model. Consequently, the knowledge point entity extraction and relationship extraction methods based on the pre-trained model can effectively extract structured information and facilitate the automatic construction of knowledge graphs.
- 30 October 2023
- Published by MDPI AG in Preprints.org
Open AccessKnowledge Point Entity and Relation Extraction based on Pre-training Model in Education Resources
Based on the teaching material of "Robotics" course, this paper studied the automatic knowledge graph construction, including knowledge points extraction and knowledge point relation extraction. We proposed a new method of extracting the first-level, second-level, and third-level knowledge points as well as their prerequisite relations of knowledge points in the textbook. For the problem of insufficient knowledge of the pre-trained language model in the specific field, methods such as incremental pre-training and optimization of cost functions are employed to integrate subject knowledge into the pre-trained language model, thus improving its effectiveness. To overcome the problem that the traditional method of relationship extraction can not be applied directly to the extraction of teaching materials, a new scheme for knowledge point relationship extraction based on keyword relationship is proposed. The experimental data from textbooks shows that the F1 score of knowledge point extraction reaches 93%, considerably improved compared to the traditional model. Consequently, the knowledge point entity extraction and relationship extraction methods based on the pre-trained model can effectively extract structured information and facilitate the automatic construction of knowledge graphs.
- 15 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cognitive and Developmental Systems
- Vol. 16 (3) , 973-983
Multichannel Speech Enhancement Based on Neural Beamforming and a Context-Focused Post-Filtering Network
Both spatial and temporal contextual information are essential for the multichannel speech enhancement (MCSE) task. In this work, we propose a unified MCSE network composed of neural beamforming and a context-focused post-filtering network in order to fully exploit these two types of information. The network is used to estimate the optimum complex ideal ratio masks (cIRM) which can more effectively utilize the phase information in the frequency domain to reconstruct the speech waveform. To assign adaptive weights to channels, we first adopt a dilated convolution-based network which simulates the beamforming on the original multichannel input spectrum as the front-end of the multichannel acoustic model. Furthermore, we propose a post-filtering network that inputs the suggested U-Net’s output to a convolutional long short-term memory (ConvLSTM) layer, as it can properly capture contextual information and spatial correlation information of features. We conduct experiments on the VOICES, CHiME-3 and WMIR datasets, respectively. Experiments show that, in various scenarios, the proposed algorithm shows improvements over the previous state-of-the-art algorithms in terms of PESQ, STOI and SI-SNR.
- 15 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cognitive and Developmental Systems
- Vol. 16 (3) , 973-983
Multichannel Speech Enhancement Based on Neural Beamforming and a Context-Focused Post-Filtering Network
Both spatial and temporal contextual information are essential for the multichannel speech enhancement (MCSE) task. In this work, we propose a unified MCSE network composed of neural beamforming and a context-focused post-filtering network in order to fully exploit these two types of information. The network is used to estimate the optimum complex ideal ratio masks (cIRM) which can more effectively utilize the phase information in the frequency domain to reconstruct the speech waveform. To assign adaptive weights to channels, we first adopt a dilated convolution-based network which simulates the beamforming on the original multichannel input spectrum as the front-end of the multichannel acoustic model. Furthermore, we propose a post-filtering network that inputs the suggested U-Net’s output to a convolutional long short-term memory (ConvLSTM) layer, as it can properly capture contextual information and spatial correlation information of features. We conduct experiments on the VOICES, CHiME-3 and WMIR datasets, respectively. Experiments show that, in various scenarios, the proposed algorithm shows improvements over the previous state-of-the-art algorithms in terms of PESQ, STOI and SI-SNR.
- Open Access
- 28 July 2023
- Published by Qeios Ltd
Review of: "Solving the naming ambiguity of auditory localization mechanisms: HRTF & HRSL, and ILD, ITD, IPD"
This paper re-invented various concepts with a profound understanding. I recognize the potential for enhancing its quality. However, before reaching the standard for publishing, I have some concerns: 1. The paper lacks proper formatting as the sect...
- 28 July 2023
- Published by Qeios Ltd
Open AccessReview of: "Solving the naming ambiguity of auditory localization mechanisms: HRTF & HRSL, and ILD, ITD, IPD"
This paper re-invented various concepts with a profound understanding. I recognize the potential for enhancing its quality. However, before reaching the standard for publishing, I have some concerns: 1. The paper lacks proper formatting as the sect...