Chenlong Gao
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Chenlong Gao has a total of 17 co-authors. Here are the top co-authors by number of shared publications:
- Yiqiang Chen
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
5 shared publications - Qian Chen
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
4 shared publications - Xinlong Jiang
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
4 shared publications - Bingjie Yan
School of Optometry, The Hong Kong Polytechnic University, Kowloon, Hong Kong
3 shared publications - Teng Zhang
Beijing Key Laboratory of Mobile Computing and Pervasive Device, Institute of Computing Technology, Chinese Academy of Sciences & University of Chinese Academy of Sciences, Beijing, China
3 shared publications - Wuliang Huang
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
3 shared publications
Latest Publications
- Top Cited PapersOpen Access
- 24 November 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Intelligent Systems and Technology
- Vol. 16 (6) , 1-27
Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application
Large Language Models (LLMs) have showcased exceptional capabilities in various domains, attracting significant interest from both academia and industry. Despite their impressive performance, the substantial size and computational demands of LLMs pose considerable challenges for practical deployment, particularly in environments with limited resources. The endeavor to compress language models while maintaining their accuracy has become a focal point of research. Among the various methods, knowledge distillation has emerged as an effective technique to enhance inference speed without greatly compromising performance. This article presents a thorough survey from three aspects: method, evaluation, and application, exploring knowledge distillation techniques tailored specifically for LLMs. Specifically, we divide the methods into white-box KD and black-box KD to better illustrate their differences. Furthermore, we also explored the evaluation tasks and distillation effects between different distillation methods and proposed directions for future research. Through in-depth understanding of the latest advancements and practical applications, this survey provides valuable resources for researchers, paving the way for sustained progress in this field.
- 24 November 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Intelligent Systems and Technology
- Vol. 16 (6) , 1-27
Top Cited PapersOpen AccessSurvey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application
Large Language Models (LLMs) have showcased exceptional capabilities in various domains, attracting significant interest from both academia and industry. Despite their impressive performance, the substantial size and computational demands of LLMs pose considerable challenges for practical deployment, particularly in environments with limited resources. The endeavor to compress language models while maintaining their accuracy has become a focal point of research. Among the various methods, knowledge distillation has emerged as an effective technique to enhance inference speed without greatly compromising performance. This article presents a thorough survey from three aspects: method, evaluation, and application, exploring knowledge distillation techniques tailored specifically for LLMs. Specifically, we divide the methods into white-box KD and black-box KD to better illustrate their differences. Furthermore, we also explored the evaluation tasks and distillation effects between different distillation methods and proposed directions for future research. Through in-depth understanding of the latest advancements and practical applications, this survey provides valuable resources for researchers, paving the way for sustained progress in this field.
- Open Access
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 2272-2281
Correlation-Driven Multi-Modality Graph Decomposition for Cross-Subject Emotion Recognition
Multi-modality physiological signal-based emotion recognition has attracted increasing attention as its capacity to capture human affective states comprehensively. Due to multi-modality heterogeneity and cross-subject divergence, practical applications struggle with generalizing models across individuals. Effectively addressing both issues requires mitigating the gap between multimodal signals while acquiring generalizable representations across subjects. However, existing approaches often handle these dual challenges separately, resulting in suboptimal generalization. This study introduces a novel framework, termed Correlation-Driven Multi-Modality Graph Decomposition (CMMGD). The proposed CMMGD initially captures adaptive cross-modal correlations. It connects each unimodal graph to a multimodal mixed graph. To simultaneously address the dual challenges, it incorporates a correlation-driven graph decomposition module that decomposes the mixed graph into concordant and discrepant subgraphs based on the correlations. The decomposed concordant subgraph encompasses consistently activated features across modalities and subjects during emotion elicitation, unveiling a generalizable subspace. Additionally, we design a Multi-Modality Graph Regularized Transformer (MGRT) backbone specifically tailored for multimodal physiological signals. The MGRT can alleviate the over-smoothing issue and mitigate over-reliance on any single modality. Extensive experiments demonstrate that CMMGD outperforms the state-of-the-art methods by 1.79% and 2.65% on DEAP and MAHNOB-HCI datasets, respectively, under the leave-one-subject-out cross-validation strategy.
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 2272-2281
Open AccessCorrelation-Driven Multi-Modality Graph Decomposition for Cross-Subject Emotion Recognition
Multi-modality physiological signal-based emotion recognition has attracted increasing attention as its capacity to capture human affective states comprehensively. Due to multi-modality heterogeneity and cross-subject divergence, practical applications struggle with generalizing models across individuals. Effectively addressing both issues requires mitigating the gap between multimodal signals while acquiring generalizable representations across subjects. However, existing approaches often handle these dual challenges separately, resulting in suboptimal generalization. This study introduces a novel framework, termed Correlation-Driven Multi-Modality Graph Decomposition (CMMGD). The proposed CMMGD initially captures adaptive cross-modal correlations. It connects each unimodal graph to a multimodal mixed graph. To simultaneously address the dual challenges, it incorporates a correlation-driven graph decomposition module that decomposes the mixed graph into concordant and discrepant subgraphs based on the correlations. The decomposed concordant subgraph encompasses consistently activated features across modalities and subjects during emotion elicitation, unveiling a generalizable subspace. Additionally, we design a Multi-Modality Graph Regularized Transformer (MGRT) backbone specifically tailored for multimodal physiological signals. The MGRT can alleviate the over-smoothing issue and mitigate over-reliance on any single modality. Extensive experiments demonstrate that CMMGD outperforms the state-of-the-art methods by 1.79% and 2.65% on DEAP and MAHNOB-HCI datasets, respectively, under the leave-one-subject-out cross-validation strategy.
- Open Access
- 21 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 2775-2785
Buffalo: Biomedical Vision-Language Understanding with Cross-Modal Prototype and Federated Foundation Model Collaboration
Federated learning (FL) enables collaborative learning across multiple biomedical data silos with multimodal foundation models while preserving privacy. Due to the heterogeneity in data processing and collection methodologies across diverse medical institutions and the varying medical inspections patients undergo, modal heterogeneity exists in practical scenarios, where severe modal heterogeneity may even prevent model training. With privacy considerations, data transfer cannot be permitted, restricting knowledge exchange among different clients. To trickle these issues, we propose a cross-modal prototype imputation method for visual-language understanding (Buffalo) with only a slight increase in communication cost, which can improve the performance of fine-tuning general foundation models for downstream biomedical tasks. We conducted extensive experiments on medical report generation and biomedical visual question-answering tasks. The results demonstrate that Buffalo can fully utilize data from all clients to improve model generalization compared to other modal imputation methods in three modal heterogeneity scenarios, approaching or even surpassing the performance in the ideal scenario without missing modality.
- 21 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 2775-2785
Open AccessBuffalo: Biomedical Vision-Language Understanding with Cross-Modal Prototype and Federated Foundation Model Collaboration
Federated learning (FL) enables collaborative learning across multiple biomedical data silos with multimodal foundation models while preserving privacy. Due to the heterogeneity in data processing and collection methodologies across diverse medical institutions and the varying medical inspections patients undergo, modal heterogeneity exists in practical scenarios, where severe modal heterogeneity may even prevent model training. With privacy considerations, data transfer cannot be permitted, restricting knowledge exchange among different clients. To trickle these issues, we propose a cross-modal prototype imputation method for visual-language understanding (Buffalo) with only a slight increase in communication cost, which can improve the performance of fine-tuning general foundation models for downstream biomedical tasks. We conducted extensive experiments on medical report generation and biomedical visual question-answering tasks. The results demonstrate that Buffalo can fully utilize data from all clients to improve model generalization compared to other modal imputation methods in three modal heterogeneity scenarios, approaching or even surpassing the performance in the ideal scenario without missing modality.
- 28 August 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
MG-HetGNN: Metapath-Guided Heterogeneous Graph Neural Network with Semantic Aggregation for Medical Diagnosis
Deep learning-based medical diagnosis has become an essential research area with significant practical value and wide application prospects. However, modeling heterogeneous medical data, which contains diverse data types with different semantics, remains a challenge. Existing approaches mostly focus on single-modal data or fuse the data of each modality without considering the relationships between them. However, better results could be achieved when modeling heterogeneous data while considering the relationships between heterogeneous data. To address this issue, we propose a medical diagnosis method that formulates medical diagnosis as a link prediction task in heterogeneous graphs and develops a heterogeneous graph neural network MG-HetGNN. MG-HetGNN, which includes a metapath-based ego graph encoder and semantic-level feature fusion module, is a metapath- and attention-based heterogeneous graph neural network. Specifically, MG-HetGNN encodes the structural relationships under each semantics and aggregates semantic-level information to generate node embeddings. Experiments on public open datasets demonstrate that the MG-HetGNN has remarkable performance compared to the baselines.
- 28 August 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
MG-HetGNN: Metapath-Guided Heterogeneous Graph Neural Network with Semantic Aggregation for Medical Diagnosis
Deep learning-based medical diagnosis has become an essential research area with significant practical value and wide application prospects. However, modeling heterogeneous medical data, which contains diverse data types with different semantics, remains a challenge. Existing approaches mostly focus on single-modal data or fuse the data of each modality without considering the relationships between them. However, better results could be achieved when modeling heterogeneous data while considering the relationships between heterogeneous data. To address this issue, we propose a medical diagnosis method that formulates medical diagnosis as a link prediction task in heterogeneous graphs and develops a heterogeneous graph neural network MG-HetGNN. MG-HetGNN, which includes a metapath-based ego graph encoder and semantic-level feature fusion module, is a metapath- and attention-based heterogeneous graph neural network. Specifically, MG-HetGNN encodes the structural relationships under each semantics and aggregates semantic-level information to generate node embeddings. Experiments on public open datasets demonstrate that the MG-HetGNN has remarkable performance compared to the baselines.
- 28 August 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
Interactive Scene-driven Multi-stream Graph Neural Network for ADHD Diagnosis
Attention Deficit Hyperactivity Disorder (ADHD) is a prevalent childhood mental disorder that is mainly characterized by inattention, hyperactivity and impulsiveness. ADHD is not a self-limited disease, which means the symptoms may not resolve on their own as the child grows. Therefore, it is crucial to study the diagnosis methods and provide patients with early intervention treatment to control ADHD symptoms. At present, interactive diagnosis task is considered as a feasible quantitative evaluation method. In this paper, we present a novel approach for ADHD diagnosis utilizing a multi-stream graph-based model, which is based on the Schulte Grid task. Specifically, we propose an interactive scene-driven graph construction method, which builds scene graphs based on the spatio-temporal constraint relations to effectively model unstructured information in the interactive scene. To address the challenge of limited ADHD motion data, we employ both coarse-grained time-frequency domain statistical features and fine-grained local time series variation features for feature extraction. Moreover, we leverage lightweight multi-stream graph neural networks to integrate interactive scene information and motion sensor data to model the entire testing process, and then fuse them for ADHD subtype classification. We conducted comparison and ablation experiments on a real dataset to evaluate the effectiveness of our proposed model. Our results show that the proposed approach outperforms other baseline models and demonstrate the potential of our method to improve ADHD diagnosis.
- 28 August 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
Interactive Scene-driven Multi-stream Graph Neural Network for ADHD Diagnosis
Attention Deficit Hyperactivity Disorder (ADHD) is a prevalent childhood mental disorder that is mainly characterized by inattention, hyperactivity and impulsiveness. ADHD is not a self-limited disease, which means the symptoms may not resolve on their own as the child grows. Therefore, it is crucial to study the diagnosis methods and provide patients with early intervention treatment to control ADHD symptoms. At present, interactive diagnosis task is considered as a feasible quantitative evaluation method. In this paper, we present a novel approach for ADHD diagnosis utilizing a multi-stream graph-based model, which is based on the Schulte Grid task. Specifically, we propose an interactive scene-driven graph construction method, which builds scene graphs based on the spatio-temporal constraint relations to effectively model unstructured information in the interactive scene. To address the challenge of limited ADHD motion data, we employ both coarse-grained time-frequency domain statistical features and fine-grained local time series variation features for feature extraction. Moreover, we leverage lightweight multi-stream graph neural networks to integrate interactive scene information and motion sensor data to model the entire testing process, and then fuse them for ADHD subtype classification. We conducted comparison and ablation experiments on a real dataset to evaluate the effectiveness of our proposed model. Our results show that the proposed approach outperforms other baseline models and demonstrate the potential of our method to improve ADHD diagnosis.