Leijun Cheng
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Leijun Cheng has a total of 10 co-authors. Here are the top co-authors by number of shared publications:
- Xiaoyu Tan
National University of Singapore
5 shared publications - Xihe Qiu
Shanghai University of Engineering Science
5 shared publications - Wei Chu
INF Technology (Shanghai) Co., Ltd., Shanghai, China
3 shared publications - Yinghui Xu
Fudan University, Shanghai, China
3 shared publications - Yuan Cheng
AI3 Institute, Fudan University, Shanghai, China
3 shared publications - Yuan Qi
Shanghai Academy of Artificial Intelligence for Science, Shanghai, China
3 shared publications
Latest Publications
- 10 November 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Artificial Intelligence
- Vol. 7 (5) , 2920-2930
ArMA: Mitigating Catastrophic Forgetting using Attention-Regularized Model Averaging in Continual Fine-tuning Large Language Models
Recent advancements in continual fine-tuning have aimed to enhance the instruction-following capabilities of large language models (LLMs) within domain-specific contexts. However, these models often suffer from catastrophic forgetting, manifesting as a significant decline in performance on general domain tasks. This presents a substantial challenge for developers who seek to improve performance on a specific domain without compromising efficacy across previously established tasks. To mitigate the negative impact of fine-tuning on generalization and enable the model to adapt to new tasks while preserving its ability on general-domain tasks, we propose a novel framework, ARMA. This framework addresses catastrophic forgetting in the continual fine-tuning of LLMs through attention-regularized model averaging. Unlike the typical model average, which utilizes various metrics to measure and balance the general capabilities, our proposed ARMA framework employs a heuristic based on attention regularization to universally consider all general instruction tasks. This approach is predicated on the hypothesis that models exhibiting similar attention distributions over the input instructions will likely yield comparable outcomes. Our evaluation across multiple benchmarks demonstrates that attention-regularized model averaging balances the trade-offs between different task capabilities in the continual fine-tuning of LLMs and effectively reserves the general domain capabilities, which will potentially benefit complex industrial domain-specific applications.
- 10 November 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Artificial Intelligence
- Vol. 7 (5) , 2920-2930
ArMA: Mitigating Catastrophic Forgetting using Attention-Regularized Model Averaging in Continual Fine-tuning Large Language Models
Recent advancements in continual fine-tuning have aimed to enhance the instruction-following capabilities of large language models (LLMs) within domain-specific contexts. However, these models often suffer from catastrophic forgetting, manifesting as a significant decline in performance on general domain tasks. This presents a substantial challenge for developers who seek to improve performance on a specific domain without compromising efficacy across previously established tasks. To mitigate the negative impact of fine-tuning on generalization and enable the model to adapt to new tasks while preserving its ability on general-domain tasks, we propose a novel framework, ARMA. This framework addresses catastrophic forgetting in the continual fine-tuning of LLMs through attention-regularized model averaging. Unlike the typical model average, which utilizes various metrics to measure and balance the general capabilities, our proposed ARMA framework employs a heuristic based on attention regularization to universally consider all general instruction tasks. This approach is predicated on the hypothesis that models exhibiting similar attention distributions over the input instructions will likely yield comparable outcomes. Our evaluation across multiple benchmarks demonstrates that attention-regularized model averaging balances the trade-offs between different task capabilities in the continual fine-tuning of LLMs and effectively reserves the general domain capabilities, which will potentially benefit complex industrial domain-specific applications.
- Open Access
- 20 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 2584-2595
Struct-X: Enhancing the Reasoning Capabilities of Large Language Models in Structured Data Scenarios
- 20 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 2584-2595
Open AccessStruct-X: Enhancing the Reasoning Capabilities of Large Language Models in Structured Data Scenarios
- Open Access
- 24 August 2024
- Published by Association for Computing Machinery (ACM)
- p. 5762-5772
Enhancing Personalized Headline Generation via Offline Goal-conditioned Reinforcement Learning with Large Language Models
Recently, significant advancements have been made in Large Language Models (LLMs) through the implementation of various alignment techniques. These techniques enable LLMs to generate highly tailored content in response to diverse user instructions. Consequently, LLMs have the potential to serve as robust, customizable recommendation systems in the field of content recommendation. However, using LLMs with user individual information and online exploration remains a challenge, which are important perspectives in developing personalized news headline generation algorithms. In this paper, we propose a novel framework to generate personalized news headlines using LLMs with extensive online exploration. The proposed approach involves initially training an offline goal-conditioned policy using supervised learning. Subsequently, online exploration is employed to collect new data for the next training iteration. Results from simulations, experiments, and real-word scenario demonstrate that our framework achieves outstanding performance on established benchmarks and can effectively generate personalized headlines under different reward settings. By treating the LLM as a goal-conditioned agent, the model can perform online exploration by modifying the goals without frequently retraining the model. To the best of our knowledge, this work represents the first investigation into the capability of LLMs to generate customized news headlines with goal-conditioned reinforcement learning via supervised learning within LLMs.
- 24 August 2024
- Published by Association for Computing Machinery (ACM)
- p. 5762-5772
Open AccessEnhancing Personalized Headline Generation via Offline Goal-conditioned Reinforcement Learning with Large Language Models
Recently, significant advancements have been made in Large Language Models (LLMs) through the implementation of various alignment techniques. These techniques enable LLMs to generate highly tailored content in response to diverse user instructions. Consequently, LLMs have the potential to serve as robust, customizable recommendation systems in the field of content recommendation. However, using LLMs with user individual information and online exploration remains a challenge, which are important perspectives in developing personalized news headline generation algorithms. In this paper, we propose a novel framework to generate personalized news headlines using LLMs with extensive online exploration. The proposed approach involves initially training an offline goal-conditioned policy using supervised learning. Subsequently, online exploration is employed to collect new data for the next training iteration. Results from simulations, experiments, and real-word scenario demonstrate that our framework achieves outstanding performance on established benchmarks and can effectively generate personalized headlines under different reward settings. By treating the LLM as a goal-conditioned agent, the model can perform online exploration by modifying the goals without frequently retraining the model. To the best of our knowledge, this work represents the first investigation into the capability of LLMs to generate customized news headlines with goal-conditioned reinforcement learning via supervised learning within LLMs.
- Open Access
- 10 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 2384-2389
Enhancing Task Performance in Continual Instruction Fine-tuning Through Format Uniformity
In recent advancements, large language models (LLMs) have demonstrated remarkable capabilities in diverse tasks, primarily through interactive question-answering with humans. This development marks significant progress towards artificial general intelligence (AGI). Despite their superior performance, LLMs often exhibit limitations when adapted to domain-specific tasks through instruction fine-tuning (IF). The primary challenge lies in the discrepancy between the data distribution in general and domain-specific contexts, leading to suboptimal accuracy in specialized tasks. To address this, continual instruction fine-tuning (CIF), particularly supervised fine-tuning (SFT), on targeted domain-specific instruction datasets is necessary. Our ablation study reveals that the structure of these instruction datasets critically influences CIF performance, with substantial data distributional shifts resulting in notable performance degradation. In this paper, we introduce a novel framework that enhances CIF by promoting format uniformity. We assess our approach using the Llama2 chat model across various domain-specific instruction datasets. The results demonstrate not only an improvement in task-specific performance under CIF but also a reduction in catastrophic forgetting (CF). This study contributes to the optimization of LLMs for domain-specific applications, highlighting the significance of data structure and distribution in CIF.
- 10 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 2384-2389
Open AccessEnhancing Task Performance in Continual Instruction Fine-tuning Through Format Uniformity
In recent advancements, large language models (LLMs) have demonstrated remarkable capabilities in diverse tasks, primarily through interactive question-answering with humans. This development marks significant progress towards artificial general intelligence (AGI). Despite their superior performance, LLMs often exhibit limitations when adapted to domain-specific tasks through instruction fine-tuning (IF). The primary challenge lies in the discrepancy between the data distribution in general and domain-specific contexts, leading to suboptimal accuracy in specialized tasks. To address this, continual instruction fine-tuning (CIF), particularly supervised fine-tuning (SFT), on targeted domain-specific instruction datasets is necessary. Our ablation study reveals that the structure of these instruction datasets critically influences CIF performance, with substantial data distributional shifts resulting in notable performance degradation. In this paper, we introduce a novel framework that enhances CIF by promoting format uniformity. We assess our approach using the Llama2 chat model across various domain-specific instruction datasets. The results demonstrate not only an improvement in task-specific performance under CIF but also a reduction in catastrophic forgetting (CF). This study contributes to the optimization of LLMs for domain-specific applications, highlighting the significance of data structure and distribution in CIF.