Chenxu Wang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Chenxu Wang has a total of 3 co-authors. Here are the top co-authors by number of shared publications:
- Liping Jing
Beijing Jiaotong University, Beijing, China
2 shared publications - Yilin Lyu
School of Computer Science and Technology Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence Beijing Jiaotong University State Key Laboratory of Advanced Rail Autonomous Operation
2 shared publications - Zicheng Sun
School of Computer Science and Technology Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence Beijing Jiaotong University State Key Laboratory of Advanced Rail Autonomous Operation
2 shared publications
Latest Publications
- Open Access
Continual Gradient Low-Rank Projection Fine-Tuning for LLMs
Continual fine-tuning of Large Language Models (LLMs) is hampered by the trade-off between efficiency and expressiveness. Low-Rank Adaptation (LoRA) offers efficiency but constrains the model's ability to learn new tasks and transfer knowledge due to its low-rank nature and reliance on explicit parameter constraints. We propose GORP (Gradient LOw Rank Projection) for Continual Learning, a novel training strategy that overcomes these limitations by synergistically combining full and low-rank parameters and jointly updating within a unified low-rank gradient subspace. GORP expands the optimization space while preserving efficiency and mitigating catastrophic forgetting. Extensive experiments on continual learning benchmarks demonstrate GORP's superior performance compared to existing state-of-the-art approaches. Code is available at https://github.com/Wcxwcxw/GORP.
Open AccessContinual Gradient Low-Rank Projection Fine-Tuning for LLMs
Continual fine-tuning of Large Language Models (LLMs) is hampered by the trade-off between efficiency and expressiveness. Low-Rank Adaptation (LoRA) offers efficiency but constrains the model's ability to learn new tasks and transfer knowledge due to its low-rank nature and reliance on explicit parameter constraints. We propose GORP (Gradient LOw Rank Projection) for Continual Learning, a novel training strategy that overcomes these limitations by synergistically combining full and low-rank parameters and jointly updating within a unified low-rank gradient subspace. GORP expands the optimization space while preserving efficiency and mitigating catastrophic forgetting. Extensive experiments on continual learning benchmarks demonstrate GORP's superior performance compared to existing state-of-the-art approaches. Code is available at https://github.com/Wcxwcxw/GORP.
- Open Access
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 14815-14829
Continual Gradient Low-Rank Projection Fine-Tuning for LLMs
Chenxu Wang, Yilin Lyu, Zicheng Sun, Liping Jing. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 14815-14829
Open AccessContinual Gradient Low-Rank Projection Fine-Tuning for LLMs
Chenxu Wang, Yilin Lyu, Zicheng Sun, Liping Jing. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.