Abstract
While large language models (LLMs) exhibit impressive general capabilities, their performance on domainspecific tasks often requires fine-tuning with private data that cannot be shared due to privacy constraints. Directly deploying LLMs on resource-constrained clients for local fine-tuning is impractical due to their significant computation and communication costs. In addition, pre-trained LLMs are valuable intellectual property, and model owners are reluctant to distribute full model weights. To address these challenges, we propose CoT-LM, a communication-efficient, computation-light, and data-free framework for collaborative fine-tuning between small (SLMs) and large language models (LLMs). In CoT-LM, clients fine-tune lightweight SLMs locally without uploading models or private data. These SLMs provide task-specific feedback to guide server-side LLM enhancement via an efficient communication protocol that exchanges only lightweight synthetic data and feedback. The framework supports both synchronous and asynchronous collaboration and enables mutual enhancement: the LLM improves its task-specific capabilities, while clients benefit from refined synthetic data or distilled knowledge. Extensive experiments demonstrate that CoT-LM significantly boosts natural language understanding (NLU, up to 18.3% for LLMs and 8.0% for SLMs) and natural language generation (NLG, up to 31.7% for LLMs) performance across diverse tasks while preserving data privacy, model intellectual property, and generalization capabilities, achieving significant reductions in computation and communication overhead.
Funding Information
  • National Natural Science Foundation of China (62402267)
  • National Natural Science Foundation of China (62432004)
  • National Natural Science Foundation of China (62572496)
  • Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Education of China (JYB2025XDXM122)
  • China Postdoctoral Science Foundation (BX20240177)
  • China Postdoctoral Science Foundation (2025M771499)
  • Natural Science Foundation of Guangdong Province (2026A1515011265)
  • Shenzhen Science and Technology Program (JCYJ20250604175500001)
  • Young Elite Scientist Sponsorship Program by China Association for Science and Technology (ZB2025-218)
  • Breakthrough Institute

This publication has 39 references indexed in Scilit: