Enhancing Task Performance in Continual Instruction Fine-tuning Through Format Uniformity
Open Access
- 10 July 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 2384-2389
Abstract
In recent advancements, large language models (LLMs) have demonstrated remarkable capabilities in diverse tasks, primarily through interactive question-answering with humans. This development marks significant progress towards artificial general intelligence (AGI). Despite their superior performance, LLMs often exhibit limitations when adapted to domain-specific tasks through instruction fine-tuning (IF). The primary challenge lies in the discrepancy between the data distribution in general and domain-specific contexts, leading to suboptimal accuracy in specialized tasks. To address this, continual instruction fine-tuning (CIF), particularly supervised fine-tuning (SFT), on targeted domain-specific instruction datasets is necessary. Our ablation study reveals that the structure of these instruction datasets critically influences CIF performance, with substantial data distributional shifts resulting in notable performance degradation. In this paper, we introduce a novel framework that enhances CIF by promoting format uniformity. We assess our approach using the Llama2 chat model across various domain-specific instruction datasets. The results demonstrate not only an improvement in task-specific performance under CIF but also a reduction in catastrophic forgetting (CF). This study contributes to the optimization of LLMs for domain-specific applications, highlighting the significance of data structure and distribution in CIF.Keywords
Funding Information
- Shanghai Municipal Natural Science Foundation (23ZR1425400)
- National Natural Science Foundation of China (62102241)
This publication has 22 references indexed in Scilit:
- Chain-of-LoRA: Enhancing the Instruction Fine-Tuning Performance of Low-Rank Adaptation on Diverse Instruction SetIEEE Signal Processing Letters, 2024
- Fine-Tuning Large Enterprise Language Models via Ontological Reasoning2023
- Generative Relevance Feedback with Large Language Models2023
- Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language ProcessingACM Computing Surveys, 2023
- Analyzing Pre-trained and Fine-tuned Language Models2023
- Large Language Models Meet Harry Potter: A Dataset for Aligning Dialogue Agents with Characters2023
- RainProof: An Umbrella to Shield Text Generator from Out-Of-Distribution Data2023
- DeepSpeed2020
- Overcoming catastrophic forgetting in neural networksProceedings of the National Academy of Sciences, 2017
- RACE: Large-scale ReAding Comprehension Dataset From Examinations2017