StreamAdapter: Efficient Test Time Adaptation from Contextual Streams
Preprint
- 29 November 2024
- preprint
- Published by Qeios Ltd in Qeios Preprints
Abstract
In-context learning (ICL) allows large language models (LLMs) to adapt to new tasks directly from the given demonstrations without requiring gradient updates. While recent advances have expanded context windows to accommodate more demonstrations, this approach increases inference costs without necessarily improving performance. To mitigate these issues, We propose StreamAdapter, a novel approach that directly updates model parameters from context at test time, eliminating the need for explicit in-context demonstrations. StreamAdapter employs context mapping and weight absorption mechanisms to dynamically transform ICL demonstrations into parameter updates with minimal additional parameters. By reducing reliance on numerous in-context examples, StreamAdapter significantly reduce inference costs and allows for efficient inference with constant time complexity, regardless of demonstration count. Extensive experiments across diverse tasks and model architectures demonstrate that StreamAdapter achieves comparable or superior adaptation capability to ICL while requiring significantly fewer demonstrations. The superior task adaptation and context encoding capabilities of StreamAdapter on both language understanding and generation tasks provides a new perspective for adapting LLMs at test time using context, allowing for more efficient adaptation across scenarios and more cost-effective inference.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- What Do Language Models Learn in Context? The Structured Task Hypothesis.2024
- In-Context Learning Creates Task Vectors2023
- SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities2023
- Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers2023
- Wordcraft: Story Writing With Large Language Models2022
- Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning2021
- PIQA: Reasoning about Physical Commonsense in Natural LanguageProceedings of the AAAI Conference on Artificial Intelligence, 2020
- Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank2013
- The third PASCAL recognizing textual entailment challenge2007
- Long Short-Term MemoryNeural Computation, 1997