Adaptive Cascading Network for Continual Test-Time Adaptation
Open Access
- 21 October 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1763-1773
Abstract
We study the problem of continual test-time adaption where the goal is to adapt a source pre-trained model to a sequence of unlabelled target domains at test time. Existing methods on test-time training suffer from several limitations: (1) Mismatch between the feature extractor and classifier; (2) Interference between the main and self-supervised tasks; (3) Lack of the ability to quickly adapt to the current distribution. In light of these challenges, we propose a cascading paradigm that simultaneously updates the feature extractor and classifier at test time, mitigating the mismatch between them and enabling long-term model adaptation. The pre-training of our model is structured within a meta-learning framework, thereby minimizing the interference between the main and self-supervised tasks and encouraging fast adaptation in the presence of limited unlabelled data. Additionally, we introduce innovative evaluation metrics, average accuracy and forward transfer, to effectively measure the model's adaptation capabilities in dynamic, real-world scenarios. Extensive experiments and ablation studies demonstrate the superiority of our approach in a range of tasks including image classification, text classification, and speech recognition. Our code is publicly available at https://github.com/Nyquixt/CascadeTTA.Keywords
All Related Versions
Funding Information
- National Science Foundation through the Faculty Early Career Development Program (NSF CAREER) Award (NSF-IIS-2340074)
- Department of Defense under the Defense Established Program to Stimulate Competitive Research (DoD DEPSCoR) Award
This publication has 15 references indexed in Scilit:
- Are Data-Driven Explanations Robust Against Out-of-Distribution Data?2023
- Lifelong Unsupervised Domain Adaptive Person Re-identification with Coordinated Anti-forgetting and Adaptation2022
- Continual Adaptation of Visual Representations via Domain Randomization and Meta-learning2021
- Uncertainty-guided Model Generalization to Unseen Domains2021
- Learning to Learn Single Domain Generalization2020
- Adaptive Batch Normalization for practical domain adaptationPattern Recognition, 2018
- PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning2018
- Expert Gate: Lifelong Learning with a Network of Experts2017
- iCaRL: Incremental Classifier and Representation Learning2017
- Overcoming catastrophic forgetting in neural networksProceedings of the National Academy of Sciences, 2017