Trust-Region Adaptive Frequency for Online Continual Learning
Open Access
- 18 April 2023
- journal article
- research article
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 131 (7) , 1825-1839
Abstract
In the paradigm of online continual learning, one neural network is exposed to a sequence of tasks, where the data arrive in an online fashion and previously seen data are not accessible. Such online fashion causes insufficient learning and severe forgetting on past tasks issues, preventing a good stability-plasticity trade-off, where ideally the network is expected to have high plasticity to adapt to new tasks well and have the stability to prevent forgetting on old tasks simultaneously. To solve these issues, we propose a trust-region adaptive frequency approach, which alternates between standard-process and intra-process updates. Specifically, the standard-process replays data stored in a coreset and interleaves the data with current data, and the intra-process updates the network parameters based on the coreset. Furthermore, to improve the unsatisfactory performance stemming from online fashion, the frequency of the intra-process is adjusted based on a trust region, which is measured by the confidence score of current data. During the intra-process, we distill the dark knowledge to retain useful learned knowledge. Moreover, to store more representative data in the coreset, a confidence-based coreset selection is presented in an online manner. The experimental results on standard benchmarks show that the proposed method significantly outperforms state-of-art continual learning algorithms.Keywords
Funding Information
- University of Sydney
This publication has 37 references indexed in Scilit:
- Overcoming catastrophic forgetting in neural networksProceedings of the National Academy of Sciences, 2017
- Deep Residual Learning for Image Recognition2016
- The stability-plasticity dilemma: investigating the continuum from catastrophic forgetting to age-limited learning effectsFrontiers in Psychology, 2013
- Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic ProgrammingSIAM Journal on Optimization, 2013
- Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical resultsMathematical Programming, 2009
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions.Psychological Review, 1990
- Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem1989
- Random sampling with a reservoirACM Transactions on Mathematical Software, 1985
- How Does a Brain Build a Cognitive Code?1982