Continual Learning for Visual Search with Backward Consistent Feature Embedding
Authors:
- 1 June 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 16681-16690
Abstract
In visual search, the gallery set could be incrementally growing and added to the database in practice. However, existing methods rely on the model trained on the entire dataset, ignoring the continual updating of the model. Besides, as the model updates, the new model must re-extract features for the entire gallery set to maintain compatible feature space, imposing a high computational cost for a large gallery set. To address the issues of long-term visual search, we introduce a continual learning (CL) approach that can handle the incrementally growing gallery set with backward embedding consistency. We enforce the losses of inter-session data coherence, neighbor-session model coherence, and intra-session discrimination to conduct a continual learner. In addition to the disjoint setup, our CL solution also tackles the situation of increasingly adding new classes for the blurry boundary without assuming all categories known in the beginning and during model update. To our knowledge, this is the first CL method both tackling the issue of backward-consistent feature embedding and allowing novel classes to occur in the new sessions. Extensive experiments on various benchmarks show the efficacy of our approach under a wide range of setups 1 1 Code: https://github.com/ivclab/CVS.Keywords
All Related Versions
This publication has 44 references indexed in Scilit:
- Cross-batch Reference Learning for Deep Classification and Retrieval2016
- Deep Residual Learning for Image Recognition2016
- Deep Metric Learning via Lifted Structured Feature Embedding2016
- Deep learning of binary hash codes for fast image retrieval2015
- FaceNet: A unified embedding for face recognition and clustering2015
- Deep Learning for Content-Based Image Retrieval2014
- Product Quantization for Nearest Neighbor SearchIEEE Transactions on Pattern Analysis and Machine Intelligence, 2010
- Video Google: a text retrieval approach to object matching in videos2003
- Shape matching and object recognition using shape contextsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2002
- Texture features for browsing and retrieval of image dataIEEE Transactions on Pattern Analysis and Machine Intelligence, 1996