Adversary Aware Continual Learning
Open Access
- 5 September 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 126108-126121
Abstract
Continual learning approaches are useful to help a model learn new information or new tasks sequentially, while also retaining the previously acquired information. However, such approaches are known to be extremely vulnerable to the adversarial backdoor attacks, where an intelligent adversary can introduce small amount of misinformation in the form of imperceptible backdoor pattern during training to cause deliberate forgetting of a specific task or class at test time. In this work, we propose a novel defensive framework to counter such an attack and use the attacker’s primary strength – hiding the backdoor pattern by making it imperceptible to humans – against itself. To do so we train the model to learn a random perceptible pattern as decoy data during training with the deliberate intent to overpower the attacker’s imperceptible pattern. We demonstrate the effectiveness of the proposed defensive mechanism through various commonly used replay-based (both generative and exact replay-based) continual learning algorithms using continual learning benchmark variants of CIFAR-10, CIFAR-100, and MNIST datasets. We show that the proposed defensive framework considerably improves the robustness of continual learning algorithms without having any knowledge of the attacker’s target task, attacker’s target class, shape, size, and location of the attacker’s backdoor pattern. Moreover, our defensive framework does not depend on the underlying continual learning algorithm and does not rely on detecting the attack samples (to subsequently remove them from further consideration), but instead attempts to correctly classify even the attack samples and thus ensuring robustness in continual learning models. We refer to the proposed defensive framework as adversary aware continual learning (AACL).Keywords
All Related Versions
This publication has 21 references indexed in Scilit:
- A Random Ensemble of Encrypted Vision Transformers for Adversarially Robust DefenseIEEE Access, 2024
- Improving transferability of 3D adversarial attacks with scale and shear transformationsInformation Sciences, 2024
- Natural Adversarial Examples2021
- Targeted Forgetting and False Memory Formation in Continual Learners through Adversarial Backdoor Attacks2020
- iTAML: An Incremental Task-Agnostic Meta-learning Approach2020
- Adversarial Attacks and Defenses in Deep LearningEngineering, 2020
- Backdoor Suppression in Neural Networks using Input Fuzzing and Majority VotingIEEE Design & Test, 2020
- Generative Models from the perspective of Continual Learning2019
- Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks2019
- Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks2018