CUDA: Convolution-Based Unlearnable Datasets
Authors:
- 1 June 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3862-3871
Abstract
Large-scale training of modern deep learning models heavily relies on publicly available data on the web. This potentially unauthorized usage of online data leads to concerns regarding data privacy. Recent works aim to make unlearnable data for deep learning models by adding small, specially designed noises to tackle this issue. However, these methods are vulnerable to adversarial training (AT) and/or are computationally heavy. In this work, we propose a novel, model-free, Convolution-based Unlearnable DAtaset (CUDA) generation technique. CUDA is generated using controlled class-wise convolutions with filters that are randomly generated via a private key. CUDA encourages the network to learn the relation between filters and labels rather than informative features for classifying the clean data. We develop some theoretical analysis demonstrating that CUDA can successfully poison Gaussian mixture data by reducing the clean data performance of the optimal Bayes classifier. We also empirically demonstrate the effectiveness of CUDA with various datasets (CIFAR-10, CIFAR-100, ImageNet-100, and Tiny-ImageNet), and architectures (ResNet-18, VGG-16, Wide ResNet-34-10, DenseNet-121, DeIT, EfficientNetV2-S, and MobileNetV2). Our experiments show that CUDA is robust to various data augmentations and training approaches such as smoothing, AT with different budgets, transfer learning, and fine-tuning. For instance, training a ResNet-18 on ImageNet-100 CUDA achieves only 8.96%, 40.08%, and 20.58% clean test accuracies with empirical risk minimization (ERM), L ∞ AT, and L2 AT, respectively. Here, ERM on the clean training data achieves a clean test accuracy of 80.66%. CUDA exhibits unlearnability effect with ERM even when only a fraction of the training dataset is perturbed. Furthermore, we also show that CUDA is robust to adaptive defenses designed specifically to break it.Keywords
All Related Versions
Funding Information
- NSF (1942230,HR001119S0026)
- ONR YIP (N00014-22-1-2271)
- NIST (60NANB20D134)
This publication has 9 references indexed in Scilit:
- Availability Attacks Create Shortcuts2022
- VirTex: Learning Visual Representations from Textual Annotations2021
- CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features2019
- A New Backdoor Attack in CNNS by Training Set Corruption Without Label Poisoning2019
- A survey on Image Data Augmentation for Deep LearningJournal of Big Data, 2019
- Adversarial Examples in the Physical World2018
- MobileNetV2: Inverted Residuals and Linear Bottlenecks2018
- Towards Evaluating the Robustness of Neural Networks2017
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015