Preprint
Abstract
We introduce Noisy Feature Mixup (NFM), an inexpensive yet effective method for data augmentation that combines the best of interpolation based training and noise injection schemes. Rather than training with convex combinations of pairs of examples and their labels, we use noise-perturbed convex combinations of pairs of data points in both input and feature space. This method includes mixup and manifold mixup as special cases, but it has additional advantages, including better smoothing of decision boundaries and enabling improved model robustness. We provide theory to understand this as well as the implicit regularization effects of NFM. Our theory is supported by empirical results, demonstrating the advantage of NFM, as compared to mixup and manifold mixup. We show that residual networks and vision transformers trained with NFM have favorable trade-offs between predictive accuracy on clean data and robustness with respect to various types of data perturbation across a range of computer vision benchmark datasets.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- On the robustness of randomized classifiers to adversarial examplesMachine Learning, 2022
- Probabilistic robustness estimates for feed-forward neural networksNeural Networks, 2021
- Adversarial Examples in the Physical World2018
- Making machine learning robust against adversarial inputsCommunications of the ACM, 2018
- Robust Large Margin Deep Neural NetworksIEEE Transactions on Signal Processing, 2017
- The Algorithmic Foundations of Differential Privacy2013
- Deep, Big, Simple Neural Nets for Handwritten Digit RecognitionNeural Computation, 2010
- Training Invariant Support Vector MachinesMachine Learning, 2002
- The Effects of Adding Noise During Backpropagation Training on a Generalization PerformanceNeural Computation, 1996
- Training with Noise is Equivalent to Tikhonov RegularizationNeural Computation, 1995