Robustness Learning via Inference-Softmax Cross Entropy in Misaligned Distribution of Image
Open Access
- 10 October 2022
- journal article
- research article
- Published by MDPI AG in Mathematics
- Vol. 10 (19) , 3716
Abstract
Adversarial examples easily mislead vision systems based on deep neural networks (DNNs) trained with softmax cross entropy (SCE) loss. The vulnerability of DNN comes from the fact that SCE drives DNNs to fit on the training examples, whereas the resultant feature distributions between the training and adversarial examples are unfortunately misaligned. Several state-of-the-art methods start from improving the inter-class separability of training examples by modifying loss functions, where we argue that the adversarial examples are ignored, thus resulting in a limited robustness to adversarial attacks. In this paper, we exploited the inference region, which inspired us to apply margin-like inference information to SCE, resulting in a novel inference-softmax cross entropy (I-SCE) loss, which is intuitively appealing and interpretable. The inference information guarantees that it is difficult for neural networks to cross the decision boundary under an adversarial attack, and guarantees both the inter-class separability and the improved generalization to adversarial examples, which was further demonstrated and proved under the min-max framework. Extensive experiments show that the DNN models trained with the proposed I-SCE loss achieve a superior performance and robustness over the state-of-the-arts under different prevalent adversarial attacks; for example, the accuracy of I-SCE is higher than SCE under the attack on the MNIST dataset. These experiments also show that the inference region can effectively solve the misaligned distribution.
Keywords
Funding Information
- National Natural Science Foundation of China (62162067, 62101480, 202005AC160007, 202001BB050076)
- Yunnan Province Science Foundation (62162067, 62101480, 202005AC160007, 202001BB050076)
This publication has 17 references indexed in Scilit:
- FDA: Feature Disruptive Attack2019
- Confidence Regularized Self-Training2019
- Feature Denoising for Improving Adversarial Robustness2019
- ArcFace: Additive Angular Margin Loss for Deep Face Recognition2019
- Rethinking Feature Distribution for Loss Functions in Image Classification2018
- Audio Adversarial Examples: Targeted Attacks on Speech-to-Text2018
- DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks2016
- Identity Mappings in Deep Residual Networks2016
- A Discriminative Feature Learning Approach for Deep Face Recognition2016
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998