Out-of-Distribution Detection by Cross-Class Vicinity Distribution of In-Distribution Data
- 26 May 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 35 (10) , 13777-13788
Abstract
Deep neural networks for image classification only learn to map in-distribution inputs to their corresponding ground-truth labels in training without differentiating out-of-distribution samples from in-distribution ones. This results from the assumption that all samples are independent and identically distributed (IID) without distributional distinction. Therefore, a pretrained network learned from in-distribution samples treats out-of-distribution samples as in-distribution and makes high-confidence predictions on them in the test phase. To address this issue, we draw out-of-distribution samples from the vicinity distribution of training in-distribution samples for learning to reject the prediction on out-of-distribution inputs. A cross-class vicinity distribution is introduced by assuming that an out-of-distribution sample generated by mixing multiple in-distribution samples does not share the same classes of its constituents. We, thus, improve the discriminability of a pretrained network by finetuning it with out-of-distribution samples drawn from the cross-class vicinity distribution, where each out-of-distribution input corresponds to a complementary label. Experiments on various in-/out-of-distribution datasets show that the proposed method significantly outperforms the existing methods in improving the capacity of discriminating between in-and out-of-distribution samples.Keywords
All Related Versions
Funding Information
- Australian Research Council Discovery (DP190101079)
- Future Fellowship (FT190100734)
This publication has 28 references indexed in Scilit:
- New Approximation Algorithms for Weighted Maximin Dispersion Problem with Box or Ball ConstraintsJournal of Optimization Theory and Applications, 2021
- MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space2021
- Outlier exposure with confidence control for out-of-distribution detectionNeurocomputing, 2021
- Open Set Domain Adaptation: Theoretical Bound and AlgorithmIEEE Transactions on Neural Networks and Learning Systems, 2020
- Online Anomaly Detection With Bandwidth Optimized Hierarchical Kernel Density EstimatorsIEEE Transactions on Neural Networks and Learning Systems, 2020
- Multiway Monte Carlo Method for Linear SystemsSIAM Journal on Scientific Computing, 2019
- A Robust AUC Maximization Framework With Simultaneous Outlier Detection and Feature Selection for Positive-Unlabeled ClassificationIEEE Transactions on Neural Networks and Learning Systems, 2018
- Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised LearningIEEE Transactions on Pattern Analysis and Machine Intelligence, 2018
- Deep Residual Learning for Image Recognition2016
- Describing Textures in the Wild2014