Wasserstein-Robust Training for One-Hidden-Layer ReLU Networks with Distributional Guarantees
Open Access
- 27 May 2026
- journal article
- research article
- Published by MDPI AG in Electronics
- Vol. 15 (11) , 2316
Abstract
Neural networks are vulnerable to adversarial perturbations, which motivates training procedures with formal robustness guarantees. In this paper, we study one-hidden-layer ReLU networks from the perspective of Wasserstein distributional robustness. Leveraging the network structure, we derive an upper bound for the intractable robust surrogate in the form of a tractable regularized empirical risk objective whose regularizer is computed through a low-rank optimization problem based on Burer–Monteiro factorization. This reformulation yields a distributional robustness certificate on the worst-case expected loss over a Wasserstein ball. The upper bound construction and distributional certificate are developed for the shallow fixed-output multiclass formulation, while the optimization analysis focuses on a binary specialization with margin loss and exact linear separability. We also analyze a modified stochastic gradient descent scheme for the resulting regularized problem in this binary linearly separable setting, and we establish a corresponding generalization bound. The experiments validate the proposed surrogate and training procedure on binary MNIST and CIFAR-10 tasks, and we added a 10-class MNIST experiment to further check the multiclass trainability of the surrogate.Keywords
Funding Information
- Department of Education of Guangdong Province (2023ZDZX1083)
This publication has 22 references indexed in Scilit:
- Cryptanalysis and Improvement of a Video Cryptosystem via Chaos and S-BoxACM Transactions on Multimedia Computing, Communications, and Applications, 2026
- PRG4CNN: A Probabilistic Model Checking-Driven Robustness Guarantee Framework for CNNsEntropy, 2025
- A Parallel Optimization Method for Robustness Verification of Deep Neural NetworksMathematics, 2024
- Robustness and Transferability of Adversarial Attacks on Different Image Classification Neural NetworksElectronics, 2024
- Adversarial Robustness with Partial IsometryEntropy, 2024
- Statistics of Robust Optimization: A Generalized Empirical Likelihood ApproachMathematics of Operations Research, 2021
- AutoZOOM: Autoencoder-Based Zeroth Order Optimization Method for Attacking Black-Box Neural NetworksProceedings of the AAAI Conference on Artificial Intelligence, 2019
- Robust Solutions of Optimization Problems Affected by Uncertain ProbabilitiesManagement Science, 2013
- A nonlinear programming algorithm for solving semidefinite programs via low-rank factorizationMathematical Programming, 2003
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998