Certified Invertibility in Neural Networks via Mixed-Integer Programming
Authors:
- Tianqi CuiTCTianqi Cui
- Thomas BertalanTBThomas Bertalan
- George J. PappasGPGeorge J. Pappas
- Manfred MorariMMManfred Morari
- Ioannis G. KevrekidisIKIoannis G. Kevrekidis
- Mahyar FazlyabMFMahyar Fazlyab
Preprint
Abstract
Neural networks are known to be vulnerable to adversarial attacks, which are small, imperceptible perturbations that can significantly alter the network's output. Conversely, there may exist large, meaningful perturbations that do not affect the network's decision (excessive invariance). In our research, we investigate this latter phenomenon in two contexts: (a) discrete-time dynamical system identification, and (b) the calibration of a neural network's output to that of another network. We examine noninvertibility through the lens of mathematical optimization, where the global solution measures the ``safety" of the network predictions by their distance from the non-invertibility boundary. We formulate mixed-integer programs (MIPs) for ReLU networks and $L_p$ norms ($p=1,2,\infty$) that apply to neural network approximators of dynamical systems. We also discuss how our findings can be useful for invertibility certification in transformations between neural networks, e.g. between different levels of network pruning.
Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- BayesFlow: Learning Complex Stochastic Models With Invertible Neural NetworksIEEE Transactions on Neural Networks and Learning Systems, 2020
- PolyNet: A Pursuit of Structural Diversity in Very Deep Networks2017
- A Proposal on Machine Learning via Dynamical SystemsCommunications in Mathematics and Statistics, 2017
- Deep Residual Learning for Image Recognition2016
- Noninvertibility in neural networks2002
- Noninvertibility and resonance in discrete-time neural networks for time-series processingPhysics Letters A, 1998
- On Some Properties of Invariant Sets of Two-Dimensional Noninvertible MapsInternational Journal of Bifurcation and Chaos, 1997
- Noninvertibility and the structure of basins of attraction in a model adaptive control systemJournal of Nonlinear Science, 1991
- Detecting strange attractors in turbulence1981
- Some further studies of nonlinear oscillations in chemical systemsThe Journal of Chemical Physics, 1973