Reachability Analysis of Sigmoidal Neural Networks
Open Access
- 13 May 2025
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Embedded Computing Systems
- Vol. 24 (3) , 1-25
Abstract
This paper extends the star set reachability approach to verify the robustness of feed-forward neural networks (FNNs) with sigmoidal activation functions such as Sigmoid and TanH. The main drawbacks of the star set approach in Sigmoid/TanH FNN verification are scalability, feasibility, and optimality issues in some cases due to the linear programming solver usage. We overcome this challenge by proposing a relaxed star (RStar) with symbolic intervals, which allows the usage of the back-substitution technique in DeepPoly to find bounds when overapproximating activation functions while maintaining the valuable features of a star set. RStar can overapproximate a sigmoidal activation function using four linear constraints (RStar4) or two linear constraints (RStar2), or only the output bounds (RStar0). We implement our RStar reachability algorithms in NNV and compare them to DeepPoly via robustness verification of image classification DNNs benchmarks. The experimental results show that the original star approach (i.e., no relaxation) is the least conservative of all methods yet the slowest. RStar4 is computationally much faster than the original star method and is the second least conservative approach. It certifies up to 40% more images against adversarial attacks than DeepPoly and on average 51 times faster than the star set. Last but not least, RStar0 is the most conservative method, which could only verify two cases for the CIFAR10 small Sigmoid network, δ = 0.014. However, it is the fastest method that can verify neural networks up to 3528 times faster than the star set and up to 46 times faster than DeepPoly in our evaluation.Keywords
Funding Information
- National Science Foundation (2220418, 2331937, and 2245853)
This publication has 16 references indexed in Scilit:
- Provably Tightest Linear Approximation for Robustness Verification of Sigmoid-like Neural Networks2022
- Verification of Sigmoidal Artificial Neural Networks using iSATElectronic Proceedings in Theoretical Computer Science, 2022
- Scalable Polyhedral Verification of Recurrent Neural Networks2021
- Reachability analysis of deep ReLU neural networks using facet-vertex incidence2021
- Tightening Robustness Verification of Convolutional Neural Networks with Fine-Grained Linear ApproximationProceedings of the AAAI Conference on Artificial Intelligence, 2021
- Verifying the Safety of Autonomous Systems with Neural Network ControllersACM Transactions on Embedded Computing Systems, 2020
- Verifying Recurrent Neural Networks Using Invariant Inference2020
- The Marabou Framework for Verification and Analysis of Deep Neural Networks2019
- Robustness Verification of Classification Deep Neural Networks via Linear Programming2019
- Formal Verification of Piece-Wise Linear Feed-Forward Neural Networks2017