Complexity vulnerability analysis using symbolic execution

Authors:
Publisher website
Google Scholar
Abstract
Summary: We describe techniques based on symbolic execution for finding software vulnerabilities that are due to algorithmic complexity. Such vulnerabilities allow an attacker to mount denial‐of‐service attacks to deny service to benign users or to otherwise disable a software system.The techniques use an efficient guided symbolic execution of a program to compute bounds on the worst‐case complexity (for increasing input sizes) and to generate test values that trigger the worst‐case behaviours. The resulting bounds are fitted to a function to obtain a prediction of the worst‐case program behaviour at any input size. Scalability is achieved by using path policies that guide the symbolic execution towards worst‐case paths. The policies are learned from the worst‐case results obtained with exhaustive exploration at small input sizes and are applied to guide exploration at larger input sizes, where unguided exhaustive exploration is not possible. To achieve precision in the analysis, the path policies take into account the history of choices made along the path when deciding which branch to execute next. Furthermore, the computation is contextpreserving, meaning that the decision for each branch depends on the history computed with respect to the enclosing method.We further report preliminary results on a complementary technique that uses machine learning for building the path policies that guide the search.The techniques are implemented in open‐source projects that build on the Symbolic Pathfinder tool for analysing Java programs. Experimental evaluation shows that the techniques can find vulnerabilities in complex Java programs and can outperform previous symbolic approaches.